.. _quickstart:

Quickstart
==========

You can't wait to start using Lightflow or have no time to follow the tutorial? No problem, just spend a few minutes with
this quickstart guide and you are on your way to using Lightflow.

This quickstart guide assumes that you have a redis database running on ``localhost`` and port ``6379``,
a MongoDB database running on ``localhost`` and port ``27017`` as well as Lightflow installed on your system. If you haven't
installed the database systems and Lightflow yet, no problem, just follow the :ref:`installation` guide.


Configuration and examples
--------------------------

Create an empty directory in your preferred location. We will use this directory in the following to store the configuration file and
the example workflows. Lightflow has no restrictions on where this directory should be located and what its name should be.

The first step is to create the global configuration file for Lightflow. This file contains, among other settings, the connection
information for redis and MongoDB. The quickest and easiest way to generate a default configuration file is to use the Lightflow command line interface.
Make sure you are located in the directory you created earlier and enter::

    $ lightflow config default .

This will create a configuration file called ``lightflow.cfg`` containing a default configuration. If you were running redis and MongoDB on
different hosts than ``localhost`` or the default port, edit the appropriate settings in the configuration file. You can find more
information about the configuration file in the section Configuration.

Lightflow ships with a number of examples that demonstrate various features of the system. We will copy these examples into a subfolder called ``examples``
inside your current directory. This will allow you to modify the examples as you see fit or use them as a starting point for your own workflows.
The command line interface offers a quick and easy way to copy the examples::

    $ lightflow config examples .

Now you will find a subfolder ``examples`` in your directory containing all example workflows. If you like, you can list all available example workflows
together with a short description, Make sure you are located in the folder containing the configuration file, then enter::

    $ lightflow workflow list


Start the workers
-----------------

Lightflow uses a worker based scheme. This means a workflow adds jobs onto a central queue from which a number of workers consume jobs and execute them.
In order for Lightflow to run a workflow, it needs at least one running worker (obviously). You can start a worker with::

    $ lightflow worker start

This will start a worker, which then waits for the first job to be added to the queue. You can start as many workers as you like, but for the quickstart
guide one worker is enough.

.. admonition:: A recommended setup for multiple workers 

   What is special about Lightflow, in comparison with other workflow systems, is that it also uses workers for running the workflow itself. This means, there
   is no central daemon and thus no single point of failure. Lightflow uses three queues for running a workflow. Two queues, labelled ``workflow`` and ``dag``, for
   managing the workflows and one queue, labelled ``task``, for executing the individual tasks of a workflow. A typical setup of workers would consist of one worker
   dealing with workflow related jobs, thus consuming jobs only from the ``workflow`` and ``dag`` queues, and one or more workers executing the actual tasks.

   You can use the ``-q`` argument in the command line interface in order to restrict the queues a worker consumes jobs from. For the recommended setup
   discussed above you would start one worker with::

       $ lightflow worker start -q workflow,dag
    
   and at least one more worker with::

       $ lightflow worker start -q task


Run a workflow
--------------

With at least one worker running, we are ready to run our first workflow. You can pick any example workflow you like and run it. In the following we
will run the most basic of all workflows, the ``simple`` workflow. You might need a second terminal in order to run the workflow as the first one
is occupied running our worker. In your second terminal enter::

    $ lightflow workflow start simple

This will send the workflow ``simple`` to the queue. Our worker will pick up the workflow and run it. The default logging level is very verbose so you
will see the worker print out a lot of information as it executes the workflow.


Where to go from here
---------------------

Congratulations, you have finished the quickstart guide. A good place to continue is to have a look at the documented example workflows. They are a great
starting point for exploring the features of Lightflow. Alternatively, head over to the tutorial section for a more structured introduction to Lightflow.