perkun can be used either directly, in command line, or as a library (libperkun).

Using perkun from the command line

In order to use perkun from the command line you will need a perkun specification written in the Perkun language, stored in a separate file. Take a look at the directory perkun-0.1.8/examples. You will find there some examples, like example2_dorban.perkun taken from the perkunwars.

Perkun specification consists of the following four sections followed by the commands:

  • values
  • variables
  • payoff
  • model

The simplest Perkun program is:


       # begin of the program
       values {}
       variables {}
       payoff {}
       model {}
       # end of the program

When you place the above text in a file (for example test.perkun) and run:


       $ perkun test.perkun

it will do nothing. That is correct. But if you remove any of the sections you will get an error. Also correct.

A valid identifier is any sequence of letters/digits beginning with a letter that is not a keyword.


       # This is a Perkun example code
       values
       {
            value false, true, do_nothing, switch;
       }
       variables
       {
            input variable alpha:{false, true};
            output variable beta:{do_nothing, switch};
       }
       payoff
       {
            set({alpha=>false},0.0);
            set({alpha=>true},100.0);
       }
       model
       {
            set({alpha=>false},{beta=>do_nothing},{alpha=>false},1.0);
            set({alpha=>false},{beta=>do_nothing},{alpha=>true},0.0);
            set({alpha=>true},{beta=>do_nothing},{alpha=>false},0.0);
            set({alpha=>true},{beta=>do_nothing},{alpha=>true},1.0);
            set({alpha=>false},{beta=>switch},{alpha=>false},0.0);
            set({alpha=>false},{beta=>switch},{alpha=>true},1.0);
            set({alpha=>true},{beta=>switch},{alpha=>false},1.0);
            set({alpha=>true},{beta=>switch},{alpha=>true},0.0);
       }
       loop(1);
       # end of example

The values section of the Perkun code contains the keyword "value" followed by the valid identifiers, separated by commas.

The values are used to denote the possible values of the variables. The Perkun variables is a complex topic and will be discussed later. You can print out all the values with the command


       cout << values << eol;

For example:


       values
       {
            value false, true, do_nothing, switch;
       }
       variables{}
       payoff{}
       model{}
       cout << values << eol;

The above Perkun code will print out:


       # values
       false
       true
       do_nothing
       switch

There are many other commands apart from "cout << values << eol;". They can all be placed directly after the model section.

One-line comments begin in Perkun with a hashtag "#".

Perkun allows three kinds of variables:

  • input variables
  • hidden variables
  • output variables
The hidden variables are essential for the machine learning. They represent the state variables of the world that are not directly visible to the agent. Yet they do affect the state of the world.

Let us forget for now the hidden variables though, and introduce a simple Perkun code with one input variable and one output variable:


       values
       {
            value false, true;
            value move, do_nothing;
       }
       variables
       {
            input variable what_I_can_see:{false, true};
            output variable action:{move, do_nothing};
       }
       payoff {}
       model {}

As you can see the variables section contains now two variables:

  • what_I_can_see
  • action

Unlike many other programming languages Perkun does not have the concept of a variable type. Instead each variable declaration is followed by a colon and a list of possible values in curly brackets, separated by commas. For example what_I_can_see may have value false or true.

Try using the command "cout << variables << eol;":


       values
       {
            value false, true;
            value move, do_nothing;
       }
       variables
       {
            input variable what_I_can_see:{false, true};
            output variable action:{move, do_nothing};
       }
       payoff {}
       model {}
       cout << variables << eol;

When you put the above text in a file and call Perkun it will respond:


       # variables
       input variable what_I_can_see:{false,true};
       output variable action:{move,do_nothing};

We know already how to declare Perkun values and variables. We know that the agent i.e. Perkun program can see the input variables and can control the output variables. The question is how to control the output variables. What do we want to achieve? What is "good" for the agent and what is not?

In order to define it we use the payoff section of the Perkun program.

Try the following code:


       values
       {
            value false, true;
            value move, do_nothing;
       }
       variables
       {
            input variable what_I_can_see:{false, true};
            output variable action:{move, do_nothing};
       }
       payoff
       {
            set({what_I_can_see=>false},0.0);
            set({what_I_can_see=>true},1.0);
       }
       model {}
       cout << payoff << eol;

It will print out the contents of the payoff section:


       # payoff
       what_I_can_see=false payoff 0
       what_I_can_see=true payoff 1

The payoff section contains the commands "set" with two arguments:

  • input state query
  • payoff value (a real number)

The query contains all input variables followed by "=>" and the corresponding value, separated by commas, enclosed in curly brackets. In short it is an expression:


       {INPUT_VARIABLE1=>VALUE,INPUT_VARIABLE2=>VALUE,...}

Note that the query must contain all the input variables. What does the payoff value mean? It is a value representing "how good" the payoff function is (for the respective combination of the input variable values). The higher the payoff value the "better" it is. In the example above the agent "likes" to see the value "true" on its input (variable what_I_can_see), while it dislikes the value "false", because for "true" the payoff value equals 1.0, while for "false" it equals 0.0.

What happens if you fail to specify payoff for some or all input variables values combinations? It will be zero. Try:


       values
       {
            value false, true;
            value move, do_nothing;
       }
       variables
       {
            input variable what_I_can_see:{false, true};
            output variable action:{move, do_nothing};
       }
       payoff {}
       model {}
       cout << payoff << eol;

After running the above code with perkun you will see zero values for both "false" and "true" of the input variable what_I_can_see.

WARNING! - Before the version 0.1.0 the payoff was random. Now it is set to zero.

Perkun hidden variables

We do not perceive directly everything that is happening in the world. Or to say it otherwise - in order to understand the world it is useful to imagine that some facts about it remain hidden. Likewise - in Perkun it is assumed that the world has a state described by the visible variables (so called input variables) and by the invisible variables (so called hidden variables). How to declare them in Perkun?


       values { value false, true; }
       variables
       {
       input variable a:{false, true}, b:{false, true};
       hidden variable c:{false, true}, d:{false, true};
       }
       payoff {}
       model {}
       cout << variables << eol;

The above Perkun code defines two input variables (a and b) and two hidden variables (c and d). All of them may have either value false or true. When you run it with perkun it will write out:


       # variables
       input variable a:{false,true};
       input variable b:{false,true};
       hidden variable c:{false,true};
       hidden variable d:{false,true};

But how can we use the hidden variables? What is it good for, to take into account the variables we do not perceive? Well, in order to understand this we need to learn how to use the model section of the Perkun program.

Perkun model

The time has come to learn the last part of the Perkun program: the model. We know already how to declare the variables (input, output or hidden ones) and how to define the payoff, i.e. tell Perkun which input variable values are good and which are not so good. In order to do its job Perkun needs to know how its decisions will affect the world, i.e. how the world state in the future depends on its state now and the Perkun's choices.

Imagine a world where we have only one variable - input variable alpha. Its value can be either false or true. Let us introduce an output variable beta which can control two actions:

  • do_nothing - alpha remains as it is now
  • switch - alpha switches to not alpha

Let us further assume that Perkun "likes" alpha=true, and "dislikes" alpha=false. For this simple world our Perkun program will look as follows:


       values
       {
            value false, true, do_nothing, switch;
       }
       variables
       {
            input variable alpha:{false, true};
            output variable beta:{do_nothing, switch};
       }
       payoff
       {
            set({alpha=>false},0.0);
            set({alpha=>true},100.0);
       }
       model
       {
            set({alpha=>false},{beta=>do_nothing},{alpha=>false},1.0);
            set({alpha=>false},{beta=>do_nothing},{alpha=>true},0.0);
            set({alpha=>true},{beta=>do_nothing},{alpha=>false},0.0);
            set({alpha=>true},{beta=>do_nothing},{alpha=>true},1.0);
            set({alpha=>false},{beta=>switch},{alpha=>false},0.0);
            set({alpha=>false},{beta=>switch},{alpha=>true},1.0);
            set({alpha=>true},{beta=>switch},{alpha=>false},1.0);
            set({alpha=>true},{beta=>switch},{alpha=>true},0.0);
       }
       cout << model << eol;

The model section is filled with the set instructions, but they have slightly different syntax than the set instructions we know from the payoff section. Each set instruction in the model section has syntax:


set(INITIAL_STATE_QUERY,ACTION_QUERY,TERMINAL_STATE_QUERY,VALUE);

The INITIAL_STATE_QUERY and TERMINAL_STATE_QUERY are regular queries containing all input and hidden variables.

The ACTION_QUERY is a regular query containing all output variables.

The VALUE is a probability value.

In short set(A,B,C,D) means "if A and you do B then the probability to jump to C is D".

When you put the above Perkun code in a file and execute it with perkun it will write down:


       # model
       # {beta=>do_nothing}
       set({alpha=>false },{beta=>do_nothing },{alpha=>false },1.00000);
       set({alpha=>false },{beta=>do_nothing },{alpha=>true },0.00000);

       # {beta=>switch}
       set({alpha=>false },{beta=>switch },{alpha=>false },0.00000);
       set({alpha=>false },{beta=>switch },{alpha=>true },1.00000);

       # {beta=>do_nothing}
       set({alpha=>true },{beta=>do_nothing },{alpha=>false },0.00000);
       set({alpha=>true },{beta=>do_nothing },{alpha=>true },1.00000);

       # {beta=>switch}
       set({alpha=>true },{beta=>switch },{alpha=>false },1.00000);
       set({alpha=>true },{beta=>switch },{alpha=>true },0.00000);

It will be done by the command "cout << model << eol;".

Note that the sum of the probabilities over the terminal state queries equals 1.0 for each initial state query and each action query.

Creating the Perkun models is difficult. The difficulty is of the quantitative nature - they tend to be very big. If you skip the set instructions in the model section the model will be filled with zeros (it used to be random before the version 0.1.0). Try running with perkun the following code:


       values { value false,true; }
       variables
       {
       input variable a:{false,true},b:{false,true},c:{false,true},
                           d:{false,true},e:{false,true};
       output variable f:{false,true},g:{false,true},h:{false,true};
       }
       payoff {}
       model {}
       cout << model << eol;

The model printed out will likely be bigger than 1MB of code. Remember, the models "explode" with the number of variables.

Congratulations, now you know everything what you have to tell Perkun. Values, variables, payoff, model. The time has come to see what Perkun can do for you given this information. It will attempt to maximize the expected value of the payoff function by appropriate choosing the actions. First you give it current values of the input variables, then it responds with the optimal action chosen (the values of the output variables). As you can see we will interact with Perkun and this interaction will involve both input and output.

The loop command

As you remember the four sections in the Perkun code may be followed by instructions, and now is the time to learn another one: loop. It takes one argument - an integer constant, which denotes how deep Perkun should look into the game tree. Try the following code:


       values
       {
            value false, true, do_nothing, switch;
       }
       variables
       {
            input variable alpha:{false, true};
            output variable beta:{do_nothing, switch};
       }
       payoff
       {
            set({alpha=>false},0.0);
            set({alpha=>true},100.0);
       }
       model
       {
            set({alpha=>false},{beta=>do_nothing},{alpha=>false},1.0);
            set({alpha=>false},{beta=>do_nothing},{alpha=>true},0.0);
            set({alpha=>true},{beta=>do_nothing},{alpha=>false},0.0);
            set({alpha=>true},{beta=>do_nothing},{alpha=>true},1.0);
            set({alpha=>false},{beta=>switch},{alpha=>false},0.0);
            set({alpha=>false},{beta=>switch},{alpha=>true},1.0);
            set({alpha=>true},{beta=>switch},{alpha=>false},1.0);
            set({alpha=>true},{beta=>switch},{alpha=>true},0.0);
       }
       loop(1);

Perkun will respond with the message and a prompt:


loop with depth 1
I expect the values of the variables: alpha
perkun>

Now it is your turn. You can enter one of the two values that the input variable alpha may have, i.e. false or true. Let us first try false:


perkun> false
belief:
alpha=false 1
optimal action:
beta=switch
perkun>

As the optimal action the beta=switch was chosen! Why? Because Perkun can see alpha=false, it does not like it (see the payoff) and knows that if it uses switch the alpha will change its value to not alpha. How does it know it? From the model. That is it, this is how Perkun works. OK. Let us continue. Let us now enter true:


perkun> true
belief:
alpha=true 1
optimal action:
beta=do_nothing
perkun>

Again Perkun chooses an optimal action, but this time it is beta=do_nothing! Why? Because it can see alpha=true, it likes it (see the payoff) and knows that do_nothing will keep the value of alpha unmodified. The last fact, again, is known from the model.

Just for fun let us lie to Perkun and enter now "false". This is impossible, because the model says that alpha after beta=do_nothing is unmodified. But let us do it:


perkun> false
result
alpha=false its probability was 0
exception caught
line 28 at T_INT_LITERAL: syntax error
error

What happened? Perkun has thrown an exception and stated that alpha=false was impossible! Quite correct. The bad news is that it terminated the session. But if you use Perkun as a library there is a way to overcome it and continue the session, so that you do not have to worry that using Perkun would break your programs.

Illegal actions

You may be wondering how to prevent some actions to be chosen by Perkun in some situations. There is a special instruction for that used in the model section. It is called "illegal".

The syntax is:


illegal(VISIBLE_STATE_QUERY,ACTION_QUERY);

The VISIBLE_STATE_QUERY contains the input variable values, while the ACTION_QUERY contains the output variable values. That is it, Perkun won't try to use these actions when these situations occur!

Example:


       illegal({where_is_Dorban=>place_Wyzima,       do_I_see_vampire=>false},
       {action=>goto_Wyzima});
       illegal({where_is_Dorban=>place_Wyzima,        do_I_see_vampire=>true},
       {action=>goto_Wyzima});
       illegal({where_is_Dorban=>place_Shadizar,     do_I_see_vampire=>false},
       {action=>goto_Shadizar});
       illegal({where_is_Dorban=>place_Shadizar,      do_I_see_vampire=>true},
       {action=>goto_Shadizar});
       illegal({where_is_Dorban=>place_Novigrad,     do_I_see_vampire=>false},
       {action=>goto_Novigrad});
       illegal({where_is_Dorban=>place_Novigrad,      do_I_see_vampire=>true},
       {action=>goto_Novigrad});

Impossible states

If we have a deterministic model then it can be represented as a collection of directed graphs, each graph for one action (fixed values of the output variables). It may happen that in these graphs certain states cannot be achieved.

There is an instruction for the model section that denotes that. It is called "impossible".

Its syntax is:


impossible(STATE_QUERY);

The STATE_QUERY contains values of the input and hidden variables.

Example:


       impossible({where_is_Dorban=>place_Wyzima,     do_I_see_vampire=>false,
       where_is_vampire=>place_Wyzima});

Interactive mode

Try to execute:


    perkun example2_dorban.perkun

You will enter an interactive mode, perkun will print a message and a prompt:


loop with depth 3
I expect the values of the variables: where_is_Dorban do_I_see_vampire 
perkun>

Type Ctrl+D to quit the interactive mode. In real life you would embed perkun in your own C++ program and make it run the interactive mode internally.

Using perkun as a library

Take a look at the perkun wars source code. It is a good example how to embed perkun in your own projects. Its configure.ac file contains the following line:


    PKG_CHECK_MODULES([PERKUN], [libperkun >= 0.0.5])

It means that perkunwars requires at minimum the perkun version 0.0.5. In src/Makefile.am there is code:


AM_CXXFLAGS = @CXXFLAGS@ \
	-I.. -I../inc \
	@XML_CFLAGS@ @GTK_CFLAGS@ @PERKUN_CFLAGS@ \
	-DDATADIR=\"$(datadir)\"

perkun_wars_LDADD = \
	@LIBS@ @GTK_LIBS@ @PERKUN_LIBS@ @XML_LIBS@

This code includes both the compilation flags for perkun (@PERKUN_CFLAGS@) as well as the perkun libraries (@PERKUN_LIBS@).

The prefered way to use perkun is to create pipes and then to fork so that the child process can instantiate a class inherited from perkun::optimizer_with_all_data. This inherited class should speak through the pipes with the parent, obtain the input through a redefined get_input function, and execute the actions through a redefined execute function. See the perkun wars for developers.