wlodkowic can be used directly, in command line, or as a library (libwlodkowic).

Using wlodkowic from the command line

In order to use wlodkowic from the command line you will need a wlodkowic specification written in the Wlodkowic language, stored in a separate file. Take a look at the directory perkun-0.1.8/examples. You will find there some examples, like example9_medical.wlodkowic.

Wlodkowic differs from perkun in so far that it requires two extra sections in its specification. All together it requires the following sections:

  • values
  • variables
  • knowledge
  • payoff
  • model
  • apriori

These six sections are followed by any number of commands.

The simplest Wlodkowic program is:


       # begin of the program
       values {}
       variables {}
       knowledge {}
       payoff {}
       model {}
       apriori {}
       # end of the program

When you place the above text in a file (for example test.wlodkowic) and run:


$ wlodkowic test.wlodkowic

it will do nothing. That is correct. But if you remove any of the sections you will get an error. Also correct.

The values section of the Wlodkowic code contains the keyword "value" followed by the valid identifiers, separated by commas.

The values are used to denote the possible values of the variables. The Wlodkowic variables is a complex topic and will be discussed later. You can print out all the values with the command


       cout << values << eol;

For example:


       values
       {
            value false, true, do_nothing, switch;
       }
       variables{}
       knowledge{}
       payoff{}
       model{}
       apriori{}
       cout << values << eol;

The above Wlodkowic code will print out:


       # values
       false
       true
       do_nothing
       switch

There are many other commands apart from "cout << values << eol;". They can all be placed directly after the apriori section.

One-line comments begin in Wlodkowic with a hashtag "#".

Wlodkowic allows three kinds of variables:

  • input variables
  • hidden variables
  • output variables
The hidden variables are essential for the machine learning. They represent the state variables of the world that are not directly visible to the agent. Yet they do affect the state of the world.

Let us forget for now the hidden variables though, and introduce a simple Wlodkowic code with one input variable and one output variable:


       values
       {
            value false, true;
            value move, do_nothing;
       }
       variables
       {
            input variable what_I_can_see:{false, true};
            output variable action:{move, do_nothing};
       }
       knowledge {}
       payoff {}
       model {}
       apriori {}

As you can see the variables section contains now two variables:

  • what_I_can_see
  • action

Unlike many other programming languages Wlodkowic does not have the concept of a variable type. Instead each variable declaration is followed by a colon and a list of possible values in curly brackets, separated by commas. For example what_I_can_see may have value false or true.

Try using the command "cout << variables << eol;":


       values
       {
            value false, true;
            value move, do_nothing;
       }
       variables
       {
            input variable what_I_can_see:{false, true};
            output variable action:{move, do_nothing};
       }
       knowledge {}
       payoff {}
       model {}
       apriori {}
       cout << variables << eol;

When you put the above text in a file and call Wlodkowic it will respond:


# variables
what_I_can_see
action

The next section, knowledge, my contain so called "unimportant" rules.


       values
       {
            value false, true;
       }
       variables
       {
            input variable patient_is_alive:{false, true},
            patient_complains_about_x:{false,true};
            hidden variable disease_1:{false,true};
       }
       knowledge
       {
            unimportant({patient_is_alive=>false},
                patient_complains_about_x=>[{false,true}]);
            unimportant({patient_is_alive=>false},disease_1=>[{false,true}]);
       }
       payoff {}
       model {}
       apriori {}
       cout << visible states << eol;

The above example will print out:


       # visible states
       patient_is_alive=>{false},patient_complains_about_x=>{false,true}
       patient_is_alive=>{true},patient_complains_about_x=>{false}
       patient_is_alive=>{true},patient_complains_about_x=>{true}

The general syntax for the "unimportant" rule is:


            unimportant(CONDITION_QUERY,VARIABLE=>[EQUIVALENCE_CLASS1,...,EQUIVALENCE_CLASSN]);

The equivalence classes are lists of values enclosed in curly brackets ({}). It is allowed to mix the input and hidden vari‐ ables in the condition query. The variable denoted by the "unimportant" rule can also be a hidden variable. Then they affect the amount of the states (try "cout << states << eol;").

We know already how to declare Wlodkowic values and variables. We know that the agent i.e. Wlodkowic program can see the input variables and can control the output variables. The question is how to control the output variables. What do we want to achieve? What is "good" for the agent and what is not?

In order to define it we use the payoff section of the Wlodkowic program.

Try the following code:


       values
       {
            value false, true;
            value move, do_nothing;
       }
       variables
       {
            input variable what_I_can_see:{false, true};
            output variable action:{move, do_nothing};
       }
       knowledge {}
       payoff
       {
            set({what_I_can_see=>false},0.0);
            set({what_I_can_see=>true},1.0);
       }
       model {}
       apriori {}
       cout << payoff << eol;

It will print out the contents of the payoff section:


       # payoff
       what_I_can_see=>{false} payoff 0.00000
       what_I_can_see=>{true} payoff 1.00000

The payoff section contains the commands "set" with two arguments:

  • input state query
  • payoff value (a real number)
The query contains all input variables followed by "=>" and the corresponding equivalence class, separated by commas, enclosed in curly brackets. In short it is an expression:


       {INPUT_VARIABLE1=>{VALUE1,VALUE2,...},INPUT_VARIABLE2=>{VALUE1,VALUE2,...},...}

It is allowed to omit the curly brackets enclosing the equivalence class if it contains only one value.

Note that the query must contain all the input variables. What does the payoff value mean? It is a value representing "how good" the payoff function is (for the respective combination of the input variable values). The higher the payoff value the "better" it is. In the example above the agent "likes" to see the value "true" on its input (variable what_I_can_see), while it dislikes the value "false", because for "true" the payoff value equals 1.0, while for "false" it equals 0.0.

Wlodkowic hidden variables

We do not perceive directly everything that is happening in the world. Or to say it otherwise - in order to understand the world it is useful to imagine that some facts about it remain hidden. Likewise - in Wlodkowic it is assumed that the world has a state described by the visible variables (so called input variables) and by the invisible variables (so called hidden variables). How to declare them in Wlodkowic?


       values { value false, true; }
       variables
       {
       input variable a:{false, true}, b:{false, true};
       hidden variable c:{false, true}, d:{false, true};
       }
       knowledge {}
       payoff {}
       model {}
       apriori {}
       cout << variables << eol;

The above Wlodkowic code defines two input variables (a and b) and two hidden variables (c and d). All of them may have either value false or true. When you run it with wlodkowic it will write out:


# variables
a
b
c
d

But how can we use the hidden variables? What is it good for, to take into account the variables we do not perceive? Well, in order to understand this we need to learn how to use the model section of the Wlodkowic program.

Wlodkowic model

The time has come to learn the model. We know already how to declare the variables (input,output or hidden ones) and how to define the payoff, i.e. tell Wlodkowic which input variable values are good and which are not so good. In order to do its job Wlodkowic needs to know how its decisions will affect the world, i.e. how the world state in the future depends on its state now and the Wlodkowic's choices.

Imagine a world where we have only one variable - input variable alpha. Its value can be either false or true. Let us introduce an output variable beta which can control two actions:

  • do_nothing - alpha remains as it is now
  • switch - alpha switches to not alpha

Let us further assume that Wlodkowic "likes" alpha=true, and "dislikes" alpha=false. For this simple world our Wlodkowic program will look as follows:


       values
       {
            value false, true, do_nothing, switch;
       }
       variables
       {
            input variable alpha:{false, true};
            output variable beta:{do_nothing, switch};
       }
       knowledge {}
       payoff
       {
            set({alpha=>false},0.0);
            set({alpha=>true},100.0);
       }
       model
       {
            set({alpha=>false},{beta=>do_nothing},{alpha=>false},1.0);
            set({alpha=>false},{beta=>do_nothing},{alpha=>true},0.0);
            set({alpha=>true},{beta=>do_nothing},{alpha=>false},0.0);
            set({alpha=>true},{beta=>do_nothing},{alpha=>true},1.0);
            set({alpha=>false},{beta=>switch},{alpha=>false},0.0);
            set({alpha=>false},{beta=>switch},{alpha=>true},1.0);
            set({alpha=>true},{beta=>switch},{alpha=>false},1.0);
            set({alpha=>true},{beta=>switch},{alpha=>true},0.0);
       }
       apriori{}
       cout << model << eol;

The model section is filled with the set instructions, but they have slightly different syntax than the set instructions we know from the payoff section. Each set instruction in the model section has syntax:


set(INITIAL_STATE_QUERY,ACTION_QUERY,TERMINAL_STATE_QUERY,VALUE);

The INITIAL_STATE_QUERY and TERMINAL_STATE_QUERY are regular queries containing all input and hidden variables.

The ACTION_QUERY is a regular query containing all output variables.

The VALUE is a probability value.

In short set(A,B,C,D) means "if A and you do B then the probability to jump to C is D".

Note that the sum of the probabilities over the terminal state queries equals 1.0 for each initial state query and each action query.

Wlodkowic apriori section

The apriori section can also be filled with the "set" instructions, they have the following command:


set(INPUT_STATE_QUERY,HIDDEN_STATE_QUERY,VALUE);

The INPUT_STATE_QUERY and HIDDEN_STATE_QUERY are regular queries containing all input or all hidden variables, respectively.

Congratulations, now you know everything what you have to tell Wlodkowic. Values, variables, knowledge, payoff, model, apriori. The time has come to see what Wlodkowic can do for you given this information. It will attempt to maximize the expected value of the payoff function by appropriate choosing the actions. First you give it current values of the input variables, then it responds with the optimal action chosen (the values of the output variables). As you can see we will interact with Wlodkowic and this interaction will involve both input and output.

The loop command

As you remember the sections in the Wlodkowic code may be followed by instructions, and now is the time to learn another one: loop. It takes one argument - an integer constant, which denotes how deep Wlodkowic should look into the game tree. Try the following code:


       values
       {
            value false, true, do_nothing, switch;
       }
       variables
       {
            input variable alpha:{false, true};
            output variable beta:{do_nothing, switch};
       }
       knowledge {}
       payoff
       {
            set({alpha=>false},0.0);
            set({alpha=>true},100.0);
       }
       model
       {
            set({alpha=>false},{beta=>do_nothing},{alpha=>false},1.0);
            set({alpha=>false},{beta=>do_nothing},{alpha=>true},0.0);
            set({alpha=>true},{beta=>do_nothing},{alpha=>false},0.0);
            set({alpha=>true},{beta=>do_nothing},{alpha=>true},1.0);
            set({alpha=>false},{beta=>switch},{alpha=>false},0.0);
            set({alpha=>false},{beta=>switch},{alpha=>true},1.0);
            set({alpha=>true},{beta=>switch},{alpha=>false},1.0);
            set({alpha=>true},{beta=>switch},{alpha=>true},0.0);
       }
       apriori
       {
       set({alpha=>false},{},1.0);
       set({alpha=>true},{},1.0);
       }
       loop(1);

Wlodkowic will respond with the message and a prompt:


loop with depth 1
I expect the values of the variables: alpha
alpha>

For each input variable requested type all its values separated by space (remember, the whole equivalence class may contain more than one value due to the knowledge section's unimportant rules).

Now it is your turn. You can enter one of the two values that the input variable alpha may have, i.e. false or true. Let us first try false:


       alpha> false
       belief:
       /alpha=>{false}=>1.00000
       executing beta=>{switch}
       alpha>

In this simple example there are no hidden variables so the belief does not contain any.

As the optimal action the beta=switch was chosen! Why? Because Wlodkowic can see alpha=false, it does not like it (see the payoff) and knows that if it uses switch the alpha will change its value to not alpha. How does it know it? From the model. That is it, this is how Wlodkowic works. OK. Let us continue. Let us now enter true:


       alpha> true
       belief:
       /alpha=>{true}=>1.00000
       executing beta=>{do_nothing}
       alpha>

Again Wlodkowic chooses an optimal action, but this time it is beta=do_nothing! Why? Because it can see alpha=true, it likes it (see the payoff) and knows that do_nothing will keep the value of alpha unmodified. The last fact, again, is known from the model.

The test command

You may call the command test to print out the problems with the model to the standard error stream:


test();

Prolog generators

The following instruction prints out a Prolog code that will help you to create the model:


cout << prolog generator << eol;

Illegal actions

You may be wondering how to prevent some actions to be chosen by Wlodkowic in some situations. There is a special instruction for that used in the model section. It is called "illegal".

The syntax is:


illegal(VISIBLE_STATE_QUERY,ACTION_QUERY);

The VISIBLE_STATE_QUERY contains the input variable values, while the ACTION_QUERY contains the output variable values. That is it, Perkun won't try to use these actions when these situations occur!

Example:


       illegal({where_is_Dorban=>place_Wyzima, do_I_see_vampire=>false}, {action=>goto_Wyzima});
       illegal({where_is_Dorban=>place_Wyzima, do_I_see_vampire=>true}, {action=>goto_Wyzima});
       illegal({where_is_Dorban=>place_Shadizar, do_I_see_vampire=>false}, {action=>goto_Shadizar});
       illegal({where_is_Dorban=>place_Shadizar, do_I_see_vampire=>true}, {action=>goto_Shadizar});
       illegal({where_is_Dorban=>place_Novigrad, do_I_see_vampire=>false}, {action=>goto_Novigrad});
       illegal({where_is_Dorban=>place_Novigrad, do_I_see_vampire=>true}, {action=>goto_Novigrad});

Impossible states

If we have a deterministic model then it can be represented as a collection of directed graphs, each graph for one action (fixed values of the output variables). It may happen that in these graphs certain states cannot be achieved.

There is an instruction for the model section that denotes that. It is called "impossible".

Its syntax is:


impossible(STATE_QUERY);

The STATE_QUERY contains values of the input and hidden variables.

Example:


impossible({where_is_Dorban=>place_Wyzima, do_I_see_vampire=>false, where_is_vampire=>place_Wyzima});

Using wlodkowic as a library

In order to use wlodkowic as a library add the following line to your configure.ac:

    PKG_CHECK_MODULES([WLODKOWIC], [libwlodkowic >= 0.1.8])

If the library libwlodkowic is installed in your system this will cause the symbols WLODKOWIC_CFLAGS and WLODKOWIC_LIBS to become defined. You can use them as follows in your Makefile.am:

    AM_CXXFLAGS = @CXXFLAGS@ \
	-I.. -I../inc \
	@WLODKOWIC_CFLAGS@ \
	-DDATADIR=\"$(datadir)\"

some_project_LDADD = \
	@LIBS@ @WLODKOWIC_LIBS@