Hlearn(Hamlearn) is a machine-learning-algorythm created by Patrick Hammer (Ch4os),
where a agent can learn to survive in a world without meta-information from the programmer.
It is able to classify certain things as good/bad for itself automatically by his own wellness-attributes,
like the battery-state or its own health for example.
Sheme of the learning system:
The perception system:
A agent has to be able to operate in a complex world. And such a complex world has to give him a lot of sensor inputs
if the Agent should be able to recognice the things which are important for him.
So the inputs are high-dimensional, and the agent has to deal with that.
The solution is a general classification algorythm. I've used self organizing maps for that.
You can read about it on wikipedia: SelfOrganizingMap@Wiki
The prediction system:
This system gives the agent the ability to use the best strategy, which gives the best long-time-reward.
It does that by integrating the reward for all actions over n think-steps.
The quality system:
The agent has to build a copy of his environment to give good predictions.
I've used a self-organizing-map for that because it is good in recognicing incomplete patterns.
The idea: value=predicted_next_step target_value=next_step -> now optimice and calculate error
After some optimication time the Agent has a good copy of the environment which makes it possible for him to predict.
And the error can be used to control how far the agent looks into the future. (n-step-value of the prediction system).
When the predictions are good -> it's better to look more far into the future.
The motor system:
The output of the prediction system is a action.
A action can be "walking", "climb up a ladder" etc.
The simulation can use this simple actions as agent-output.
But what when it should learn walking by itself and when there are 10 muscles to control just one foot?
Then this system is the solution:
At the beginning a action like "0" will generate a randomized "muscle-control-pattern" like "011110100011000..."
When the reward of this action is good, it will save it. That's all.
Hlearn example implementation in C, downloadable at:
Hamlib@Sourceforge
A simple test-implementation in sage-Python:
(since this learning model profits from generalisation and since this implementation doesn't use
the SOM-perception-system, it uses a Q[input][action]=reward array like Q-learning instead)
