You can train an AI that beats our robots in a couple of minutes, on your own computer. You don't need a graphics card, or to know much about AI. Our AIs are called Mensch, after Mensch ärgere dich nicht, the German name for Ludo.
It's a small neural network: a few lines of code, and a lot of numbers it learned. It looks at a board and gives it a score. On each turn, it scores the board after each possible move, and takes the best.
It learns by playing against itself, over and over. After every move, it nudges its numbers so its scores get closer to what happened next. At the end of each game, the winner's last board gets pulled towards "good" and the others towards "bad". At first it's guessing. After thousands of games, it has worked out for itself what's worth doing.
This idea made a computer very good at backgammon in the 1990s. Its name is TD-Gammon, if you want to read more.
cd and a space, drag the ludowars-trainer folder onto the window, and press Enter.node tools/train.mjs --name "My first AI" --hidden 64 --games 100000 --out my-first-ai.jsonIt plays 100,000 games against itself, and tells you how it's going. On a laptop that takes a minute or two. When it's done, you have a file called
my-first-ai.json. That's your AI.There are two easy things to change in that command:
--games. More practice means a stronger AI, but it takes longer.--hidden. A bigger network can learn more, but it's slower to train and needs more games.This is how often ours win against three copies of the longer robot, and three of careful, our best one. A fair share would be 25%, so anything above that is winning:
| AI | Trained on | File size | Against longer | Against careful |
|---|---|---|---|---|
| Mensch-32 | 60,000 games | 40 KB | 66% | 34% |
| Mensch-64 | 400,000 games | 79 KB | 69% | 38% |
| Mensch-256 | 250,000 games | 316 KB | 75% | 43% |
All three were trained with this same trainer, so you can beat them. The Mensch-256 took about 12 minutes on a laptop. You can download them to compare, and pick them in the Sandbox and on Play.
Ready for more? The trainer is short and yours to change: tools/train.mjs. You could try training against our robots instead of itself, more than one hidden layer, or a different way of learning. Or design your own kind of AI altogether, as below.
An AI file holds two things: the code that does the thinking, and the numbers it learned. The trainer writes both for you. This is the whole code of Mensch-64:
function(weights, nn) {
// Mensch-64: a neural network with one hidden layer of 64 relu units.
// It scores the board after each possible move, and plays the best.
function score(board) {
var ids = nn.board_ids(board); // whose each piece is, and where
var x = nn.relu(nn.linear_ids(ids, weights['layer1.weight']));
x = nn.linear(x, weights['layer2.weight']);
return x.data[0];
}
return nn.play_best(score);
}
The code is yours to replace. Use a bigger network, a transformer, or something nobody has tried yet. All it has to do is run in time.
shape and its data, as in PyTorch: weights['layer1.weight'].shape is [64, 236] here.nn.play_best(score) makes one from a function that scores a board.Here is a different design, one attention block like a very small transformer. Each of the 16 pieces is a token, and the pieces look at each other before the AI scores the board:
// An AI of your own design: one attention block, like a very small transformer. See howto/ai.html.
// Each of the 16 pieces on the board is a token. The tokens look at each other, are averaged, and give one score.
// attention.py is the same network in PyTorch, to train it and write the file.
function(weights, nn) {
var w = function(name) { return weights[name]; };
var HEADS = 2;
function score(board) {
var x = nn.embedding(w('embed.weight'), nn.board_ids(board)); // [16 pieces, width]
var h = nn.layer_norm(x, w('norm1.weight'), w('norm1.bias'));
var q = nn.linear(h, w('query.weight'), w('query.bias'));
var k = nn.linear(h, w('key.weight'), w('key.bias'));
var v = nn.linear(h, w('value.weight'), w('value.bias'));
x = nn.add(x, nn.linear(nn.attention(q, k, v, HEADS), w('mix.weight'), w('mix.bias')));
h = nn.layer_norm(x, w('norm2.weight'), w('norm2.bias'));
h = nn.relu(nn.linear(h, w('grow.weight'), w('grow.bias')));
x = nn.add(x, nn.linear(h, w('shrink.weight'), w('shrink.bias')));
var pooled = nn.mean_rows(x); // [width]
return nn.linear(pooled, w('out.weight'), w('out.bias')).data[0];
}
return nn.play_best(score);
}
It's in the trainer download as tools/examples/attention.js, next to attention.py, the same network in PyTorch. Nobody has trained it yet, so it plays badly. Training it well is up to you.
A tensor is {shape, data}, with the last dimension changing fastest, as in PyTorch. Each function gives back a new tensor, and does the same sums as the PyTorch on the right.
| nn. | What it does | In PyTorch |
|---|---|---|
linear(x, weight, bias) | A fully connected layer. Leave out bias if you don't have one | nn.Linear |
linear_ids(ids, weight, bias) | A linear layer on counts of ids, only quicker: linear(one_hot(ids, inputs), ...) | nn.Linear |
embedding(table, ids) | One row of the table for each id | nn.Embedding |
attention(q, k, v, heads) | Attention, with the heads side by side in the columns | F.scaled_dot_product_attention |
layer_norm(x, weight, bias) | Normalise the last dimension | nn.LayerNorm |
relu, tanh, sigmoid, gelu | The usual activations. gelu is the tanh version | F.relu and so on |
softmax(x) | Along the last dimension | F.softmax(x, -1) |
add(a, b), mul(a, b), scale(a, s) | Add or multiply. b can be the shape of a's last dimensions, like a bias | a + b, a * b, a * s |
matmul(a, b), transpose(a) | For tensors with 2 dimensions | a @ b, a.T |
rows(a, start, end), mean_rows(a) | Some of the rows, or their average | a[start:end], a.mean(0) |
tensor(shape, data), one_hot(ids, size), map(a, f), size(shape) | Make a tensor, or count how many numbers a shape holds | |
board(move) | The 16 positions of a possible move, as you see them | |
board_ids(board) | One id from 0 to 235 for each piece: whose it is, and where. See what goes in | |
play_best(score) | A robot that plays the move whose board scores best |
node tools/check_ai.mjs my-ai.json
This loads your file the way the site does and runs its checks. Then it plays 100 games against our robots, and tells you how it did and roughly how long a turn would take on our server.
If you already know PyTorch, train any way you like and save with our exporter, tools/export_pytorch.py, which is in the trainer download. For a Mensch, a stack of layers:
import torch.nn as nn from export_pytorch import export, features net = nn.Sequential(nn.Linear(236, 128), nn.ReLU(), nn.Linear(128, 1)) # ... train it, feeding it features(board) ... export(net, "mybot.json", name="My bot", author="Me")
Your model must be an nn.Sequential of Linear layers, each optionally followed by ReLU, Tanh or Sigmoid. features(board) turns a board into the inputs below.
For a design of your own, write the code that runs it, and save with export_code:
from export_pytorch import export_code
export_code(net, "mybot.json", code=open("mybot.js").read(), name="My bot",
architecture="Attention-32", score=my_score)
Every tensor in net.state_dict() goes in the file under its own name, so your code reads weights['embed.weight'] and so on. score(board) is your PyTorch score for a board of 16 positions. We keep it for a few boards and compare it with what your code gives, which catches the two not doing the same sums. tools/examples/attention.py does all this.
You'll need games to learn from, so use static/javascripts/ludoengine.js, or write your own copy of the rules.
You only need these if you write your own trainer or exporter.
A board is 16 positions as you see them, after your move: your 4 pieces, then those of the player who moves next, and so on round the table. Positions are the ones explained in how to build a robot. A Mensch turns them into 236 numbers:
4 players x 59 positions = 236 inputs player 0: you player 1: the player who moves next player 2: the player after that player 3: the player after that input number = 59 * player + slot position -1 (in the corner) slot 0 position 0 to 56 slot position + 1 position 57 or more (home) slot 58
Each input counts how many of that player's pieces are there, so it's a number from 0 to 4. nn.board_ids(board) gives the input number of each piece instead, which is what an embedding wants.
For a Mensch, one number: the score of that board. The highest score gets played, and only the order matters, so any scale works. If two moves tie, the first one is played. A design of your own can choose its move any way it likes.
This is what the trainer writes: a list of fully connected layers, which the site writes the code for. The last has 1 unit. Each layer has:
relu, tanh, sigmoid or lineartrue or false, and true if you leave it outThe site names the shape after its hidden layers: 64 units is a Mensch-64, and 512 then 32 is a Mensch-512-32. Your AI can have any name you like, shown next to its shape.
{
"format": "ludowars-model",
"version": 1,
"name": "My bot",
"author": "Me",
"layers": [
{"units": 64, "activation": "relu", "bias": false},
{"units": 1, "activation": "linear", "bias": false}
],
"weights": "AAAgQQAAgD8...",
"check": [
{"board": [-1,-1,-1,-1, -1,-1,-1,-1, -1,-1,-1,-1, -1,-1,-1,-1], "score": 0.2415},
{"board": [1,12,30,-1, 5,20,53,-1, 14,2,-1,-1, 57,40,8,3], "score": -1.4021}
]
}
weights is all the numbers as one long list of 32 bit little-endian floats, encoded as base64. Go through the layers in order, writing each layer's weights, then its biases if it has them. A layer's weights go output by output:
weight[output 0][input 0], weight[output 0][input 1], ... weight[output 1][input 0], weight[output 1][input 1], ...
That's the order PyTorch uses for Linear.weight. Keras keeps its kernels the other way round, so transpose them first. In the code, layer 1 is weights['layer1.weight'] and weights['layer1.bias'], and so on.
{
"format": "ludowars-model",
"version": 2,
"name": "My bot",
"author": "Me",
"architecture": "Attention-32",
"code": "function(weights, nn) { ... }",
"weights": {
"embed.weight": {"shape": [236, 32], "data": "AAAgQQAAgD8..."},
"out.bias": {"shape": [1], "data": "AACAPw=="}
},
"check": [
{"board": [-1,-1,-1,-1, -1,-1,-1,-1, -1,-1,-1,-1, -1,-1,-1,-1], "score": -0.4303}
]
}
architecture is a short label for your design, shown next to its name. Each table's data is its numbers as 32 bit little-endian floats in base64, last dimension fastest. Names are letters, numbers, dots, dashes and underscores.
check is a few boards, with the score your own tool gives each one. We work out the same scores when we load your file, and refuse it if they differ by more than 0.1%. That catches weights in the wrong order, or code that doesn't do the same sums as your training, which would otherwise give you an AI that loads fine and plays badly. A Mensch must have at least one. In a file of your own design they're up to you, and they need your robot to have a score, as nn.play_best gives it.
An AI file can hold code, so it runs walled off like any robot: on our server, and on the Play page. The Sandbox runs it in the page, so only load AI files there that you wrote or trust.