Intelligent & Learning Agents

Three assignments on sequential decision-making, from bandits through planning to game-tree search.

• Regret minimisation — UCB, KL-UCB and Thompson Sampling on Bernoulli bandits, with the KL-UCB confidence bound located by binary search. An optimised variant recomputes the per-arm bounds once every eight pulls instead of every pull, trading a little precision for far fewer KL evaluations. A second task inverted the objective entirely: Poisson-damage targets where the aim is the fewest pulls rather than the most reward.
• Planning — an encoder/planner/decoder pipeline that enumerates every non-busting hand of a blackjack-like card game as an MDP state, then solves it two ways: Howard’s policy iteration, with the policy-evaluation step solved directly as a linear system, and the standard LP formulation through PuLP/CBC.
• Search — mini-chess agents running alpha-beta to depth four, with MVV-LVA move ordering to improve pruning and an evaluation combining material popcount over bitboards, centre control, pawn advancement and check.