Overall ML experience
I dabbled with ML since 2019, taking courses in Data science, Deep RL and Multi-agent AI. I am familiar with ML fundamentals, modern DL frameworks and the latest generative AI advances. However, I have yet to develop hands-on research experience with LLMs.
Relevant projects:
RL coursework
RL Coursework - Google Drive
Notebook 1: Implemented bandit agents using a UCB, REINFORCE and $\epsilon$-greedy policies. Analysing regret plots from various experiments and answer theoretical questions about convergence behaviour and edge cases.
Notebook 2: Implemented on-policy TD-learning agent and analysed performance in a tabular setting. Implemented and analysed various off-policy Bellman operators with function approximation.
Notebook 3: Implemented in jax an actor-critic agent with a softmax policy parametrised by a tiny neural network. Implemented the Adam optimiser from scratch. Run multiple experiments, also varying the policy and the TD update (using $\epsilon$-greedy besides softmax policy). Answered theoretical questions about convergence curves and potential improvements.
Notebook 4: Implemented various off-policy multi-step returns and analysed their convergence and variance.
Multi-agent AI coursework
Multi-Agent AI Coursework - Google Drive
Individual coursework:
- Computed and visualised the Nash equilibria for ****two-player two-action normal-form games.
- Computed the Nash equilibrium of the 𝑁 firms, and also the market price in the limit of the Nash equilibrium for the Cournot Duopoly (nonzero-sum) game. Run various experiments to analyse player strategies.
- Derives (analytically) the payoff function from the policies of two agents in a repeated game with Markov strategies. Implemented a game simulation to validate the analytical solution.
Group coursework:
- Implemented various gradient-based algorithms (1) Infinitesimal Gradient Ascent (IGA), 2) IGA WoLF (Win or Learn Fast), 3) IGA-PP) to solve a 2-player, 2-action normal form game. Visualised and analysed (Lyapunov Stability Analysis) their learning processes, weaknesses, strengths and convergence properties. Performed hyper-parameter fine tuning using random search and investigated the effect of choosing different initial strategies, step size, prediction length. Run multiple experiments testing different payoff matrices and algorithm combinations
- Implemented a game simulator for a pollution tax game (a variant of stochastic game, general sum). Analysed various variations of policy/value iteration, Q-learning, IGA (applied to a multi-agent setting) to find an equilibrium for this game.
Other projects