Publishing my master’s thesis

2023-12-19T00:00:00+00:00

Hey everyone!

It’s been a while and there is much to talk about, but that will have to wait for another day. Today, I’m just here to publish my master’s thesis. You can find it here.

Many thanks to João [1] for his great supervision!

[1] http://joao-carvalho.net/

Publishing my bachelor’s thesis

2020-02-07T00:00:00+00:00

Hey everyone!

Since I got a message asking me about some details on my bachelor’s thesis, I went on the webs and was surprised to find that there is no PDF version available. I’ll go ahead, and finally publish my thesis here.

Oh, and great thanks again to Sven [1], who was a great supervisor!

[1] http://www.sven-amann.de/

On Reinforcement Learning

2019-11-13T00:00:00+00:00

Hi!

This is kind of a long story, so feel free to skip to the tl;dr. :)

In the winter semester 2018/19, I attended a lecture offered by the Intelligent Autonomous Systems (IAS) [1] group at TU Darmstadt. These lectures have the reputation to be very time-intensive, but they are just as much rewarding. Only for that semester, there was a rare opportunity to get the whole swoop of reinforcement learning in the form of a lecture, a project, and a seminar paper. Though this meant sacrificing some CP for the semester, I felt like this was a great chance to get into the topic. So I did.

As I was thinking about a topic for my seminar paper, most other students chose something pragmatic like a survey, or comparisons between approaches. Since I had a good understanding of the basics of machine learning from the previous lecture in Statistical Machine Learning, I wanted to do something that fits my style better. So I thought about writing an introductory paper to reinforcement learning. Especially focused on policy gradients, as I intuitively liked the implications of directly working with a policy.

The first draft I came up with attempted to bring in a ton of references to current SotA approaches. I showed it to my supervisor, Samuele Tosatto (who was very patient with me; thanks for that!), who gave me a ton of criticism. Constructive criticism, that is. After that meeting, I had a much clearer idea about what I wanted to achieve with the paper. In essence, I wanted a formal introduction to policy gradient approaches. I changed the name to “An Introduction to Policy Gradients”, and set myself three terms I wanted to adhere to. My paper should be

complete, i.e., formally define everything we use,
concise, i.e., only introduce necessary definitions,
comprehensible, i.e., avoid logical shortcuts.

To be honest, my understanding is that all papers should follow these rules. Alas that this has not been my experience. Don’t get me wrong, I know this is hard, and it gets harder the more sophisticated the ideas we want to convey become. But you notice when someone is not trying hard enough. ;)

When I was done with the seminar, and sent in the finished paper, I thought about the naming again. I feel like “An Introduction” is not what people want to read. Hence I figured it was better to go with the name Samuele suggested.

TL;DR: I would like to introduce you to my paper “On Policy Gradients”. Feel free to check it out at https://arxiv.org/abs/1911.04817.

[1] https://www.ias.informatik.tu-darmstadt.de/

My first blog

2018-08-30T00:00:00+00:00

Hi! This is my first blog. :D

We’ll see if this evolves to something, but for now I will probably not post much here.

Mattis Kaemmerer

Publishing my master’s thesis

Publishing my bachelor’s thesis

On Reinforcement Learning

My first blog