Connect-4 Game Playing Framework (C4GPF)c4board

Download C4GPF from GitHub

Connect-4 (Connect-Four) is an interesting game for several reasons. It is a non-trivial game, most humans will have difficulties to play well against a master level player (human or computer). At the same time it is of medium complexity (4.5·1012 states). It can be solved by tree-based methods (minimax, alpha-beta pruning).

Over the last years we have worked on a Java-based Connect-4 framework which makes it easily to develop, test and play with trainable Connect-4 agents. This work is now ready to be shared with other interested researchers or game-playing users. You can download it for your own research from this GitHub link.

"Why another Connect-4 program?", you might ask. There are many Connect-4 realizations, but – at least to our knowledge – no other one which incorporates a fast-playing perfect player. This player can be used to benchmark the strength of other, self-trained agents. We derived a variety of trainable agents (more details in our recent TCIAIG-paper "Online Adaptable Learning Rates for the Game Connect-4"). We have developed the framework in such a way that it is for an experienced Java programmer fairly easy to add his / her own agent. Once added, several methods for evaluation are at the disposal of the user.

So we want to encourage with this Connect-4 framework other researchers to benchmark their game-playing agents against other Connect-4 agents with well-known strength. If you are interested in having your Connect-4 agent being added to the C4GPF framework, feel free to contact us via eMail.

Features of C4GPF:

  • built-in reinforcement learning agent (TD-learning)
  • eligibility traces
  • several adaptive step-size learning schemes: TCL, IDBD, …
  • N-tuple features
  • perfect-playing Minimax agent with alpha-beta pruning and opening book
  • interface "Agent.java" for easy plug-in of new agents
  • several benchmarking options (competitions, move inspections, …)

Getting started:

  • Read the file CFour/READM.txt on GitHub
  • Read the file CFour/src/doc/index.htm = CFour/src/doc/Help.pdf (help file for the GUI of C4GPF)

Authors of C4GPF:

  • Markus Thill (markus.thill "at" fh-koeln.de)
  • Wolfgang Konen (wolfgang.konen "at" fh-koeln.de)

Download C4GPF from GitHub


Publications Games

2023

Konen, Wolfgang

Towards Learning Rubik's Cube with N-tuple-based Reinforcement Learning Journal Article

In: arXiv preprint arXiv:2301.12167, 2023.

Links | BibTeX

Seven, Meltem

KI-Agenten im Vergleich: Erfolg von Agenten des Ludii General Game Systems in Partien gegen perfekte oder starke Agenten am Beispiel der Spiele Vier Gewinnt, Nim und Othello Masters Thesis

TH Köln – University of Applied Sciences, 2023, (Bachelor thesis).

Links | BibTeX

2022

Konen, Wolfgang

The GBG Class Interface Tutorial V2.3: General Board Game Playing and Learning Technical Report

TH Köln 2022.

Links | BibTeX

Fabig, Niklas

Reinforcement Learning für Standard Operating Procedures einer Atomkraftwerkssimulation Masters Thesis

TH Köln -- University of Applied Sciences, 2022, (Master thesis).

BibTeX

Weitz, Ann

Untersuchung von selbstlernenden Reinforcement Learning Agenten im computergenerierten Spiel Yavalath Masters Thesis

TH Köln -- University of Applied Sciences, 2022, (Bachelor thesis).

Links | BibTeX

Raycheva, Ralitsa

Reinforcement Learning in Simulationsspielen: Repräsentation von großen Aktions- und Zustandsräumen am Beispiel von Ökolopoly Masters Thesis

TH Köln -- University of Applied Sciences, 2022, (Bachelor thesis).

Links | BibTeX

Cöln, Julian

KI-Konzepte für das Erlernen nicht-deterministischer Spiele am Beispiel von "EinStein würfelt nicht!" Masters Thesis

TH Köln -- University of Applied Sciences, 2022, (Bachelor thesis).

Links | BibTeX

Weitz, Ann

Entwicklung einer allgemeinen Schnittstelle zwischen Ludii und dem GBG Framework Technical Report

TH Köln -- University of Applied Sciences 2022, (Praxisprojekt).

Links | BibTeX

Scheiermann, Johannes; Konen, Wolfgang

AlphaZero-Inspired Game Learning: Faster Training by Using MCTS Only at Test Time Journal Article

In: IEEE Transactions on Games, 2022.

Links | BibTeX

Scheiermann, Johannes; Konen, Wolfgang

AlphaZero-Inspired Game Learning: Faster Training by Using MCTS Only at Test Time Journal Article

In: arXiv preprint arXiv:2204.13307, 2022, (Preprint of the IEEE ToG 2022 paper).

Links | BibTeX

2021

Meissner, Simon

Untersuchung des Spiel- und Lernerfolgs künstlicher Intelligenzen für ein nichtdeterministisches Spiel mit imperfekten Informationen: Blackjack in der Game-Learning-Umgebung ’General Board Game’ (GBG) Masters Thesis

TH Köln -- University of Applied Sciences, 2021, (Bachelor thesis).

Links | BibTeX

Zeh, Tim

Untersuchung von allgemeinen KI-Agenten für das Spiel Poker im General Board Games Framework Masters Thesis

TH Köln -- University of Applied Sciences, 2021, (Master thesis).

Links | BibTeX

Raycheva, Ralitsa

Erstellung eines Custom Environments in OpenAI Gym für das Spiel Ökolopoly Technical Report

TH Köln -- University of Applied Sciences 2021, (Praxisprojekt).

Links | BibTeX

Konen, Wolfgang; Bagheri, Samineh

Final adaptation reinforcement learning for N-player games Journal Article

In: arXiv preprint arXiv:2111.14375, 2021.

Links | BibTeX

2020

Scheiermann, Johannes

AlphaZero-inspirierte KI-Agenten im General Board Game Playing Masters Thesis

TH Köln -- University of Applied Sciences, 2020, (Bachelor thesis).

Links | BibTeX

Konen, Wolfgang; Bagheri, Samineh

Reinforcement Learning for N-Player Games: The Importance of Final Adaptation Proceedings Article

In: Vasile, Massimiliano; Filipic, Bogdan (Ed.): 9th International Conference on Bioinspired Optimisation Methods and Their Applications (BIOMA) , Bruxelles. Video (15 min) + slides available at https://youtu.be/OcpX7ITeH9w, 2020.

Links | BibTeX

Konen, Wolfgang

The GBG Class Interface Tutorial V2.2: General Board Game Playing and Learning Technical Report

Research Center CIOP (Computational Intelligence, Optimization and Data Mining), Oct 2020.

Links | BibTeX

Konen, Wolfgang; Bagheri, Samineh

Final Adaptation Reinforcement Learning for N-Player Games Technical Report

Research Center CIOP (Computational Intelligence, Optimization and Data Mining) 2020.

Links | BibTeX

Scheiermann, Johannes

Sind (trainierte) General-Purpose-RL-Agenten im Brettspiel Othello stärker als (untrainierte) General-Game-Playing Agenten? Technical Report

TH Köln, Institut für Informatik 2020, (Praxisprojekt).

Links | BibTeX

2019

Konen, Wolfgang

General Board Game Playing for Education and Research in Generic AI Game Learning Proceedings Article

In: Perez, Diego; Mostaghim, Sanaz; Lucas, Simon (Ed.): IEEE Conference on Games, London, 2019.

Links | BibTeX

Konen, Wolfgang

The GBG Class Interface Tutorial V2.0: General Board Game Playing and Learning Technical Report

Research Center CIOP (Computational Intelligence, Optimization and Data Mining) 2019.

Links | BibTeX

Barsnick, Felix

Implementierung und Untersuchung eines Turniersystems für KI-Agenten in Brettspielen Masters Thesis

TH Köln -- University of Applied Sciences, 2019, (Master thesis).

Links | BibTeX

Cöln, Julian; Dittmar, Yannick

Untersuchung von KI Agenten im Spiel Othello Technical Report

TH Köln, Institut für Informatik 2019.

Links | BibTeX

2017

Galitzki, Kevin

Selbstlernende Agenten für das skalierbare Spiel Hex: Untersuchung verschiedener KI-Verfahren im GBG-Framework Masters Thesis

TH Köln -- University of Applied Sciences, 2017, (Bachelor thesis).

Links | BibTeX

Konen, Wolfgang

The GBG Class Interface Tutorial: General Board Game Playing and Learning Technical Report

Research Center CIOP (Computational Intelligence, Optimization and Data Mining) Cologne University of Applied Science, 2017, (e-print published at http://www.gm.fh-koeln.de/ciopwebpub/Kone17a.d/TR-GBG.pdf).

Links | BibTeX

Kutsch, Johannes

KI-Agenten fur das Spiel 2048: Untersuchung von Lernalgorithmen für nichtdeterministische Spiele Masters Thesis

TH Köln -- University of Applied Sciences, 2017, (Bachelor thesis, 2nd prize in CBC award 2018).

Links | BibTeX

2016

Bagheri, Samineh; Thill, Markus; Koch, Patrick; Konen, Wolfgang

Online Adaptable Learning Rates for the Game Connect-4 Journal Article

In: IEEE Transactions on Computational Intelligence and AI in Games, vol. 8, no. 1, pp. 33-42, 2016, (accepted 11/2014).

Links | BibTeX

2015

Konen, Wolfgang

Reinforcement Learning for Board Games: The Temporal Difference Algorithm Technical Report

Research Center CIOP (Computational Intelligence, Optimization and Data Mining) Cologne University of Applied Sciences, 2015.

Links | BibTeX

Konen, Wolfgang

Reinforcement Learning für Brettspiele: Der Temporal Difference Algorithmus Technical Report

Research Center CIOP (Computational Intelligence, Optimization and Data Mining) Cologne University of Applied Science, 2015, (Updated version 2015).

Links | BibTeX

2014

Thill, Markus; Konen, Wolfgang

Connect-4 Game Playing Framework (C4GPF) Miscellaneous

2014.

Links | BibTeX

Konen, Wolfgang; Koch, Patrick

Adaptation in Nonlinear Learning Models for Nonstationary Tasks Proceedings Article

In: Bartz-Beielstein, T.; Filipic, B. (Ed.): PPSN'2014: 13th International Conference on Parallel Problem Solving From Nature, Ljubljana, pp. 292–301, Springer, Heidelberg, 2014.

Links | BibTeX

Bagheri, Samineh; Thill, Markus; Koch, Patrick; Konen, Wolfgang

Online Adaptable Learning Rates for the Game Connect-4 Technical Report

CIplus no. TR 03/2014, 2014, (Preprint version of the article in IEEE Transactions on Computational Intelligence and AI in Games, 2016).

Links | BibTeX

Thill, Markus; Bagheri, Samineh; Koch, Patrick; Konen, Wolfgang

Temporal Difference Learning with Eligibility Traces for the Game Connect-4 Proceedings Article

In: Preuss, Mike; Rudolph, Günther (Ed.): CIG'2014, International Conference on Computational Intelligence in Games, Dortmund, pp. 84 – 91, 2014.

Links | BibTeX

2012

Thill, Markus; Koch, Patrick; Konen, Wolfgang

Reinforcement learning with n-tuples on the game Connect-4 Proceedings Article

In: Coello Coello, Carlos; Cutello, Vincenzo; others, (Ed.): PPSN'2012: 12th International Conference on Parallel Problem Solving From Nature, Taormina, pp. 184–194, Springer, Heidelberg, 2012.

Links | BibTeX

Thill, Markus

Reinforcement Learning mit N-Tupel-Systemen für Vier Gewinnt Masters Thesis

TH Köln -- University of Applied Sciences, 2012, (Bachelor thesis, 1st prize in Opitz award 2013, Festo award 2012, Ferchau award 2012).

Links | BibTeX

2011

Konen, Wolfgang

Self-configuration from a Machine-Learning Perspective Technical Report

Research Center CIOP (Computational Intelligence, Optimization and Data Mining) Cologne University of Applied Science, Faculty of Computer Science and Engineering Science, no. 05/11; arXiv: 1105.1951, 2011, ISSN: 2191-365X, (e-print published at http://arxiv.org/abs/1105.1951 and Dagstuhl Preprint Archive, Workshop 11181 "Organic Computing -- Design of Self-Organizing Systems").

Links | BibTeX

2009

Konen, Wolfgang; Bartz-Beielstein, Thomas

Reinforcement learning for games: failures and successes -- CMA-ES and TDL in comparision Proceedings Article

In: GECCO '09: Proceedings of the 11th Annual Conference Companion on Genetic and Evolutionary Computation Conference, pp. 2641–2648, ACM, Montreal, Québec, Canada, 2009.

Links | BibTeX

Konen, Wolfgang; Bartz-Beielstein, Thomas

Evolutionsstrategien und Reinforcement Learning für strategische Brettspiele Technical Report

Cologne University of Applied Sciences 2009.

Links | BibTeX

2008

Konen, Wolfgang; Bartz-Beielstein, Thomas

Reinforcement Learning: Insights from Interesting Failures in Parameter Selection Proceedings Article

In: Rudolph, Günter; others, (Ed.): PPSN'2008: 10th International Conference on Parallel Problem Solving From Nature, Dortmund, pp. 478–487, Springer, Berlin, 2008.

Links | BibTeX

Konen, Wolfgang

Reinforcement Learning für Brettspiele: Der Temporal Difference Algorithmus Technical Report

Cologne University of Applied Sciences 2008.

Links | BibTeX

Konen, Wolfgang; Bartz-Beielstein, Thomas

Reinforcement Learning für strategische Brettspiele Technical Report

Cologne University of Applied Sciences 2008.

Links | BibTeX