An OpenCV Experiment: PokerEye

Check out the source code! GitHub

What really is PokerEye?

PokerEye is a software I independently wrote while studying at the University of Central Florida to test my ability to use machine-learning libraries to parse live data from the computer screen. I figured I would approach this by mixing it with a personal hobby of mine: poker.

At PokerEye's current stage, it automatically takes screenshots from a live poker game using the Ignition Casino poker client and parses the player's stack, cards in their hand, the board, and the pot. Further, it uses this data to calculate the player's odds to win and tie, resulting in a statistical edge over the opponents.

Let's break it down.

Retrieving Active Table Windows

Firstly, since this project was developed using MacOS (M1 chip) and written in Python, the methods for retrieving screenshots of the table window from the Ignition Casino client are not publicly well-documented. I had to find an efficient way to retrieve a list of all active windows, including the window's title and bounds box (this represents the coordinates, width, and height of the window). I used the Quartz library to retrieve this information. This is where my initial problems (of many) began to arise. To retrieve the window's title, permission must be enabled for the IDE in the computer's System Preferences > Security & Privacy > Screen Recording (view below - PyCharm CE is my IDE):

Now that we have the required permissions to begin parsing all active window data, we have to sift through the windows and find all windows attached to the parent window with the title Ignition Casino Poker (using kCGWindowOwnerName), this is the client that Ignition Casino uses to manage all active tables. All subwindows of this client are typically active table windows (sometimes there will be another window attached to this client if the client's settings are open, but our program detects it as invalid and continues to search for tables instead). From here, we can store the table window using its window number, (kCGWindowNumber), title (kCGWindowName), and bounds (kCGWindowBounds). After gathering information about the table window we can finally begin screenshotting.

Screenshotting the Table Window

PokerEye utilizes Pillow for taking screenshots of specified bounds, and OpenCV to parse the screenshots by modifing their colors. Using the bounds of the window, we are able to successfully take a screenshot of the table window (view below):

After screenshotting the window, we can now begin parsing the data that is present in the screenshot. PokerEye parses the player's stack (240), hand (8♥ Q♦), its current position (dealer), seat number (9), the total pot (10), the board (2♠ 8♣ 7♦), and the blinds (2/4). Here is a visual representation of what PokerEye crops to read each of the values (outlined in red, dotted boxes):

After cropping each section from the screenshot, I used Google's Tesseract-OCR Engine to parse the crops into tangible values we can use for calculations. This may sound simple on paper, but this process actually took the majority of the time as I had to manipulate the colors using OpenCV to convert the crops into grayscale and use the correct tesseract configuration for each of the varying types of data. I searched for color codes to accurately determine the suit of the cards (red is hearts, blue is diamonds, green is clubs, and black is spades). Now we have all of the data we need to begin calculations.

Calculating Odds

To begin calculating odds, we must first realize that we do not know the other player's hands, so our calculations must simulate what the opponents could have. For sake of simplicity, I found a library on GitHub called pokertude that assisted in these calculations. I modified the library to work with PokerEye, and now we can input our player's hand and board to simulate the odds of winning and tying, along with a list of all possible ways our player could lose that hand. For now, these simple calculations are enough for a player to have a statistical advantage over the opponents, but I do plan on writing my own method of calculating odds to incorporate automatically calculated pot odds and expected value from calling or placing a bet.

Displaying our Calculations

Now that we have calculated our odds of winning, it would be pretty impractical to display these details in an ugly terminal window, so I simply created a window (using Python's standard GUI framework: Tkinter) that attaches to the left side of the table window at all times that beautifully displays the current game's state and all calculations to go along with it. Here is the final product: