Frogger Ep1 : Graphics Speed and Flicker

Stop Press :

Whilst in the process of writing the next article in this series it has become apparent that the ESP8266 processor being used (and at first thought suitable for this project) just doesn’t have enough inputs available (certainly not on the NODEMCU Dev. board I was using) to handle the SPI screen and the user direction controls. No matter how I tried to figure it and using odd-ball inputs for digital inputs or trying to multiplex the inputs this processor is just too input restricted to handle it. The Arduino has enough but it doesn’t have enough RAM/ROM/Speed to handle Frogger. So if you are planning to build this project do not order a ESP8266 based development board. You will need to order a ESP32 based board which I’ve had up and running with all inputs, so is definitely fine! However the main article below on solving speed and flicker is still valid and is still required on a ESP32 based system so read on…

 

Original article
Frogger was a classic Arcade game from the early 80’s (released 1981 in most parts of the world) by Konami. Have a quick look at You Tube (type in Arcade Frogger) for an idea of game-play if you are not sure what it is. We are going to attempt to re-create this game on our small MCU’s ad screens.

Before embarking on any project you need a general idea of whether the hardware can do what you want it to. In the last project, Space Invaders, we used a Arduino Nano and a 128×64 BW OLED display. This was just about OK for this game with it’s low graphical requirements and speed. However for Frogger it uses colour graphics with around 12 or so different colours (from initial inspection, the internet suggests the hardware supported 32). So the BW OLED was out. Colour graphics also require more memory and more data has to be written to screen per graphic per second, so speed could be an issue. In addition there were lots more of graphics compared to Space Invaders (and as mentioned each requiring more memory to store).

The resolution of Frogger (number of pixels across and down) of the arcade game was 224 across by 256 down. Screens of this resolution did not exist, it was also proving hard to find even a 256×256 res screen and cost was beginning to rise for higher res screens. However 128×128 colour screens were plentiful and quite cheap. Using a 128×128 would also mean we just need to half the pixel data in each graphic, something any bitmap art package such as the free GIMP or others can do easily.

The next choice was the Microprocessor to use. From experience I was also aware that to avoid flicker when updating the display (more on why flicker occurs later) we need an area of static RAM put aside as a screen buffer (a place where we can build up our display prior to showing it). The driver code for the BW 128×128 OLED from Adafruit used such a buffer and even then its small requirements for a 1 bit per pixel display used up a reasonable amount of the 2Kb precious RAM memory.

For this display using the ST7735 driver chip using a buffer memory is not an option on the Arduino. This is because that to support the number of colours required (and for technical ease of programming we need 16bits to represent each pixel. Admittedly this gives us 65536 different colours, a little more than the 12 or so required. But the next level down in colours on this chip is 12bits per pixel giving 4096 colours but 12 bits is an awkward number of bits for systems based on multiples of 8bits, you would spend some time shifting and manipulating bits within bytes to maximise memory use, this may still work out to have advantages but would require more time invested as all existing driver code that I could find works with 16bit colours.

So we are set on using 16bits (2bytes per pixel). How much memory then do we need to a small resolution 128×128 pixel display. Well the number of pixels is 128 x 128 = 16,384 and each of those pixels requires 16 bits (2 bytes) to store the colour information; 16384 x 2 = 32766 bytes! 32kb of ram just to hold a buffer for the display. Which as mentioned to avoid flicker is essential. Drivers you find on the internet don’t usually have these buffers but they also are not designed to run games, just “normal” apps. Games with there varied and fast moving graphics can generate unacceptable amounts of flicker.

So with those memory requirements we can rule out Arduino for this project. The next chip that looks promising is the ESP8266, lots of program memory and around 80Kof RAM of which after overheads there is about 50K available. Using 32K up for a screen buffer would leave around 18K, plenty for our needs. It also runs at two user settable speeds, either 80Mhz or 160Mhz. It also supports Wifi but at present that is surplus to our requirements, we jut want this chip for its memory and speed.

So the hardware is set, a 128×128 colour TFT screen and a ESP8266 processor, looking around I chose the NodeMCU ESP8266 12E development board. Costs were roughly the same for both at around the price of a decent cup of coffee (delivered – the hardware not the coffee!).

Initial Experiments
OK, first task is to move a graphic around the screen and see what happens! Let’s grab one from the original game, here’s the one for when the Frogger gets home at the top of the screen.

We’ll move this from the top of the screen to the bottom – but first let’s jut get it on screen. Ensure you’ve set up your screen to work with your MCU, see this guide for setting up the NodeMCU ESP8266 to work in the Arduino environment and this guide for getting a 7735 128×128 TFT screen to work with this processor. Presuming you’ve done this put the code below into your Arduino environment. Note if your not sure how we are putting this graphic into our code then see this guide on how this is accomplished.

This should provide the following output.

All good so far! Now let’s add some simple code to move the frog from the top to the bottom of the screen. All we need do is draw it at a new Y position, here’s the code;

… and here’s the resultant video of the output

Why do we get the two vertical lines appearing. If you look carefully at the original graphic the first line of the graphic is the very top of the frogs eyes which consist of two lots of two green pixels, here’s a magnified view.

So we plot this to screen and then move the Y position 1 place down and re-plot. Note that  we are now re-plotting (re-drawing) over the entire graphic apart from the very top line of pixels (as we’ve moved down), which leaves these pixels on the screen. We then repeat this leaving another two pixels and so on and so forth leaving a vertical line behind us. What we need to do is to remove all the image first (blank to background, black in this case) and then re-draw the graphic. Some modern games creation systems will automatically do this for you but of course we are back to basics (like the good old days!) with doing everything ourselves. Which is a good thing, we can learn and appreciate exactly how game screens are created. So we have two options, w can remove just the area where the graphic is plotted and do this for all graphics on screen or just blank the entire screen. There are reasons why you would choose one over the other. If you only have a few graphics to re-plot then just erase what you have to. If you have a complex screen with many graphic then just blanking the entire screen before re-drawing can be the way to go. You may even do a sort of mix between the two, i.e. completely blanking some portions of a screen but not others. For out purposes we’ll blank all the screen. Here’s the code;

Watchdogs
Note that we have moved the code into the loop function and used a command called ESP.wdtDisable() and ESP.wdtEnable(); On the Arduino you could be fairly casual about what code you out in the setup and what you did in the loop, but on the ESP it has things called “watchdogs”. As the ESP was designed with the internet of things in mind it would not be good for the end consumer of their Smart Fridge crashed and they need to switch it off and on again to get the software running. To help prevent bad software etc. from causing this situation there are two watchdog timers that look to see if the device has come out of the main loop periodically, this indicates to the watchdog that everything is OK. If we take longer than about 1 second or so to do something then the device re-boots. We can disable this with ESP.wdtDisable() and re-enable with  ESP.wdtEnable(). However this is just enabling or disabling the first watchdog. A second watchdog watches the first watchdog! and if it detects that it has’t been working for about 8 seconds it also re-boots the device, that’s why about 3/4 the way down the screen the device re-boots. Look at the video below of the result;

Flicker and Speed!
First thing to notice is that the image flickers badly. This is because we are erasing the screen to black and re-drawing, meaning that you don’t see anything at times. Secondly the speed is awful, the frog now travels down the screen incredibly slowly now. Any of these two factors would make the game unplayable but we have both! But don’t worry they are solvable.

Solving the Speed
The SPI bus used is capable of any speed the hardware can handle, there is no official limit! By default the Adafruit library sets the speed at 4Mhz (roughly 4million bits per second with some overhead). But testing has shown that even with the rubbish long wires used the ST6635 driver chip can handle much more. However this proved not to be the main issue for the speed. Looking at the driver code it was designed for a lot of flexibility (understandably) and was not always taking advantage of features of the ST7735. I read through the nearly 200 page data sheet for this chip to see if I could see if there was anything we could do to improve speed of data transfer – it was a heavy read!  And there was! Back in the day (from the first display driver chips of the 70’s on-wards) for bitmapped driver display chips they have an address counter which tells the driver chip where to store the next byte for the display, when it has stored a byte it automatically increments this counter. And you could manually change this if you wanted. What most of the Adafruit code was doing was setting this address counter for nearly every pixel sent. But if a pixel is directly next to the previous one (or even the first on a row after plotting the last on the row above) then there is no need to do this and it just means you are sending more data over the bus that is not needed when you could have been sending the data itself. I knew this would have a dramatic improvement on speed. But to make best use of this technique to improve speed needs a screen buffer….

Solving the flicker – the screen buffer
As mentioned earlier a screen buffer is an area of memory where you can build up your display without the user seeing it and then display it all at once completely removing the previous screen display. This removes all flicker. The reason Adafruit did not go this root is because they want to sell their screens to a wide an audience as possible and Arduino’s or other low memory processors just don’t have the memory to spare. So their solution was excellent to be compatible with a wide range of devices but not good for gaming or where graphics have to move on a display. So we have to use a screen buffer to get the flicker free graphics and therefore a processor like the ESP8266 which has good memory and is just the price of a decent cup of coffee (on a development board) is perfect. We could also now look at how to push this buffer screen to the display in the shortest possible time. Adafruit have a routine for “blitting” (a back in the day phrase that was used to describe graphic chips moving pixels to the display quickly and usually just in hardware) images to screen called drawRGBBitmap. However it is not optimised for speed and sends the position of every pixel to be drawn to the screen first as mentioned in the paragraph above. This dramatically reduces speed by a factor of something like 14-16 times. So I wrote a new routine that just “blits” this screen buffer to the screen as quickly as it can. These two techniques solved all problems, the screen buffer for flicker and more efficient transfer of bits to the display when sending this buffer.

So I modified the Adafruit screen driver for the ST7735 again. If you’ve been following this guide and connected the colour screen to your ESP8266 then you are already using the new XTronical driver,  as this was already a modified version of the Adafruit version. This version is able to use (or not) a screen buffer. You can turn it off an on as required for your particular use. It defaults to “Off” so people going through other guides in this site that use it on Arduino etc. can still have their code work as it always has. I won’t go into the technical details of how the driver has been optimised but just tell you how to turn on and use the screen buffer in your code (it is really simple!).  So first things first if you’ve already installed the XTronical St7735 driver from some time ago then it is likely an old version and needs removing. See this guide if your not sure how to manually remove a library. Then download and install the latest version below (if your not sure how to install libraries manually then see this guide).

XTronical-ST7735-Library

This version includes an example for using the screen buffer. Go to “File->Examples->XTronical ST7735 Library->Screen Buffer Test” and load it up. Go ahead and compile it (CTRL R) and you should get the error below

error: ‘class Adafruit_ST7735’ has no member named ‘displayBuffer’

This is because by default the screen buffer is turned off so that as standard compatibility is maintained from previous versions so that existing program code with small memory MCU’s is not broken. You need to purposely enable the screen buffer. To do this open up the file “XTronical_ST7735.h” in a text editor of your choice (I use the excellent NotePad++). This file will be within the “XTronical-ST7735-Library” folder which is within the Arduino Library folder (see this guide to locate it if your not sure). Un-comment the line that reads //#define SCREEN_BUFFER so it just reads #define SCREEN_BUFFER. Save and close this file. You will now be able to compile the example and upload it. You should have an output the same as below

Notice here how it is significantly faster and yet we are actually plotting 8 frogs not 1! and all with no screen flicker whatsoever!

So now we’re set, we have our “graphics engine” which is fast enough for the game. Note that it only runs at around 12 FPS (12 Frames Per Second), basically it can update the entire screen 12 times a second, which is quite slow by some standards but for this game it will be fine. There is actually still room for even more optimisation of the display driver code to make it at least twice as fast (ie. 24FPS) and perhaps more, but that would require more investment of  time in this area and as this is fast enough for our needs and I want to get coding Frogger I’m leaving as is for now.

That’s all for this episode, click here for the Next