AWBW Scraper

 

Intro

Due to Nintendo’s announcement on a remake of Advance Wars 1+2 which happens to be one of my favorite games from my childhood, I stumbled upon the “Advance Wars by Web” website in which it’s possible to play matches under the “chess-by-mail” concept. One of the many neat features this site has, is that replays can be saved and viewed online. To extend this, I decided to code a script that allowed me to download the frames of the turns and compile them into gif or video files.

CodeDev

The first thing to do, was to figure out how the replay system works. From the website’s FAQs: “…all replays are deleted from AWBW after 3 weeks to preserve storage space. If you want to save the replay of a game before it is deleted, you can click Download on the Replay page to download a local copy of the replay. To view the saved replay, use the Upload Replay tool to load the replay back onto AWBW. This will allow you to view the replay for 48 hours. If you need more time, simply upload the replay again.”. Fortunately, I had saved all my previous matches, so I could use those files to test the replay system out.


After uploading one of my replays, I checked the URL: https://awbw.amarriner.com/2030.php?games_id=347352&ndx=0 (probably won’t work now because it’ll get deleted). A couple of things immediately stood out:

  1. The game id is clearly visible
  2. The ndx number seemed to match the game’s “turn”

With this in mind, I figured it wouldn’t be so difficult to use selenium to automate a script that allowed me to take screenshots of the battlefield at each point of the match.

Setting up globals

First thing I did was to create a file to store all the globals I was going to use in the project, which are: paths, website’s URL, headless mode and sleep time for selenium (these will be explained once we go into the code).

DRV_PTH = './chromedriver/chromedriver'
OUT_PTH = '/home/chipdelmal/Documents/AWBW/'

SLEEP = 12
HEADLESS = True
BASE_URL = 'https://awbw.amarriner.com/2030.php?games_id={}&ndx='
MAP_ONLY = True

Getting user input

There’s a couple of things to manually tweak before running the script. The first one is to input the game’s ID number, which can be easily obtained through the replay’s URL: https://awbw.amarriner.com/2030.php?games_id=347352&ndx=0 (in this case, the ID is 347352). The second one, is to get the number of turns the match took. For this, we go to the replay’s page, click the drop-down list and select the last entry (it’s the one located besides the “Download” button). This will change the URL slightly to: https://awbw.amarriner.com/2030.php?games_id=347352&ndx=74 and the number we’re looking for is the one that comes besides the ndx=, which is the number we asign to TRN (74 in this case). Now, we look for a nice zoom level by clicking the zoom and finding one that fits in our screen resolution (the number of clicks is the number we set to the ZOM variable). Finally, we give our match an arbitrary name (for the folders to be created), which we store in the NAME variable.

(ID, NAME, TRN, ZOM) = ('357091', '04_Booyah', 57, 8)
OUT_PTH = '/home/chipdelmal/Documents/AWBW/'

Launching selenium

First thing was to generate the base URL to sweep through the frames and create an output directory to store the frames:

URL = cst.BASE_URL.format(ID)
OUT_PTH = path.join(cst.OUT_PTH, NAME)
if not path.exists(OUT_PTH):
    os.makedirs(OUT_PTH)

Next, we can setup selenium to startup in a large resolution if we’re running it headless, or maximized if it’s not going to run headless. Additionally, we launch the driver with the options and the path in which we saved it (can be downloaded here):

options = webdriver.ChromeOptions()
if cst.HEADLESS:
    options.add_argument("--window-size=1920x1080")
    options.add_argument("--headless")
else:
    options.add_argument("--start-maximized")
driver = webdriver.Chrome(cst.DRV_PTH, options=options)
driver.get('{}{}'.format(URL, 0))

Once the driver’s launched, we auto-click the “zoom in” button however many times we set it up to:

for i in range(ZOM):
    driver.find_element_by_id('zoom-in').click()

Iterating through turns

Now let’s go to the fun part. We want to iterate through the turns of the match, so we append the ndx value to the base URL and wait for some time so that we allow the page to load (AWBW’s server is fairly unstable, so I tend to set the SLEEP value to 10 or 15 seconds). After doing so, we take a screenshot of the area we want to save (MAP_ONLY=True saves the map, whilst MAP_ONLY=False saves the whole battle screen). After taking our screenshot, we simpy save it to disk:

for ndx in range(0, TRN+1):
    print('* Parsing ({}/{})'.format(ndx, TRN), end='\r')
    driver.get('{}{}'.format(URL, ndx))
    sleep(cst.SLEEP)
    if cst.MAP_ONLY:
        element = driver.find_element_by_id('gamemap-container')
    else:
        element = driver.find_element_by_id('gamecontainer')
    imgName = '{}_turn_{}.png'.format(prep, str(ndx).zfill(3))
    imgPath = path.join(OUT_PTH, imgName)
    element.screenshot(imgPath)

frames

Shutting down the driver

Finally, we close the driver!

sleep(cst.SLEEP)
print('* Done ({}/{})'.format(ndx, TRN), end='\r')
driver.close()

Creating GIF

The final step for us is to create our GIF from the exported images. This can be easily done with ImageMagick:

convert -delay 120 -loop 0 ./*.png "AWBW_Match.gif"

Bash Script

Now, to go a step further, we can automate the whole process in a bash script:

ID=$1
NAME=$2
TURNS=$3
ZOOM=$4
PTH=$5

IN="$PTH/$NAME/*.png"
OUT_LG="$PTH/$NAME/${NAME}"

python main.py $ID $NAME $TURNS $ZOOM $PTH
convert -delay 120 -loop 1 $IN "${OUT_LG}.gif"
convert -delay 120 -loop 1 $IN "${OUT_LG}.mp4"

Which can be called as:

./awbw.sh "357091" "01_ComradesInArms-Chip_Tomas" "57" "15" $OUT

Code Repo