The emulator component of the Glowforge Utilities Python package provides a convenient way to understand how the device interacts with the cloud service, as well as collecting stepper files for analysis. In this post, I’ll show how to configure and run the emulator, and dive into how it works.
This post assumes you are running the emulator on a Linux host, though it is just at home on Windows. Currently, the utilities support Python 2.7. I’ll be pushing an update soon to add support for Python 3.
EDIT 2018-08-17: Python 3 is now supported.
EDIT 2020-08-08: This is for version 0.5.19 and lower.
Configuration
Grab the code and install the prerequisites:
~$ git clone https://github.com/ScottW514/Glowforge-Utilities -b 0.5.19
~$ cd Glowforge-Utilities/
~/Glowforge-Utilities$ pip install -r requirements.txt
PIP will install the requests package for interaction with the cloud’s web API, and the lomond package for the WebSockets connection.
Next up, we need some important information from the Glowforge device itself.
From the Glowforge factory hardware/firmware console:
root@gf-abc-123:~# export PYTHONPATH=/glowforge/python
root@gf-abc-123:~# python -c "import glowforge.util.machine as machine; print machine.serial(allow_invalid=True)"
ABC-123
root@gf-abc-123:~#
The example output above, ABC-123 (your output will be different), for our purposes, is the hostname. Yes, we are calling the machine.serial() method, but it is not the Glowforge ‘serial number’ used when talking to the cloud (hey, I didn’t write it). You could, of course, skip this step and simply remove the ‘gf-’ from the command prompt to get the this value, but I wanted to show you the official way.
To get the cloud serial number, enter the following:
root@gf-abc-123:~# python -c "import glowforge.util.machine as machine; print machine.id()"
12345678
root@gf-abc-123:~#
The number you get, 12345678 in our example, is the serial number used to connect to the cloud.
Lastly, we need the password:
root@gf-abc-123:~# python -c "import glowforge.util.machine as machine; print machine.password()"
0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
root@gf-abc-123:~#
Back to our emulator host:
~/Glowforge-Utilities$ cp gf-machine-emulator.cfg.sample gf-machine-emulator.cfg
~/Glowforge-Utilities$ vi gf-machine-emulator.cfg
Update the following configuration lines with the values you pulled from your Glowforge:
serial: 12345678
hostname: ABC-123
password: 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
The rest can be left as their default values.
Running the Emulator
To fire up the emulator, all you need to do is run the gf-machine-emulator.py script:
~/Glowforge-Utilities$ python gf-machine-emulator.py
(INFO) emulator:__init__ INITIALIZED
...
The emulator script contains the following code:
from GF.DEVICE.emulator import Emulator
gf = Emulator('gf-machine-emulator.cfg')
gf.connect()
gf.run()
All of the emulation functions are wrapped inside the Emulator class, which are broken down into three main tasks: initialization, connection, and message processing.
INITIALIZATION
This is fairly simple:
# From DEVICE.emulator.py -> Emulator.__init__()
def __init__(self, cfg_file):
# Load the provided configuration file, and store it in the class 'cfg' variable.
self.cfg = configuration.parse(cfg_file)
# Initialize the class variable that we use to store the 'requests' session object.
self.session = None
# Create the thread safe queues that we will use to send and receive messages over the WebSocket
self.q = {'rx': Queue.Queue(), 'tx': Queue.Queue()}
# Set up Logging
# If console logging is enabled in the config file:
if self.cfg['GENERAL.CONSOLE_LOG_LEVEL']:
logging.basicConfig(format='(%(levelname)s) %(module)s:%(funcName)s %(message)s',
level=self._log_level(self.cfg['GENERAL.CONSOLE_LOG_LEVEL']))
else:
# Otherwise, we log to a file:
logging.basicConfig(filename=self.cfg['GENERAL.LOG_FILE'],
format='%(asctime)s (%(levelname)s) %(module)s:%(funcName)s %(message)s',
level=self._log_level(self.cfg['GENERAL.LOG_LEVEL']))
# Done with the initialization process
logging.info('INITIALIZED')
CONNECTION: Authentication
To establish the connection with the cloud, the first thing we need to do is authenticate the machine to the service:
# From DEVICE.emulator.py -> Emulator
def connect(self):
# Grab a persistent web session. Once authenticated, this session is used for all web API's.
self.session = connection.get_session(self.cfg)
# Authenticate machine
if not authenticate_machine(self.session, self.cfg):
return False
Authentication is handled by the GF Utilities’ DEVICE.authentication module.
def authenticate_machine(s, cfg):
...
# The machine's cloud serial number and password are JSON encoded
# and sent in the body of an HTTP POST to the cloud's web API.
r = request(s, cfg['SERVICE.SERVER_URL'] + '/machines/sign_in', 'POST',
data={'serial': 'S' + str(cfg['MACHINE.SERIAL']), 'password': cfg['MACHINE.PASSWORD']})
# If we are successful, the cloud responds with an authorization token to
# use with future web API requests, and another token for authenticating to the WebSocket.
if r:
rj = r.json()
logging.debug(rj)
cfg['SESSION.WS_TOKEN'] = rj['ws_token']
cfg['SESSION.AUTH_TOKEN'] = rj['auth_token']
# We add the API token to the session so it will send it with all future calls
update_header(s, 'Authorization', "Bearer %s" % cfg['SESSION.AUTH_TOKEN'])
logging.info('SUCCESS')
return True
else:
logging.error('FAILED')
return False
CONNECTION: Firmware Check
At this point, we can now check with the web API to see if new firmware is available, which introduces us to the actions interface provided by the DEVICE.actions module (and begins our trip down the rabbit hole):
# From DEVICE.emulator.py -> Emulator
def connect(self): # (continued)
...
# Check for new firmware
if self.cfg['GENERAL.FIRMWARE_CHECK']:
fw = actions.run_action('web', 'firmware_check', s=self.session, cfg=self.cfg)
if fw:
actions.run_action('web', 'firmware_download', s=self.session, cfg=self.cfg, fw=fw)
The DEVICE.actions module maps desired actions into functions that execute the appropriate web or WebSocket API located in the DEVICE.web_api and DEVICE.ws_api modules, respectively.
The firmware_check call to the web API is mapped to the following function:
def _api_firmware_check(s, cfg):
...
# Send event to web API to let it know we are checking for the latest firmware
_process_actions(s, cfg, {'MILLI_EPOCH': _milli_epoch()}, {0: _ACTIONS['firmware_check'][0]})
# Grab the version of the latest firmware from the web API
r = request(s, cfg['SERVICE.SERVER_URL'] + '/update/current', 'GET')
...
# Send event to web API to let it know what version we found, and what version we are running.
_process_actions(s, cfg, {'AVAILABLE_FIRMWARE': rj['version']},
{0: _ACTIONS['firmware_check'][1], 1: _ACTIONS['firmware_check'][2]})
...
The first thing this function does is make a call to the modules local _process_actions() function. The purpose of _process_actions() is to execute a set of predefined actions. These actions are defined in DEVICE.__init___.py, as seen for the firmware_check call:
_ACTIONS = {
# The call we are making
'firmware_check':
{
# The group of actions for this call
0: {
# The first action for this group
0: {'delay': 0, 'act': True, 'api': 'web', 'run': 'event', 'msg': '{"local": "<MACHINE.FIRMWARE>", "type": "event", "id": <MILLI_EPOCH>, "event": "firmware_update:check:starting"}'},
},
...
The actions are grouped together depending how they are executed. In this case, each action is simply the sending of an ‘event’ message to the web API. Each group consists of a single submission of messages. Essentially, the following:
_process_actions(s, cfg, {'MILLI_EPOCH': _milli_epoch()}, {0: _ACTIONS['firmware_check'][0]})
is just sending a status message to the Glowforge web API telling it that we are checking for the latest firmware version.
Continuing on - if the firmware version is newer than we have listed in our configuration file, the Emulator.connect method will run the firmware_download action that will download the new firmware package and save it in the ./_RESOURCES/FW folder:
def _api_firmware_download(s, cfg, fw):
...
download_file(s, fw['download_url'], '%s/glowforge-fw-v%s.fw' % (cfg['GENERAL.FIRMWARE_DIR'], fw['version']))
...
CONNECTION: Establish WebSocket
The Emulator.connect method makes a call to the DEVICE.websocket module’s ws_connect() method to establish the WebSocket:
# From DEVICE.emulator.py -> Emulator
def connect(self): # (continued)
...
# Establish WebSocket Connection
if not websocket.ws_connect(self.cfg, self.q):
return False
return True
GF Utilities provides the WsClient class, which handles all of the underlying details of the WebSocket. The ws_connect() function kicks that off:
def ws_connect(cfg, q):
# Create the WebSocket client object
ws = WsClient(cfg, q)
# Start the background process that establishes the WebSocket session, and spawns
# the WebSocket processing thread
ws.start()
...
When the start() method of the WsClient object is called, it establishes the connection to the Glowforge WebSocket service, and then starts up a new thread to process the WebSocket messages:
#From DEVICE.websocket.py -> WsClient
# Continuously loop through the incoming WebSocket events
for event in persist(self.ws):
...
if event.name == 'text':
# Service has sent us a text message, stick it in the queue
self.q_rx.put(event.text)
while self.ready:
if not self.q_tx.empty():
# There are messages in the outgoing queue we need to send to service
send_msg = self.q_tx.get()
self.ws.send_text(send_msg)
self.q_tx.task_done()
else:
break
The WsClient object now occupies its own thread where it sits and listens for incoming messages from the service, and outgoing messages from the emulator (it also handles pings, and other housekeeping for us). If it gets a message from the service, it tosses it in the incoming queue for later processing by the emulator. If the emulator has a message to send to the service, it tosses it in the outgoing queue where it is picked up WsClient and sent to the service.
EMULATOR MESSAGE PROCESSING LOOP
The last task for our gf-machine-emulator.py script is to kick off the Emulator's message processing loop by making a call to its run() method:
#From DEVICE.emulator.py -> Emulator
def run(self):
while True:
# Check the incoming queue to see if the service has a message for us
if not self.q['rx'].empty():
msg = json.loads(self.q['rx'].get())
if msg['status'] == 'ready':
# Run the associated WebService action
actions.run_action('ws', msg['action_type'], s=self.session, q=self.q, cfg=self.cfg, msg=msg)
...
In the next post, we’ll look at how the emulator handles requests from the service, including the homing and printing processes.