Link here

Implementing a Remote Instrument Front Panel

How to make a clickable pixel-perfect remote web interface to control your embedded GUI device


Implementing a "Remote Front Panel" Using the Webserver

Some Mosaic controllers offer a touchscreen-based "Graphical User Interface" (GUI) that enables a user to control the instrument by touching buttons and menus on the screen. The software that runs the GUI is called the "GUI Toolkit" and is available as a pre-coded "kernel extension" library from Mosaic Industries. These GUI-based controllers include the QVGA Controller and the PDQScreen; the feature is not recommended for use on the QScreen due to hardware limitations that slow its ability to serve the screen image over the web.

A web-based "Remote Front Panel" can be added to a touchscreen-based controller using an EtherSmart or WiFi Wildcard based on the Lantronix XPort or WiPort web devices. The remote front panel enables instrument access management, allowing a user to control the instrumentation via a web browser. The browser displays a bitmapped image of the GUI screen exactly as it appears on the controller’s LCD (Liquid Crystal Display) as illustrated in Figure 1-14. Clicking on the web-based image with a mouse has the same effect as touching a button on the touchscreen: both the local screen image and the web image are updated. There is a delay before the new image appears on the web, because it takes time to extract, format and serve out the bitmapped image. Use of the remote front panel feature can expand the capabilities of an instrument, offering the possibility of web-based remote control, reporting, and diagnostics via an Ethernet or WiFi connection.

To implement this feature in your application, you can incorporate the source code presented in Appendix C (for C programmers) or E (for Forth programmers). The GUI Toolkit functions that enable this feature are described in a special section at the end of the EtherSmart/WiFi Wildcard Glossary document.

The remainder of this section describes how the remote front panel works.

ethernet web gui

Figure 1-14 Web-based remote front panel running on the QVGA Controller.

The HTML text shown in Listing 1-16 creates the image shown in Figure 1-14:

Listing 1-16 Remote front panel HTML source from gui_response_text.html.

This file is made into a resource by the Image Converter program as described in the section above titled "Introduction to the Dynamic Webserver: Using the Image Converter to Create Web Resources", and in Appendix A. Briefly, we place the gui_response_text.html file along with the GUI demo program’s button graphics in a directory, invoke the Image Converter from the development environment, select the correct platform, check both "GUI images" and "Web Files", and click "Convert Files Now". The HTML in Listing 1 16 can now be referred to by the generated constants GUI_RESPONSE_TEXT_HTML_XADDR and GUI_RESPONSE_TEXT_HTML_SIZE; the constant names are based on the specified file name.

A standard set of <html>, <head>, and <title> tags start the web page generation in Listing 1 16. The <H3> header style is used to display the "EtherSmart/GUI Remote Front Panel" announcement. The <H4> tag formats the instruction to "Click on the screen image to operate the touchscreen remotely". The next line is the key to the technique:

<a href="/gui_response.cgi"><img src="/screen_image.bmp" ismap></a>

The ismap keyword at the end of the tag tells the browser that the image should be treated as an "imagemap". Any mouse click in the image causes the browser to send the x and y coordinates of the mouse click (relative to the upper left corner of the image) as a specially formatted query field appended to the URL. The EtherSmart/WiFi driver’s pre-coded HTTP_Imagemap function extracts the coordinates from the URL for use by the GUI Toolkit. The anchor tag declares an href reference to the URL /gui_response.cgi that is invoked each time the image is clicked. The handler for this URL invokes the HTTP_Imagemap function to extract the x and y mouse click coordinates, and passes them to the GUI Toolkit’s Simulated_Touch_To_Image function. This processes the coordinates and updates the screen as if an actual touch to the front panel had occurred. The <img> tag specifies that the image itself is served out by the /screen_image.bmp URL as specified by the image src keyword. The </body> and </html> tags complete the page source.

To start the web display, we need a starting web handler function that serves the initial screen image when the webserver is first connected to the controller. This web handler is called Remote_Panel_Start in Listing 1 17 and is designated as the home page invoked by either the / or /index.html URL. This function creates an HTTP header in the HTTP_Outbuf using the HTTP_Put_Headers function. It then calls the Init_Screen_Image function, which invokes the GUI Toolkit function Screen_To_Image to place the bitmap image in a buffer.

Because this web handler calls a GUI Toolkit function (Screen_To_Image), task synchrony issues come into play. The GUI Toolkit often runs in its own task, and the webserver runs in the Ethernet task. To avoid multitasking errors and resource conflicts, functions that involve both GUI and HTTP functions must be synchronized with one another. This is done automatically if some simple programming rules are followed:

Programming rules for the Ethernet/GUI Remote Front Panel on the EtherSmart/WiFi Wildcard

  1. Each web handler function that invokes a GUI Toolkit function (Graphic_To_Image, Screen_To_Image, Simulated_Touch_To_Image, or Screen_Has_Changed) must be posted using HTTP_Add_GUI_Handler. The standard non-GUI function HTTP_Add_Handler must not be used for these handler functions.
  2. Each web handler function that invokes a GUI Toolkit function must use one and only one call to HTTP_GUI_Send_2Buffers to send out the HTTP header and content.
  3. Each GUI handler function that changes the screen appearance must call the Screen_Has_Changed function.
  4. For QScreen Controllers only, the initialization statement Globalize_TVars(TVARS); must be invoked when the application starts up.

Following these rules allows the various tasks to work together in harmony to implement the remote front panel.

Let’s continue examining how the functions in Listing 1-17 work. We now have the initial screen image as shown in Figure 1-14 in the web browser window as placed by the Remote_Panel_Start handler function. Now the user clicks on one of the keypad buttons (say, the "7" button) in the image. This causes the following URL and formatted "imagemap" query string to be sent to the EtherSmart/WiFi Wildcard:


This URL is handled by the GUI_Response handler function whose source is shown in Listing 1-17. This handler calls the HTTP_Imagemap function which extracts the x and y values from the URL (43 and 69, respectively in this example). Because C functions can return only one parameter, the two 16-bit coordinates are combined into a 32-bit value that is declared as a TWO_INTS union as defined in the types.h file in the mosaic include directory of the C distribution. The handler stores x and y into separate variables, and passes them to the GUI Toolkit function Simulated_Touch_To_Image. This function treats the coordinates as those of an actual touch on the front panel, calls any GUI handlers associated with the button, updates the local display, and stores a bitmap image of the display into the specified buffer at BITMAP_BUFFER. GUI_Response then invokes HTTP_GUI_Send_2Buffers to serve out the HTTP header from HTTP_Outbuf, and the HTTP content from the resource string specified by GUI_RESPONSE_TEXT_HTML_XADDR and GUI_RESPONSE_TEXT_HTML_SIZE.

The web content served out by the GUI_Response handler function is shown in Listing 1-16. It uses the img tag to refer to the URL /screen_image.bmp. This URL is serviced by the Screen_Image_Response handler in Listing 1-17. When this handler is called, the bitmap image of the screen has already been created and is sitting in the BITMAP_BUFFER. The first 4 bytes at BITMAP_BUFFER contain a 32-bit count, with the data following. After generating the HTTP header using the standard function calls, the Screen_Image_Response function gets the size of the bitmap by invoking

image_size = FetchInt(BITMAP_BUFFER+2)

The remote front panel feature is limited to image sizes that can be represented as a 16-bit number, so the 2 most significant bytes of the count can be skipped. Two calls to HTTP_Send_Buffer serve out the HTTP header from HTTP_Outbuf, and the bitmap image data starting at BITMAP_BUFFER+4. Note that the Screen_Image_Response handler does not call any GUI Toolkit routines, so it need not follow the first two rules for Ethernet/GUI functions (shown in the box, above). Thus it can use two calls to HTTP_Send_Buffer instead of HTTP_GUI_Send_2Buffers, and it is posted via the HTTP_Add_Handler function in Install_GUI_Web_Handlers as shown in Listing 1-17.

Some additional steps are required to complete the remote front panel. These are additions to the GUI demo program that creates and manages the touchscreen-based image.

  1. Place a call to the Ether_Check_GUI function in the application task loop that runs the GUI toolkit. The GUI toolkit function is called Service_GUI_Events on the QVGA controller and Service_Touch on the QScreen. Search for the GUI Toolkit routine (or its equivalent on other controllers) in the GUI demo program, and insert the following function call in the main task loop:

    Ether_Check_GUI( E_MODULENUM ); // service remote front panel handler
  2. Add a call to the Screen_Has_Changed in each GUI handler function that changes the screen appearance.

Please read the "Notes" at the bottom of Listing 1-17 for a reminder of these and other required steps.

The GUI demo program which implements the touchscreen display screen shown in Figure 1-14 is started by the GUI_Demo function. To start both the GUI demo program and the remote front panel feature, we invoke the high level function named Ether_GUI_Web_Demo (for an EtherSmart Wildcard) or WiFi_GUI_Web_Demo (for a WiFi Wildcard) in Listing 1-17. The definition of main can be edited to invoke the function that corresponds to your Wildcard (EtherSmart or WiFi). It calls Ether_Task_Setup_Default, Install_GUI_Web_Handlers, and GUI_Demo to initialize the required tasks and services. After calling this high-level initialization function, browsing into the EtherSmart/WiFi Wildcard’s home page (port 80 at the Wildcard’s IP address) displays the remote front panel in your browser.

For C programmers, the C_Ether_gui_demo_includer.txt file in the EtherSmart_WC_Code\Ether_GUI_Demo directory loads the install.txt and image_data.txt files and the C compiler output file for the Ethernet GUI demo program. For Forth programmers, the Ether_gui_demo_includer.4th file in the \Ether_Demo directory loads these files and the source code file for the Ethernet/WiFi demo program. These commented files provide a useful template to manage the loading of this demonstration program.

Listing 1-17 Excerpts from Ethernet/WiFi/GUI remote front panel source code.

See also → Implementing Remote Instrument Front Panel

This page is about: Clickable Remote Web Interface, Remote Instrument Control, Touchscreen GUI, Access Embedded Web Server from Any Web Browser – An EtherSmart or WiFi Wildcard implements a web-based "Remote Front Panel" that allows a user to operate the unit via a web browser. The browser displays a bitmapped image of the GUI screen, and clicking on the web-based image with a mouse has the same effect as pressing a touchscreen button.