Screen Scraping

Boston WorkStation versus Screen Scraping

Screen Scraping Introduction and History

Around this time, an alternate approach was developed. This approach eliminated the complexities associated with the differences in terminal types by taking advantage of the terminal emulator on the desktop. Terminal Emulators are essentially parsers that retrieve/generate the screen rendering instructions and data from the mainframe to terminal data stream. Once rendered, a screen is a screen regardless of the terminal type used. Boston Software Systems was an originator of this approach with their MS DOS based version of Boston WorkStation. This approach is best called Scripting, although since the end results appear similar, the term Screen Scraping is sometimes used to describe what Boston WorkStation does.

Terminal emulator-based applications are still quite prevalent in health care (and other industries as well) due to the stability of the mainframes and user interaction efficiency. Other display types have come to prominence over the years ­ windows based, Web, Java®, and various types of remote desktops. However the basic need to get information into and out of these systems or to automate tedious jobs remains.

As these changes on the user's desktop occurred, various integration standards arose to address the need to share information between the ever increasing numbers of disparate systems. In a perfect world, these standards cause Boston Software Systems seamless data sharing between all applications. The reality is that while a lot of data is being shared via "standards", there is a great deal of data that is not shared­ and here either scripting or humans bridge the gap.

Download a case study where Boston WorkStation is used for screen scraping.

Screen scraping and historical challenges

"A scripting [or screen scraping] approach to interfacing is unreliable". Many long time developers are wary of screen-based technologies based on a bad first hand experience or from the assumption that if there is any change on the screen, a screen based approach is destined to fail and will be difficult to maintain over time.

The reality is that application vendors typically do not make wholesale changes very often to the way their application screens accept interaction from and display data to a user. As you can imagine, wholesale changes would mean retraining their internal support, training an implementation services staff plus thousands of end users. Hence, changes are typically either small, or if they are large, they are well planned out in advance. Wholesale changes to the way an application screen accepts interaction from and displays data to a user would of course cause a Boston WorkStation script written against the previous version of the display to "fail" and would require modification just as a "traditional" interface would fail. If the application changes its functionality, at a minimum any existing interfaces to that application would need to be tested to assure they still work as designed, and changes made to the interface.

In many cases, a screen-based approach may very well be the only realistic option available. Boston WorkStation has developed a number of features and capabilities to handle unexpected changes and provides powerful tools to overcome objections to screen-based interfaces.

Screen Scraping and Healthcare Applications

Non-positional screen condition detection and retrieval handles many screen changes

The techniques of the past (and those used today by some products) are highly dependant upon positional methods of detecting screen conditions. In character based systems, these approaches would rely on row and column positions. In graphical applications, they use optical character recognition (OCR) or positional based component interaction via windows messaging. While these methods are also available to Boston WorkStation users, there are additional, more powerful features that make Boston WorkStation scripts surprisingly flexible and able to function even after changes have been made to the screen that would cause other screen scraping or scripting tools to fail.

In character-based systems, Boston WorkStation can detect the screen name based on any number of text labels anywhere on the screen regardless of case or position. The cursor position can also be detected relative to field labels and/or based on being in a range of positions. Boston WorkStation also automatically determines the status of the screen ­ whether or not it is ready for interaction. This prevents the false positives on cursor positions that other scripting tools can get based on changing screen response time.

In graphical applications ­and this includes Windows, Web and Java applications Boston WorkStation has a unique searching capability. Components can be searched for based on a combination of what they are, their text, and even what they can do. Using this unique searching capability, components on a graphical application can be found regardless of their position on the screen (even if the screen is minimized). Once found, the scripter has access to numerous properties of the component ­ whether it has focus, its text, what it is, where it is, what it can do ­and then can interact with it.

Using these advanced features, Boston WorkStation scripts are not affected by changes in "where the fields are" on the screen or by new features or capabilities being added that move things around. As long as the fields are still there, the script will work as designed even after the change.

Single product deployment provides convenient over
time management capabilities

Boston WorkStation provides a number of tools to assist in detecting and fixing script failure or business exceptions. Boston WorkStation scripts can not only detect the unexpected, scripts can also attempt to recover by performing any actions a person could to recover from an unexpected screen condition (error) ­it can even reboot the machine if necessary. In addition, the error information and even a history of the screens encountered and actions performed can be easily captured and logged. And notifications can be sent via email, secure FTP and a variety of other communication methods including invoking Web Services. Using this information it is easy to track down the exact point in the script where a failure occurred, and using the built-in development environment, the change can be made, tested and put into production without requiring any additional development tools. However, the script itself is Visual Basic ­ not a proprietary "BWS invented" language. Therefore scripts can be compiled if desired and of course the full power of Visual Basic can be leveraged in the script regardless of how it is deployed.

Screen Scraping Summary

The position taken by Boston Software Systems has always been to not compete against standards. In fact, our product has features that embrace existing and future integration standards. Boston WorkStation can understand and if needed output HL7, X.12, ASTM and even supports Service Oriented Architectures using XML / Web Services. "If a standards-based interface exists­ use it" has always been our response when asked why someone would use Boston WorkStation verses any other product. But we do not live in a perfect world. There have always been, and logic dictates, always will be gaps that Standards will not address. While it seems logical to assume interacting with an application's screen is an unreliable way to update or retrieve information from an application. the Boston WorkStation approach provides a cost effective and reliable way to read from, write to, or react to what is on or done on an application's screen. There are also several unique capabilities Boston WorkStation offers. By acting as an "electronic employee", Boston WorkStation can perform any interaction on a PC that a person can, including interacting with multiple systems at once. And can even react to or intercept an end user's interaction with a system, providing user time access to business data or even altering / augmenting the functionality of the application - without changing it. The Boston WorkStation approach is vendor agnostic, it does not require any understanding of the technical "behind the scenes" workings of the target system or its databases, nor does it require any interaction with or the cooperation of the vendor of the target system. Boston WorkStation simply gets the job done, often when there simply is no other alternative ­ and done in a way that is much more reliable than "conventional wisdom" may suggest.