One of the key aspects of any leading organisation today is to create an equal and socially inclusive environment for their employees. So, when it comes to Assistive Technology (AT) users, especially in the public sector industry, providing them with the necessary support to carry out their day-to-day duties is of vital importance. This is where voice recognition software such as Dragon NaturallySpeaking comes into picture. It has been developed by Nuance Communications and runs on Microsoft Windows and macOS.
The Siebel CRM application, after migrating to Open UI, has provided businesses and developers with a much wider scope of customization. Since it's primarily built on jQuery, integrating with external application through open source code has become a possibility. In this blog post, I'll explain how the Siebel CRM application can be integrated with Dragon Speech Recognition software to enable execution of the various processes and activities within CRM through voice command and also how to create such custom commands within Siebel CRM through jQuery and also through Dragon macros.
The Requirement:
A client of ours who recently upgraded to Siebel IP15 had a requirement to have a voice enabled Siebel CRM application. They have an Assistive Technology user group and a certain section of it uses voice recognition software to perform their daily official activities. Since Siebel CRM is a core application for the client and with their long standing commitment towards equality and human rights, addressing this need is of vital importance. Although the client did try out the solution of enabling voice commands for the Siebel CRM application through Windows Speech Recognition (WSR), achieving seamless speech detection turned out to be a distant dream and it also was not able to reach the different processes within CRM as creating custom command in WSR for Siebel CRM had its own limitations.
The Challenge:
Dragon software doesn't work adequately with Siebel IP15 on
it's out of box setup. This is because:
·
Dragon is created to tag/access a limited set of HTML elements
through a very specific set of attributes.
·
Dragon then makes use of those HTML attributes to flag the
elements through commands created using macros.
·
The Out-of-box Siebel application, however, has a structure
which is built by making use of a vast set of HTML elements, designed to be
accessed through keyboard and mouse, but not by voice commands.
·
So, by default, Siebel doesn't have most of the HTML attributes
required by Dragon to flag elements and a lot of the elements in Siebel Open UI
are not supported in Dragon.
The Solution:
In order to make Dragon work, we have to customize Siebel
Open UI through JavaScript (.js) files, to support and allow Dragon macros to
flag the different elements of Siebel. This would enable users to access all
different functionalities of Siebel which are currently not supported in IP15.
Dragon provides the necessary guidelines to enable an HTML page to function well with its program. I will not get into the details in regards to how and why an HTML element needs to be constructed in dragon in a specific fashion, as you can find the detailed documentation regarding the same in Dragon's website. The basic idea is that Dragon commands can associate to specific HTML element types which are:
- Anchor Elements
- Image & Imagemap Links
- Buttons
- Edit/Input Controls
- Text Areas
- List Items or Select boxes
- Label Elements
- Frames
By default, the
click/selection event on every control is triggered through a command preceded
with the word 'Click', which can be disabled from the Commands tab
under the Options dialog box of Dragon.
Some of the key points to keep in mind while working with Dragon are as follows:
1.
The first and the most important point to remember is that to
make an element accessible by speech, their needs to be some text associated
with it, which will be used as part of the voice command. For an element which
has readable text in the forefront, say link or button, the associated command
to access it through voice can be the intrinsic text itself. If no text for a
control is displayed on the UI, they need to set to the ALT or TITLE attribute
of the element. In case of input elements, the NAME attribute
can also be used to set up the spoken text. In case of duplicate associated
texts, all elements with the same text will be flagged, with numbers displayed
on screen to differentiate the controls.
2.
The elements which do not belong to the stated list of
permissible controls can be accessed by modifying the role attribute.
Say, for example, we have a div element and we want to trigger a click event on
it. We can convert the same to be identified by Dragon as a button/text/link by
setting the role attribute for the same as button/text/link.
3.
It is also noteworthy to avoid special characters in the text
associated to a control if it's supposed to form the voice command. For
example, Dragon won't be able to recognise a text such as "Save &
Close" on a button; instead it should be changed to "Save and
Close".
With these aspects in mind, we will need to modify the
Siebel Open UI application's HTML structure, to make it accessible by voice.
Note that, the same cannot be applied to the HI application as it doesn't allow
UI modification to set up custom voice commands. Also, since the HI application
is based on ActiveX, the ActiveX controls in the application cannot be
associated to voice commands as they technically do not lend themselves to
speech recognition.
Some of the more generic elements won't require any
modification, such as Screen/View tabs, OK/Go buttons, hyperlinks, etc. and can
be accessed through the provided text. But these elements mostly constitute the
navigation and querying aspects of the application but will not allow users to
carry out complex functionalities.
Figure 1: Architecture of Siebel Open UI Integration with Dragon
Siebel Open UI constructs the UI layer through the physical renderer
files. The primary file to control most of the HTML structure is "htmltmplmgr.js". So, to modify the
controls, we would need to add "alt"
or "title" attributes to the
elements we want to control. Since this is a core OOTB file, and extending the
same isn't possible, we need to be careful while modifying it. An ideal
approach would be to separate out the code based on a flag, which can be set
based on responsibility or a click event. This would avoid any potential
conflicts with the existing functionalities which might occur. It must be taken
into account that since we are modifying a vanilla file, any new patch or an
upgrade to the current version would probably replace the file with a newer
version. In such cases, we would need to merge the custom code to the new file.
Similarly, we can customise different elements by modifying
the code from other vanilla files. This should be done by extending the vanilla
file, wherever possible, such as the "physicalrender.js"
file which lends itself for extension. There are certain elements or features
which are problematic for speech recognition, such as the scroll bar, in which
case we need to create a permitted custom element in the foreground and set up
the click event to handle the desired effect programmatically.
In case of list applets, we have <th> and <td> tags which compose the headers and columns of the table respectively. For obvious reasons, these elements cannot be replaced and hence, need to be modified by changing the role attribute from "gridcell" to "button" (as changing them to text or link would create conflicts with other elements displayed in the table). Similarly, there are <div> elements throughout the application whose role either needs to be added or changed to allow Dragon to access them through voice.
Figure 2: Indicates all HTML elements which are links being flagged with
Click Link command
Now, all elements
which are permissible by Dragon for association can also be accessed by element
name. For example, "Click Link", "Click Button",
"Click Image" etc. will flag all such elements visible on the UI and
number them accordingly, as shown in Figure 2, for users to choose the desired
the element. So, it's recommended to set unique association text through
alt/title/name for the distinct elements used to perform different functions in
order to reduce the time taken by Dragon to process the HTML elements which it
does every time a user starts speaking.
We also have
several keyboard shortcuts in the Siebel CRM application. Siebel CRM allows us
to configure new shortcuts for custom or vanilla functionalities through Accelerators of
the Command object type, as well. We can create custom voice commands
for these shortcuts through the Dragon tool itself. The software allows macros
to be created from the front end which can be done from the Command
Browser under Tools. These macros can be exported to a data (.dat)
file which can then be deployed either as a patch for Dragon or be imported by
the users themselves.
In Conclusion:
One of the primary features of Siebel Open UI is its
flexibility towards customisation which enables us to design a more interactive
UI and also to incorporate external applications such as Dragon to support the
needs of Assistive Technology users. Since, it allows implementation of open
source code, the solution provided in the above description can be approached
in more than one way. We have had multiple customisations done on the CRM
application since the initial development to assist AT users in accessing
different features of Siebel and also to modify certain processes to better
serve their needs.
Although, the above solution is provided for integrating
Dragon with Siebel CRM, the same can be applied to any other HTML based
application, which allows modification to the UI layer.
I hope to have covered most of the fundamental details in my explanation. However, if you would like to discuss further on the topic, feel free to comment below.