UI Automation and Active Accessibility
Microsoft Active Accessibility is the legacy API that was introduced in Windows 95, and was designed to make Windows applications accessible. Microsoft UI Automation is the new accessibility model for Microsoft Windows and is intended to address the needs of assistive technology products and automated testing tools. UI Automation offers many improvements over Microsoft Active Accessibility. This topic explains the differences between the two technologies.
This topic contains the following sections.
- Programming Languages
- Servers and Clients
- UI Elements
- Tree Views and Navigation
- Roles and Control Types
- States and Properties
- Events
- Accessing Active Accessibility Properties and Objects from UI Automation
- Related Topics
Programming Languages
Microsoft Active Accessibility is based on the Component Object Model (COM) with support for dual interfaces, and therefore, is programmable in C/C++, Microsoft Visual Basic 6.0, and scripting languages.
When UI Automation was introduced, the client API was limited to managed code, while the provider API included both managed and unmanaged implementations. With Windows 7, a new COM-based client API was introduced to make it easier to program UI Automation client applications in C/C++.
Servers and Clients
In Microsoft Active Accessibility, servers and clients communicate directly, largely through the server implementation of the IAccessible interface.
In UI Automation, a core service lies between the server (provider) and the client. The core service makes calls to the interfaces implemented by providers and provides additional services, such as generating unique run-time identifiers for user interface (UI) elements. Client applications gain access to this core service by creating a CUIAutomation object. This object supports a set of client interfaces that are separate from the provider interfaces. For more information, see Creating the CUIAutomation Object.
UI Automation providers can provide information to Microsoft Active Accessibility clients, and Microsoft Active Accessibility servers can provide information to UI Automation client applications. However, because Microsoft Active Accessibility does not expose as much information as UI Automation, the two models are not fully compatible.
UI Elements
Microsoft Active Accessibility presents a UI element as an IAccessible interface paired with a child identifier. It is difficult to compare two IAccessible pointers to determine if they refer to the same element.
In UI Automation, every element is represented as an object that exposes the IUIAutomationElement interface to clients. Elements can be compared by their run-time identifiers, which are retrieved by using IUIAutomationElement::GetRuntimeId.
Tree Views and Navigation
The UI elements on the screen can be seen as a tree structure with the desktop as the root, application windows as immediate children, and elements within applications as further descendants.
In Microsoft Active Accessibility, many UI elements that are irrelevant to end users are exposed in the tree structure. Client applications must examine all elements in the tree to determine which elements are meaningful.
UI Automation client applications see the UI through a filtered view. The view contains only elements that give information to the user or that the user can interact with. Predefined views that include only control elements and only content elements are available, and client applications can define custom views. UI Automation makes it easier to describe the UI to the user, and to help the user interact with applications.
In Microsoft Active Accessibility, navigation between elements is spatial, for example, moving to the element that lies to the left on the screen, logical, for example, moving to the next menu item or the next item in the tab order in a dialog box, or hierarchical, for example, moving to the first child element in a container or from a child element to its parent element. Hierarchical navigation is complicated by the fact that child elements are not always objects that implement IAccessible.
In UI Automation, all UI elements are COM objects that expose the IUIAutomationElement interface and support the same basic functionality. From the standpoint of the provider, COM objects implement an interface that is inherited from IRawElementProviderSimple. Navigation is mainly hierarchical; that is, from parents to children, and from one sibling to the next. However, navigation between siblings has a logical element, as it may follow the tab order. A client can navigate from any starting-point, using any filtered view of the tree, by using IUIAutomationTreeWalker. A client can also navigate to particular children or descendants by using IUIAutomationElement::FindFirst and IUIAutomationElement::FindAll. For example, it is easy to retrieve all elements in a dialog box that support a specified control pattern.
Navigation in UI Automation is more consistent than in Microsoft Active Accessibility. Some elements, such as drop-down lists and pop-up windows, appear twice in the Microsoft Active Accessibility tree, and navigation from those elements may have unexpected results. It is difficult to implement Microsoft Active Accessibility properly for a rebar control. UI Automation enables reparenting and repositioning, so that an element can be placed anywhere in the tree, despite the hierarchy imposed by ownership of windows.
Roles and Control Types
Microsoft Active Accessibility uses the accRole property (IAccessible::get_accRole) to retrieve a description of the element role in the UI, such as ROLE_SYSTEM_SLIDER or ROLE_SYSTEM_MENUITEM. The role of an element is the main clue to its available functionality. Interaction with a control is achieved by using fixed methods such as IAccessible::accSelect and IAccessible::accDoDefaultAction. The interaction between the client application and the UI is limited to what can be done through IAccessible.
In contrast, UI Automation decouples the control type of the element, which is described by the IUIAutomationElement::CurrentControlType property, from its expected functionality. Functionality is determined by the control patterns that are supported by the provider through its implementation of specialized interfaces. Control patterns can be combined to describe the full set of functionality that is supported by a particular UI element. Some providers are required to support a particular control pattern. For example, the provider for a check box must support the Toggle control pattern. Other providers are required to support one or more of a set of control patterns. For example, a button must support either the Toggle or the Invoke control pattern. Still others support no control patterns. For example, a pane that cannot be moved, resized, or docked has no control patterns.
UI Automation supports custom controls, which are identified by the UIA_CustomControlTypeId constant and can be described by the IUIAutomationElement::CurrentLocalizedControlType property.
The following table maps Microsoft Active Accessibility roles to UI Automation control types.
Active Accessibility role | UI Automation control type |
---|---|
ROLE_SYSTEM_PUSHBUTTON | Button |
ROLE_SYSTEM_CLIENT | Calendar |
ROLE_SYSTEM_CHECKBUTTON | CheckBox |
ROLE_SYSTEM_COMBOBOX | ComboBox |
ROLE_SYSTEM_CLIENT | Custom |
ROLE_SYSTEM_LIST | DataGrid |
ROLE_SYSTEM_LISTITEM | DataItem |
ROLE_SYSTEM_DOCUMENT | Document |
ROLE_SYSTEM_TEXT | Edit |
ROLE_SYSTEM_GROUPING | Group |
ROLE_SYSTEM_LIST | Header |
ROLE_SYSTEM_COLUMNHEADER | HeaderItem |
ROLE_SYSTEM_LINK | Hyperlink |
ROLE_SYSTEM_GRAPHIC | Image |
ROLE_SYSTEM_LIST | List |
ROLE_SYSTEM_LISTITEM | ListItem |
ROLE_SYSTEM_MENUPOPUP | Menu |
ROLE_SYSTEM_MENUBAR | MenuBar |
ROLE_SYSTEM_MENUITEM | MenuItem |
ROLE_SYSTEM_PANE | Pane |
ROLE_SYSTEM_PROGRESSBAR | ProgressBar |
ROLE_SYSTEM_RADIOBUTTON | RadioButton |
ROLE_SYSTEM_SCROLLBAR | ScrollBar |
ROLE_SYSTEM_SEPARATOR | Separator |
ROLE_SYSTEM_SLIDER | Slider |
ROLE_SYSTEM_SPINBUTTON | Spinner |
ROLE_SYSTEM_SPLITBUTTON | SplitButton |
ROLE_SYSTEM_STATUSBAR | StatusBar |
ROLE_SYSTEM_PAGETABLIST | Tab |
ROLE_SYSTEM_PAGETAB | TabItem |
ROLE_SYSTEM_TABLE | Table |
ROLE_SYSTEM_STATICTEXT | Text |
ROLE_SYSTEM_INDICATOR | Thumb |
ROLE_SYSTEM_TITLEBAR | TitleBar |
ROLE_SYSTEM_TOOLBAR | ToolBar |
ROLE_SYSTEM_TOOLTIP | ToolTip |
ROLE_SYSTEM_OUTLINE | Tree |
ROLE_SYSTEM_OUTLINEITEM | TreeItem |
ROLE_SYSTEM_WINDOW | Window |
States and Properties
Microsoft Active Accessibility elements support a common set of properties. Some properties, such as accState, must describe different conditions, depending on the element role. Servers must implement all methods of IAccessible that return a property, even those properties that are not relevant to the element.
UI Automation defines additional properties, some of which correspond to states in Microsoft Active Accessibility. Some properties are common to all elements, but other properties are specific to control types and control patterns. A UI Automation provider does not have to implement irrelevant properties, but can return a null value for any properties it does not support. The UI Automation core service can obtain some properties from the default window provider, and these are amalgamated with properties explicitly implemented by the provider.
As well as supporting many more properties, UI Automation enables better performance by allowing properties to be cached.
The following table shows the correspondence between some properties in the two models.
Active Accessibility property accessor | UI Automation property ID | Remarks |
---|---|---|
get_accKeyboardShortcut | UIA_AccessKeyPropertyId or UIA_AcceleratorKeyPropertyId | UIA_AccessKeyPropertyId takes precedence if both are present. |
get_accName | UIA_NamePropertyId | |
get_accRole | UIA_ControlTypePropertyId | See the previous table for mapping roles to control types. |
get_accValue | UIA_ValueValuePropertyId or UIA_RangeValueValuePropertyId | Valid only for control types that support IUIAutomationValuePattern or IUIAutomationRangeValuePattern. Range values are normalized to 0-100, to be consistent with Microsoft Active Accessibility behavior. Values are represented as strings. |
get_accHelp | UIA_HelpTextPropertyId | |
accLocation | BoundingRectangleProperty | |
get_accDescription | Not supported. | accDescription did not have a clear specification in Microsoft Active Accessibility, which resulted in servers placing different pieces of information in this property. |
get_accHelpTopic | Not supported. |
The following table shows the UI Automation properties that correspond to Microsoft Active Accessibility state constants.
Active Accessibility state | UI Automation property | Triggers WinEvent state change? |
---|---|---|
STATE_SYSTEM_CHECKED | UIA_ToggleToggleStatePropertyId for check box. UIA_SelectionItemIsSelectedPropertyId for radio button. | Y |
STATE_SYSTEM_COLLAPSED | UIA_ExpandCollapseExpandCollapseStatePropertyId (value = ExpandCollapseState_Collapsed). | Y |
STATE_SYSTEM_EXPANDED | UIA_ExpandCollapseExpandCollapseStatePropertyId (value = ExpandCollapseState_Expanded or ExpandCollapseState_PartiallyExpanded). | Y |
STATE_SYSTEM_FOCUSABLE | UIA_IsKeyboardFocusablePropertyId. | N |
STATE_SYSTEM_FOCUSED | UIA_HasKeyboardFocusPropertyId. | N |
STATE_SYSTEM_HASPOPUP | UIA_ExpandCollapsePattern for menu items. | N |
STATE_SYSTEM_INVISIBLE | UIA_IsOffscreenPropertyId (value = True and IUIAutomationElement::GetClickablePoint fails). | N |
STATE_SYSTEM_LINKED | UIA_ControlTypePropertyId (value = UIA_HyperlinkControlTypeId). | N |
STATE_SYSTEM_MIXED | UIA_ToggleStateId (value = ToggleState_Indeterminate. | N |
STATE_SYSTEM_MOVEABLE | UIA_CanMovePropertyId | N |
STATE_SYSTEM_MUTLISELECTABLE | UIA_CanSelectMultiplePropertyId. | N |
STATE_SYSTEM_OFFSCREEN | UIA_IsOffscreenPropertyId. | N |
STATE_SYSTEM_PROTECTED | UIA_IsPasswordPropertyId. | N |
STATE_SYSTEM_READONLY | UIA_RangeValueIsReadOnlyPropertyId and UIA_ValueIsReadOnlyPropertyId. | N |
STATE_SYSTEM_SELECTABLE | SelectionItemPattern is supported. | N |
STATE_SYSTEM_SELECTED | UIA_SelectionItemIsSelectedPropertyId. | N |
STATE_SYSTEM_SIZEABLE | UIA_TransformCanResizePropertyId. | N |
STATE_SYSTEM_UNAVAILABLE | UIA_IsEnabledPropertyId. | Y |
For a complete list of property identifiers, see Property Identifiers.
Events
Unlike Microsoft Active Accessibility, the event mechanism in UI Automation, does not rely on Windows event routing, which is closely tied to window handles, and does not require the client application to set up hooks. Subscriptions to events can be fine-tuned to particular parts of the tree, not just to particular events. Providers can also fine-tune raising events by keeping track of which events are being listened for.
It is also easier for clients to retrieve the elements that raise events because these are passed directly to the event callback. Properties of the element are prefetched automatically, if a cache request was supplied when the client subscribed to the event.
The following table shows the correspondence of Microsoft Active Accessibility WinEvents and UI Automation events.
WinEvent | UI Automation Event ID |
---|---|
EVENT_OBJECT_ACCELERATORCHANGE | UIA_AcceleratorKeyPropertyId property change. |
EVENT_OBJECT_CONTENTSCROLLED | UIA_VerticalScrollPercentPropertyId or UIA_HorizontalScrollPercentPropertyId property change on the associated scroll bars. |
EVENT_OBJECT_CREATE | UIA_StructureChangedEventId. |
EVENT_OBJECT_DEFACTIONCHANGE | No equivalent. |
EVENT_OBJECT_DESCRIPTIONCHANGE | No exact equivalent; perhaps UIA_HelpTextPropertyId or UIA_LocalizedControlTypePropertyId property change. |
EVENT_OBJECT_DESTROY | UIA_StructureChangedEventId. |
EVENT_OBJECT_FOCUS | UIA_AutomationFocusChangedEventId. |
EVENT_OBJECT_HELPCHANGE | UIA_HelpTextPropertyId change. |
EVENT_OBJECT_HIDE | UIA_StructureChangedEventId. |
EVENT_OBJECT_LOCATIONCHANGE | UIA_BoundingRectanglePropertyId property change. |
EVENT_OBJECT_NAMECHANGE | UIA_NamePropertyId property change. |
EVENT_OBJECT_PARENTCHANGE | UIA_StructureChangedEventId. |
EVENT_OBJECT_REORDER | Not consistently used in Microsoft Active Accessibility. No directly corresponding event is defined in UI Automation. |
EVENT_OBJECT_SELECTION | UIA_SelectionItem_ElementSelectedEventId. |
EVENT_OBJECT_SELECTIONADD | UIA_SelectionItem_ElementAddedToSelectionEventId. |
EVENT_OBJECT_SELECTIONREMOVE | UIA_SelectionItem_ElementRemovedFromSelectionEventId. |
EVENT_OBJECT_SELECTIONWITHIN | No equivalent. |
EVENT_OBJECT_SHOW | UIA_StructureChangedEventId. |
EVENT_OBJECT_STATECHANGE | Various property-changed events. |
EVENT_OBJECT_VALUECHANGE | UIA_RangeValueProperty and UIA_ValueValuePropertyId changed. |
EVENT_SYSTEM_ALERT | No equivalent. |
EVENT_SYSTEM_CAPTUREEND | No equivalent. |
EVENT_SYSTEM_CAPTURESTART | No equivalent. |
EVENT_SYSTEM_CONTEXTHELPEND | No equivalent. |
EVENT_SYSTEM_CONTEXTHELPSTART | No equivalent. |
EVENT_SYSTEM_DIALOGEND | UIA_WindowClosedEventId. |
EVENT_SYSTEM_DIALOGSTART | UIA_WindowOpenedEventId. |
EVENT_SYSTEM_DRAGDROPEND | No equivalent. |
EVENT_SYSTEM_DRAGDROPSTART | No equivalent. |
EVENT_SYSTEM_FOREGROUND | UIA_AutomationFocusChangedEventId. |
EVENT_SYSTEM_MENUEND | UIA_MenuClosedEventId. |
EVENT_SYSTEM_MENUPOPUPEND | UIA_MenuClosedEventId. |
EVENT_SYSTEM_MENUPOPUPSTART | UIA_MenuOpenedEventId. |
EVENT_SYSTEM_MENUSTART | UIA_MenuOpenedEventId. |
EVENT_SYSTEM_MINIMIZEEND | UIA_Window_WindowVisualState_PropertyId property change. |
EVENT_SYSTEM_MINIMIZESTART | UIA_Window_WindowVisualState_PropertyId property change. |
EVENT_SYSTEM_MOVESIZEEND | UIA_BoundingRectanglePropertyId property change. |
EVENT_SYSTEM_MOVESIZESTART | UIA_BoundingRectanglePropertyId property change. |
EVENT_SYSTEM_SCROLLINGEND | UIA_ScrollVerticalScrollPercentPropertyId or UIA_ScrollHorizontalScrollPercentPropertyId property change. |
EVENT_SYSTEM_SCROLLINGSTART | UIA_ScrollVerticalScrollPercentPropertyId or UIA_ScrollHorizontalScrollPercentPropertyId property change. |
EVENT_SYSTEM_SOUND | No equivalent. |
EVENT_SYSTEM_SWITCHEND | No equivalent, but a UIA_AutomationFocusChangedEventId event signals that a new application has received the focus. |
EVENT_SYSTEM_SWITCHSTART | No equivalent. |
No equivalent. | UIA_MultipleViewCurrentViewPropertyId property change. |
No equivalent. | UIA_ScrollHorizontallyScrollablePropertyId property change. |
No equivalent. | UIA_ScrollVerticallyScrollablePropertyId property change. |
No equivalent. | UIA_ScrollHorizontalScrollPercentPropertyId property change. |
No equivalent. | UIA_ScrollVerticalScrollPercentPropertyId property change. |
No equivalent. | UIA_Scroll_HorizontalViewSize_PropertyId property change. |
No equivalent. | UIA_Scroll_VerticalViewSize_PropertyId property change. |
No equivalent. | UIA_ToggleToggleStatePropertyId property change. |
No equivalent. | UIA_WindowWindowVisualStatePropertyId property change |
No equivalent. | UIA_AsyncContentLoadedEventId event. |
No equivalent. | UIA_ToolTipOpenedEventId event. |
Accessing Active Accessibility Properties and Objects from UI Automation
A key feature of UI Automation that is not available in Microsoft Active Accessibility is the ability to fetch multiple properties with a single cross-process operation.
Existing Microsoft Active Accessibility clients can take advantage of this ability by using the IUIAutomationLegacyIAccessiblePattern interface. This interface represents a control pattern that exposes Microsoft Active Accessibility properties and methods on UI elements. When retrieving elements, an application can request that this control pattern and its properties be cached.
IUIAutomationLegacyIAccessiblePattern also enables clients to obtain Microsoft Active Accessibility properties from elements that have no native support for IAccessible.
Changes in the properties of an IUIAutomationLegacyIAccessiblePattern do not raise UI Automation events.