[jawsscripts] UIA
- From: "Travis Roth" <travis@xxxxxxxxxxxxxx>
- To: <jawsscripts@xxxxxxxxxxxxx>
- Date: Thu, 19 Mar 2015 10:15:03 -0500
Hi,
Following up on the below post, has anyone found good tutorial type
documentation? It'd be ideal to have some task-based workflows to follow I
think.
For example, how does one use FindAll() given the only thing you know is
that you have an application window and somewhere in it is a certain type of
control? For example, say you have Excel and you want to get the worksheet
tabs at the bottom of the window.
-----Original Message-----
From: jawsscripts-bounce@xxxxxxxxxxxxx
[mailto:jawsscripts-bounce@xxxxxxxxxxxxx] On Behalf Of Stefan Moisei
Sent: Monday, March 16, 2015 3:25 PM
To: jawsscripts@xxxxxxxxxxxxx
Subject: [jawsscripts] Re: Learning More About Scripting and modifying a
keystroke for a specific application
Here's a post from Doug Lee, one of the greatest scripters in my opinion,
that talks about this problem. I completely agree with him. I find myself in
the position of scripting visual studio and found out I really need UIA to
do something meaningful.
For years I have complained, as many probably have, that it is hard to know
how to start off a new JAWS scripter in a way that will empower him or her
to script the average application without getting hopelessly lost or buried
in details. A common initial approach has been to teach window hierarchy
navigation and cursor manipulation, but these techniques increasingly fail
to be sufficient to fix major problems in current applications. This leaves
new scripters stuck in dead ends wondering where to go next. I believe it is
time to change where we start them off.
I believe, starting in recent JAWS 15 updates, that we finally have one
consistent, all-empowering path for scripters to take. There are of course
caveats, but I'll get to that. This path will give scripters access to many
things that previously required entire different sets of function calls -
window tree functions here, MSAA functions there, HTML DOM access, etc.
JAWS 15, particularly starting with the 15.0.5056 update, introduces direct
scripter access to the Microsoft User Interface Automation (UIA) API. This
single API provides a way to get to pretty much everything else one would
usually need: window tree information, MSAA properties, information from
HTML nodes, and of course, native UIA support for newer platforms like WPF
and Silverlight. I regard the current interface as complicated, I admit; but
I think it is worth learning for any JAWS scripter because of how much it
can do.
Wherever JAWS itself provides ways to get information not provided by this
UIA interface, the UIA interface can help you find where to start using
other methods. For example, if you need to test the style bits of a window
though the UIA interface does not provide a means for this, you can find the
window's UIA object via the UIA interface, get the nativeWindowHandle for
it, and then call the JAWS getWindowStyleBits() function on that handle. In
other words, whereas we used to think of navigating the window tree as the
first step to finding things, I now think we should consider navigating the
UIA tree the first step. What we do from there, in both cases, will depend
on what we need.
The definitive reference document for the JAWS UIA interface is located at
http://www.freedomscientific.com/documentation/scripts/JAWS-UIAScriptAPI.asp
.
It is likely to be a tough read for anyone who is not already familiar with
objects and object-oriented programming, so I imagine my post is going to
start a lot of chatter on this list about how this system works.
The caveats:
1. As I said, it's complicated: Scripters must understand some concepts of
object-oriented programming and COM in order to use this system. Objects, in
JAWS scripting at least, have long been considered an advanced topic. I say
it's time to consider them central to scripting.
2. It can be slow. Wise scripters should minimize UIA queries because of
this, by caching results, asking for as few nodes as possible by crafting
searches carefully, etc.
3. As already mentioned, this system will not work before JAWS 15.0.5056.
(The fine print: Some of this works as far back as JAWS 14, but several
things aren't there that far back, some key items are there but with
different names, etc.) This means that you can't write scripts this way if
you want compatibility with old JAWS versions.
4. As the aforementioned Freedom Scientific reference document for the UIA
script API says, some of the features of UIA do not work on all Windows
versions. The document implies that support for this system may start in
Windows 7, but I have successfully used much of this under Windows Vista and
even XP.
In conclusion:
I do not think we are at the point of saying the old ways of navigating
window trees, working directly with MSAA objects, etc. are obsolete and
useless. I simply think the new UIA interface is the single most promising
place for a new scripter to begin, since it doesn't miss much in terms of
what it can let you find. Direct window tree traversal and FindWindow calls,
MSAA getFocusObject calls and direct MSAA object access, etc., will
sometimes run faster and should not be dismissed as valid pursuits. The UIA
interface, though, certainly seems sufficient for solving many current
scripting problems, even if in a "fix first, optimize later" sort of way.
-----Original Message-----
From: Robert Spangler
Sent: Saturday, March 14, 2015 9:03 PM
To: jawsscripts@xxxxxxxxxxxxx
Subject: [jawsscripts] Learning More About Scripting and modifying a
keystroke for a specific application
Hello:
I have just joined this list. I am looking to expand my JAWS scripting
knowledge and experience beyond what is provided in the scripting tutorial
from FS. Are there any classes or additional books/documents that I can use
for furthering my scripting knowledge? In addition, are there scripting
certifications such as the one provided for knowing how to operate JAWS?
Also, if I wish to change what a keystroke does in a particular application,
how would I do that? The particular example is within iTunes, where
alt+ctrl+right/left arrows are assigned to rewind/fast forward but are
alt+ctrl+table
navigations in JAWS.
Thanks for your responses!
Robert
__________�
View the list's information and change your settings at
http://www.freelists.org/list/jawsscripts
---
This email is free from viruses and malware because avast! Antivirus
protection is active.
http://www.avast.com
__________�
View the list's information and change your settings at
http://www.freelists.org/list/jawsscripts
__________�
View the list's information and change your settings at
http://www.freelists.org/list/jawsscripts
Other related posts: