[jawsscripts] Re: UIA

  • From: Soronel Haetir <soronel.haetir@xxxxxxxxx>
  • To: jawsscripts@xxxxxxxxxxxxx
  • Date: Thu, 19 Mar 2015 11:09:35 -0800

I am not aware of any jaws UIA introduction in particular, but do
believe the UIA reference mentioned in the quoted post along with the
basics of scripting manual along with the much expanded information to
be found in recent versions of FSDN should be sufficient.

On 3/19/15, Travis Roth <travis@xxxxxxxxxxxxxx> wrote:
> Hi,
> Following up on the below post, has anyone found good tutorial type
> documentation? It'd be ideal to have some task-based workflows to follow I
> think.
> For example, how does one use FindAll() given the only thing you know is
> that you have an application window and somewhere in it is a certain type
> of
> control? For example, say you have Excel and you want to get the worksheet
> tabs at the bottom of the window.
>
>
> -----Original Message-----
> From: jawsscripts-bounce@xxxxxxxxxxxxx
> [mailto:jawsscripts-bounce@xxxxxxxxxxxxx] On Behalf Of Stefan Moisei
> Sent: Monday, March 16, 2015 3:25 PM
> To: jawsscripts@xxxxxxxxxxxxx
> Subject: [jawsscripts] Re: Learning More About Scripting and modifying a
> keystroke for a specific application
>
> Here's a post from Doug Lee, one of the greatest scripters in my opinion,
> that talks about this problem. I completely agree with him. I find myself
> in
> the position of scripting visual studio and found out I really need UIA to
> do something meaningful.
>
> For years I have complained, as many probably have, that it is hard to know
> how to start off a new JAWS scripter in a way that will empower him or her
> to script the average application without getting hopelessly lost or buried
> in details. A common initial approach has been to teach window hierarchy
> navigation and cursor manipulation, but these techniques increasingly fail
> to be sufficient to fix major problems in current applications. This leaves
> new scripters stuck in dead ends wondering where to go next. I believe it
> is
> time to change where we start them off.
>
> I believe, starting in recent JAWS 15 updates, that we finally have one
> consistent, all-empowering path for scripters to take. There are of course
> caveats, but I'll get to that. This path will give scripters access to many
> things that previously required entire different sets of function calls -
> window tree functions here, MSAA functions there, HTML DOM access, etc.
>
> JAWS 15, particularly starting with the 15.0.5056 update, introduces direct
> scripter access to the Microsoft User Interface Automation (UIA) API. This
> single API provides a way to get to pretty much everything else one would
> usually need: window tree information, MSAA properties, information from
> HTML nodes, and of course, native UIA support for newer platforms like WPF
> and Silverlight. I regard the current interface as complicated, I admit;
> but
> I think it is worth learning for any JAWS scripter because of how much it
> can do.
>
> Wherever JAWS itself provides ways to get information not provided by this
> UIA interface, the UIA interface can help you find where to start using
> other methods. For example, if you need to test the style bits of a window
> though the UIA interface does not provide a means for this, you can find
> the
> window's UIA object via the UIA interface, get the nativeWindowHandle for
> it, and then call the JAWS getWindowStyleBits() function on that handle.
> In
> other words, whereas we used to think of navigating the window tree as the
> first step to finding things, I now think we should consider navigating the
> UIA tree the first step. What we do from there, in both cases, will depend
> on what we need.
>
> The definitive reference document for the JAWS UIA interface is located at
> http://www.freedomscientific.com/documentation/scripts/JAWS-UIAScriptAPI.asp
> .
> It is likely to be a tough read for anyone who is not already familiar with
> objects and object-oriented programming, so I imagine my post is going to
> start a lot of chatter on this list about how this system works.
>
> The caveats:
>
> 1. As I said, it's complicated: Scripters must understand some concepts of
> object-oriented programming and COM in order to use this system. Objects,
> in
> JAWS scripting at least, have long been considered an advanced topic. I say
> it's time to consider them central to scripting.
>
> 2. It can be slow. Wise scripters should minimize UIA queries because of
> this, by caching results, asking for as few nodes as possible by crafting
> searches carefully, etc.
>
> 3. As already mentioned, this system will not work before JAWS 15.0.5056.
> (The fine print: Some of this works as far back as JAWS 14, but several
> things aren't there that far back, some key items are there but with
> different names, etc.) This means that you can't write scripts this way if
> you want compatibility with old JAWS versions.
>
> 4. As the aforementioned Freedom Scientific reference document for the UIA
> script API says, some of the features of UIA do not work on all Windows
> versions. The document implies that support for this system may start in
> Windows 7, but I have successfully used much of this under Windows Vista
> and
> even XP.
>
> In conclusion:
>
> I do not think we are at the point of saying the old ways of navigating
> window trees, working directly with MSAA objects, etc. are obsolete and
> useless. I simply think the new UIA interface is the single most promising
> place for a new scripter to begin, since it doesn't miss much in terms of
> what it can let you find. Direct window tree traversal and FindWindow
> calls,
> MSAA getFocusObject calls and direct MSAA object access, etc., will
> sometimes run faster and should not be dismissed as valid pursuits. The UIA
> interface, though, certainly seems sufficient for solving many current
> scripting problems, even if in a "fix first, optimize later" sort of way.
> -----Original Message-----
> From: Robert Spangler
> Sent: Saturday, March 14, 2015 9:03 PM
> To: jawsscripts@xxxxxxxxxxxxx
> Subject: [jawsscripts] Learning More About Scripting and modifying a
> keystroke for a specific application
>
> Hello:
>
> I have just joined this list.  I am looking to expand my JAWS scripting
> knowledge and experience beyond what is provided in the scripting tutorial
> from FS.  Are there any classes or additional books/documents that I can
> use
> for furthering my scripting knowledge?  In addition, are there scripting
> certifications such as the one provided for knowing how to operate JAWS?
>
> Also, if I wish to change what a keystroke does in a particular
> application,
> how would I do that?  The particular example is within iTunes, where
> alt+ctrl+right/left arrows are assigned to rewind/fast forward but are
> alt+ctrl+table
> navigations in JAWS.
>
> Thanks for your responses!
> Robert
>
>
> __________�
>
> View the list's information and change your settings at
> http://www.freelists.org/list/jawsscripts
>
>
> ---
> This email is free from viruses and malware because avast! Antivirus
> protection is active.
> http://www.avast.com
>
> __________�
>
> View the list's information and change your settings at
> http://www.freelists.org/list/jawsscripts
>
>
> __________�
>
> View the list's information and change your settings at
> http://www.freelists.org/list/jawsscripts
>
>


-- 
Soronel Haetir
soronel.haetir@xxxxxxxxx
__________�

View the list's information and change your settings at 
http://www.freelists.org/list/jawsscripts

Other related posts: