3D Mice

A White Paper by Spatial Freedom Pty Ltd

July 2014
www.spatialfreedom.com

Background

In the early to mid 1980’s as 3D Computer Aided Design (CAD) systems rapidly gained popularity many users began to express the need for a better and more intuitive input or control device. Whilst the traditional mouse was more than adequate for "pointing" and selection functions, it lacked the 3D interactivity that new generation, solid modelling based CAD systems required. At this time (mid 1980’s) the mouse had become the industry standard input device for computer applications, however, the mouse was largely an input device for 2D applications like word processing, spreadsheets, 2D graphic arts, etc. The CAD industry itself was maturing from a 2D application, where a range of input devices such as data tablets, digitizes, etc had traditionally been used, but now a more productive and ergonomic "3D" control devices was needed. Early 3D input device like control dial boxes began to emerge but none of them gained wide user acceptance.

A History of 3D Mice

The Early Days

Back in 1983 John Hilton (who later invented the Spaceball), embarked on a master’s degree in mechanical engineering at the University of Sydney, Australia. He had just completed five years of study gaining a Computer Science degree and a Mechanical Engineering degree.

Around that time the Mechanical Engineering department acquired a ComputerVision CADDS4 system. Mr. Hilton’s unusual combination of degrees coupled with the availability of a CAD system naturally led to a keen interest in CAD.

In studying the various 3D CAD systems available at the time it puzzled Mr. Hilton as to why they all used 2D input devices and dial boxes for controlling the 3D image. Researching various literature he found several types of 3D and 6D devices going back to 1965 where Ivan Sutherland had developed a head mounted display - the forerunner for today’s cyberspace head mounted displays. (6D devices provide simultaneous 3D translate and 3D rotate control.) The devices allowed a handle of some description to be moved around with the hand and the image on the screen to move correspondingly. It appeared researchers were following the thought, "If only I could hold it in my hand." But these devices were only used in research labs and never in commercial applications. Cost was possibly the barrier but even the military, known to be cost-insensitive, did not use any of these devices for non-research purposes.

Mr. Hilton came to the conclusion that ergonomics was the issue as it was not practical to use these non-ergonomic 6D digitizers for CAD users. It was impractical to use a device that had to be held in the air for more than ten minutes let alone an entire working day. To solve the problem Mr. Hilton conceived of a static grip, preferably a ball, which sensed a 3D push for moving a 3D cursor around in several views. To design such a device Mr. Hilton considered the ‘flow of force’ from the grip to the base and wanted to detect all components of this flow. To detect all the components required the detection of the 3D torque as well. Drawing heavily on the Theory of Mechanism and Machines, Mr. Hilton evolved a ‘perfectly constrained’ mechanism with three arms, three hinges and three ball joints to sense and measure this flow of force.

There are six degrees of freedom between any two rigid bodies, three translational and three rotational. Each basic mechanical joint between two rigid bodies has between 1 and 5 degrees of freedom and between 1 and 5 degrees of constraint - the sum of these always being six. A perfectly constrained body is one where the joints provide exactly six degrees of constraint. Less constraint and the body can move, more and the body is over-constrained.

It was difficult finding a low cost sensor to measure the small displacements of this fledgling invention. Again the literature was researched and all the basic sensing approaches of inductive, capacitive, resistive, optical fringe counting, interference gratings and optical mask sensing were considered. Unfortunately suitable commercial sensors proved to be quite expensive. Being a poor but innovative university student Mr. Hilton discovered a cheap light dependent resistor (LDR) from a local electronics store and a 12V lamp from a car’s interior light produced an accurate result. It actually detected loud sounds in the room as well! The first working device consisted of a very crude prototype interfaced to a home computer that Mr. Hilton had built from a kit.

This device moved a 3D cursor, shown as a point in a top view and another point in a front view. But it only worked well in the dark! Blocking ambient light out was a challenge. In talking about the problem with another researcher they suggested the use of infrared sensors. These worked perfectly and the same style of sensor continues to be used in the Astroid today.

Mr. Hilton’s Masters’ thesis was originally titled, "A 3D Force Sensing Joystick" but ended up being "Development of a Device for Simultaneous Measurement of a Spatial Force and a Spatial Torque for Manual Control."

The 'Screwball'

About this time the device was given the name ‘Screwball’. This was derived from the Theory of Mechanism and Machines where spatial relative motion of a rigid body is called a motion screw and a spatial force and torque between two rigid bodies is called an action screw.

One of the university lecturers, aware of the unique nature of the Screwball, suggested applying for a patent and recommended a patent attorney. A provisional patent was filed in December 1985.

In preparing the provisional patent application Mr. Hilton considered how someone might design around it. The original Screwball had relatively long arms that were external to the grip. The basic geometry of the Screwball resolved the spatial force and torque into three pairs of parallel forces. The first Screwball had three of these forces intersecting at the centre of the device. By shifting the basic geometry so the parallel forces in each pair were located either side of the centre a new type of Screwball design was possible that would not have been covered by the patent draft. This modified design was included in the provisional patent and proved to be far more suitable for a commercial product.

By this stage Mr. Hilton had begun full time work for an Australian IT company that, amongst other things, distributed Evans & Sutherland (E&S) graphics terminals into Australia. One customer was a computer animation studio that used E&S terminals and had purchased one of the early cameras used for pioneering computer animation for movies. Mr. Hilton developed computer animation software to be used by animators for creating Australian and overseas computer animated television logos and commercials. This work on computer animation software dovetailed well into his continuing, but now part time, Screwball thesis work.

He showed an early prototype to two of the directors. They took an interest and provided seed funding to develop a professional prototype. This was used for demonstrations to key North American companies using E&S equipment. The response was very positive so a company was formed, a private investor found, and development of the Model 1003 began.

Commercial Reality; The ‘Spaceball’

The name, Screwball, was clearly not suitable so a list of about 25 alternative names were collected. "Spaceball" was the winning name as it is now commonly known in the CAD industry.

By August 1988 the first commercially available spatial (or motion) controller; The Spaceball 1003 was ready for shipping and Mr Hilton moved to Boston, MA in the USA to continue commercialisation of 3D mouse technology.

Over the course of the next 11 years Mr. Hilton oversaw the development of the Spaceball 2003, SpaceController, Spaceball 3003, Spaceball Avenger, SpaceOrb 360, Sphere360 and the Spaceball 4000.

The technology evolved through four generations as new, better and less expensive designs were produced. These were sold by a succession of companies; Spatial Systems Pty. Ltd., Spatial Systems Inc., Spaceball Technologies Inc., Spacetec IMC Corporation, Labtec Inc. (acquired by Logitech International S.A. in February 2001) and 3Dconnexion (LogiCAD 3D also being acquired by Logitech International S.A. in September 2001).

In 1999 Labtec Inc and Spacetec IMC merged, then in February 2001 Logitech International S.A. acquired the merged entity for US$125M. Logitech had already acquired 49% of Space Control (a.k.a. LogiCAD 3D), a smaller German 3D mouse competitor, and in September 2001 bought out the remaining equity to form 3Dconnexion Inc. as a wholly owned subsidiary with a monopoly position in the market.

A New Generation of 3D Mice

The

In 1999 having lost control of the technical development and disenchanted with the direction of the technology and the business Mr Hilton returned to Australia, eventually selling all of his interest in the business. Although Mr Hilton had created a new market with his invention of the Spaceball he felt that 3D mice had never really reached anywhere near their full potential, so once again he began researching and developing. In order for any new generation of 3D mouse to be commercially success and reach it full market potential, he set out with 4 basic aims that he felt needed to be embodied in his new generation (6th generation) 3D mice, they needed to be:

  1. Low cost, much less than the current generation of controllers.
  2. Highly reliable with greater sensitivity and control.
  3. Easy for anyone to install and interface with common computer hardware and operation systems. This needed to be based on widely available industry standards.
  4. Intuitive and based a specific modern user interface for 3D mice use with current CAD systems for greater overall ease of use and as a method to introduce need functionality (in software) not yet available on the market.

Any fundamental analysis done on an electromechanical device in order to reduce cost and increase reliability leads to a conclusion that greater simplicity of design is required. Greater simplicity of design is often achieved through a reduction in the number of parts, nothing particularly brilliant about these conclusions. The breakthrough was achieved when Mr Hilton conceived of a four arm design. The Spaceball technology had been based on six arms arranged like a 3D cross. A four arm design, arranged to form the corners of a tetrahedron, allowed considerable simplification of the construction. The new four arm design provided the additional benefit that it could be moulded in one single piece; this affords several major benefits over current spatial or motion controllers, namely:

  1. Low product and production costs.
  2. Better quality, hence higher reliability.
  3. Greater rigidity, providing a better platform for higher sensitivity and more accurate 3D mouse interaction.

Although a new and better design had been conceived and proven in production trails, in order to meet the requirements set for a new generation of 3D mice a standard systems interface mechanism and user interface still needed to be developed.

Systems Architecture

Previous generations of 3D mice developed interfaces to CAD applications through non-standard means developing proprietary computer interfaces. This makes them very expensive, overly difficult to install and maintain and is not optimal for high performance or the inclusion of additional functionality, as it’s developed.

The goal for the Astroid was to create a device that was as simple to install as the common computer mouse, basically a standard plug and play device. In the past 3D mice required specialist computer skills to install and keep running. This is particularly the case in large organisations where several hundred devices maybe installed within a computing environment, including many complex systems and applications, often causing these systems and applications to conflict with the 3D mouse and its interface software causing downtime, loss of productivity, frustration and often abandonment of the 3D mouse entirely.

Some of the communications problems have been resolved more recently with the advent of the Universal Serial Bus (USB). Yet many current USB devices still require their drivers to be installed before plugging the device in or issues arise. Labels warning the user to install the drivers first are common. To be a truly plug and play device requires much more sophisticated software. Spatial Freedom has developed a systems architecture that avoids the drivers-first issue and adds hot plugability, supports multiple logins and supports multiple Astroids. This allows the Astroid to be unplugged and plugged in at any time, even while the CAD program is running. Fast switching between multiple users is supported. Multiple Astroids support is designed in and under development to be ready for future games. The Astroid hardware is designed to load Microsoft’s own standard low level USB drivers so the hardware can be plugged in before or after any Astroid software is installed. To complete the functionality the Astroid Manager software has a service program that monitors the Astroids at all times and another program that manages the 3D programs. The user is totally unaware of how it works, it just works.

Beyond the hardware and systems interface software, the application interface, this is the biggest barrier to 3D mouse proliferation. Traditionally interfacing 3D mice to applications (like 3D CAD, or Google Earth) require the application developer to spend significant amounts of time and resource to program application specific interfaces. This cost comes with no financial return to the application developer who therefore sees no incentive to write application specific interfaces, hence low proliferation of 3D mice. The Astroid SDK (software developer’s toolkit) overcomes this problem because it reduces the time to interface the Astroid to 3D application from months (traditionally) to hours for the Astroid.

Another benefit of the SDK is that the motion algorithms required to deliver quality spatial control are non-trivial and require access to the view information and, unfortunately, there are a wide range of implementations out there for various reason. To address this issue the Astroid SDK includes two modules, Clarity™ and Agility™. Clarity defines a simple and comprehensive set of viewing parameters and Agility uses this format to deliver the quality Object, Camera and Pan/Zoom motion algorithms used in spatial control. To interface the Astroid involves writing calculations to produce Clarity’s view information from the 3D program’s view data then use Clarity’s view data to call one of Agility’s motion algorithms, and then convert the result back to the 3D program’s view data format. The result is smoother and higher fidelity 3D motion control, giving the user greater "feel" and responsiveness when manipulation 3D data.

The patented SDK technology provides one further benefit. Traditionally 3D mice place a heavy load on the CPU which degrades application performance. The Astroid SDK and its advanced motion algorithms place a very small load on the CPU, generally less the 1% of CPU load when the CPU is fully loaded and less than 10% of the CPU load generated by a traditional 2D mouse.

In summary this new system architecture provides the Astroid 6000 with several major advantages over currently available spatial or motion controllers. This means the Astroid;

  1. Is much easier to install, configure and maintain. In effect true plug and play capability.
  2. Comes pre-packaged with a software developers toolkit (SDK) which means that new applications, such as new CAD, CAM, CAE, 3D Visualisation and others can be quickly and easily interfaced.
  3. Streams data faster, more reliably and efficiently between the hardware controller and the software application, meaning greater motion sensitivity, control and accuracy.
  4. Provides consistent pan, zoom and spin responses, which translates to better responsiveness and control, with no need to adjust sensitivity depending on the model or the view.
  5. Places lower loads on the CPU, freeing critical CPU power to the application for greater productivity.

The SUI (Spatial User Interface)

Existing 3D mice provided no specific user interface for spatial or motion control at all, as is the case with the common computer mouse. Whilst this is quite acceptable for a mouse, which is mainly used for 2D applications like Microsoft Word, Excel, etc it’s an entirely different case when dealing with 3 dimensional models. Design and/or manufacturing engineers need to be able to freely and easily navigate, analyse, simulated, etc in a virtual 3 dimensional environment. It became plainly evident that a new generation user interface was required, one that allowed the user choices of navigation means and options for 3 dimensional spatial analysis and provide alternate modes of navigation and simulation. All of this required software that had not previously been available.

In 2005 Mr John Hilton finally completed such a spatial user interface and called it the SUI for; you guessed it; the Spatial User Interface. Analysing the problems associated with 3D data manipulation led to three main modes of spatial interaction; Object, Camera and Pan/Zoom control. Object control operates similarly to how we hold and manipulate objects in the real world. To deliver consistent pan, zoom and spin responses on a computer requires a 3D point called a motion handle. It is as though the Astroid ball is connected to the motion handle – panning, zooming and spinning operate about it. This keeps the user in control at all times and eliminates the ‘orbiting’ problem common with existing spatial control motion algorithms. Camera control operates as though the virtual camera is being pushed around in model space. This is similar to how games operate. Camera control is less intuitive than object control and takes a little practice to master but is quite powerful for design reviews, demonstrations and animations. Pan/Zoom control is useful when working in 2D such as drawing profiles for extrusions, etc or when working with 2D drawings. This mode alone makes the Astroid invaluable just for working with 2D drawings.

The elements of the SUI are then customized to fit into the user interface theme of a given 3D program.

The following icons are taken from the Astroid add-in for Solid Edge.

Each of the three modes of interaction; object, camera and pan/zoom, is selected by one of the three icons…

In Object Control mode the motion handle can be placed using , reset using and can also be panned around with the ball while holding the keyboard CTRL key depressed. To restrict the motion handle to the screen it can be "fenced in" using the Fence Motion Handle icon;

Camera Control has a Travel Speed setting that can be speed up or slowed down using the two icons;

Travel speed is displayed as a velocity value. For example, "2.5 in/sec" or "5 m/sec".

To aid walkthroughs there is a Prevent Tilting setting toggled with . This keeps the camera upright at all times allowing it to turn 360° left or right and 180° up and down.

The Dominant Axis setting uses only the largest X, Y or Z push or twist at any one time and is turned on and off using

The Options icon, , brings up a dialog box with options to show or hide the motion handle, the travel speed, the up direction and the view angle.

The last icon, , brings up the Spatial Freedom website.

The Astroid is Born

Finally all the ingredients necessary for the next generation 3D mouse had been developed, namely;

  1. The 6th generation Orion technology,
  2. The next generation systems architecture for interfacing to computer systems; the SDK,
  3. The Spatial User Interface; the SUI.

In June 2005 the Astroid 6000 was ready for launch, with the release of the necessary SUI elements for Solid Edge CAD system. The initial release of the Astroid for Solid Edge saw strong demand from the user community with initial sales beyond the expectations of Spatial Freedom Inc. Today the additional SUI elements for Inventor®, NX®, Solid Edge®, SolidWorks®, and Femap® (a CAE application) are now available.

Why Users Need an Astroid

Basically there are three compelling reasons why any 3D CAD user should use an Astroid they are:

  1. Increased Productivity
  2. Better Quality and more Functional Design
  3. High ROI (Return On Investment) of their application (e.g. 3D CAD) investment

Let’s look at the points above more closely.

Increased Productivity: Independent studies, Spatial Freedom research, as well as current users of the Astroid show that productivity is increased anywhere between 20% and 60% when they use an Astroid combined with a mouse as opposed to using a mouse alone. But why is this case? The most common answer is that the Astroid allows users to much more easily control how they view and navigate in and around and interact with 3D models, or as we say at Spatial Freedom, " pan, zoom and spin with ease ". This results in user spending less time (and frustration) manipulating 3D models into complex orientations in order to achieve the desired outcome.

The Astroid Spatial User Interface or SUI adds a set of functions to CAD systems that improve user productivity through more effective user spatial interaction. For example, a 3D control point called the "motion handle" is used to deliver consistent pan, zoom and spin responses independent of model size or view type. This eliminates the need for sensitivity adjustment commonly required in today’s CAD programs. Used in the less dominant hand the Astroid complements the mouse by more effectively taking over pan, zoom and spin control. The use of both hands speeds up modelling time and reduces fatigue on the mouse hand.

Better Quality and more Functional Design: Because the Astroid allows users to more effectively view, navigate and interact with their 3D models, they can interrogate them more thoroughly. Very often first time users of the Astroid will find flaws in their designs they never knew existed simply because they now have the ability to zoom in and orient the model easily, enabling them to find interferences or design inadequacies, which simply was not possible without an Astroid.

The ability to simply "fly thru" and "step inside" the 3D model and view it from the inside-out is often a powerful tool for finding design flaws. A capability that is not available today with any CAD system and alone available with the SUI and Astroid.

High ROI: If we assume the cost of a CAD designer or CAD user is US$60 per hour and we accept the conservative figure of a 20% productivity increase, then an Astroid, at a cost of approximately US$180, pays for itself in less than 15 hours. Another way to look at this is to take the cost invested in a CAD system. If we use the example of a mainstream system like SolidWorks, which retails at about US$4,000, plus the cost of the hardware at say US$2,000, we have an investment of US$6,000. For a small additional investment of US$180 (or less than 3% additional cost), productivity can be increased by at least 20%.

The bottom line is that for just US$180 an investment in an Astroid is probably the highest available ROI for organisations or individuals operating in this industry.