Viewpanes (Datasource Visualization)

Introduction
How it Works
Viewpanes and Windows
CViewpane
Viewpane Updating
The Pointer
Useful Window Types
Win32 Implementation
OS X Implementation
Linux Implementation
Datums, Viewers, and Editors
Rendering Protocols
Printing
Structure
Content/View Relationship
Child Viewpane Layout Algorithm
Simple Portability via the Framebuffer Approach

Introduction

“The eyes are the window to the soul.”

Although computing began in a text-based environment, the need for graphical visualization of data was always real and achieved great satisfaction with the advent of GUI systems. The problem, however, is that programmers either spend much of their time being artists or slap together hastily-done but very plain-looking interfaces. Even when attention to detail is paid, layouts must be modified if the underlying data types for the value entities change. On a large project, this quickly becomes tiresome.

A big cause of the problem is the use of absolute item positioning and sizing. Elements placed this way are prone to manual replacement whenever language, fonts, or font sizes change.



How it Works

Abstract's solution is to automate the visualization process and let the developer focus on the data that the application is managing. As a practical matter, however, there must also be some way to forge relationships between data entities concerning their presentation. This is achieved using the concept of the container viewpane, which is mapped to the intrinsic containment performed by an aggregrate type data structure. By describing a visual containment protocol for its members, the DV can lay out a data structure's member visualizations by itself. HTML frames are a good analogy.

The basic unit of data visualization is the viewpane, which is a Cartesian-addressable two-dimensional surface (it can also be 3D but the projection to the user is ultimately 2D since the computer screen is flat). A viewpane can show all of the data it is representing or only some of it, in which case we say it has a subcontent attribute. A viewpane has a rectangular bounding box but its display shape may not necessarily be rectangular. A good example is an analog clock applet: its appearance is round, but the bounding box is the square that fits the round clock face. A GUI that supports alpha channel window compositing or irregular window shapes can “clip” the viewpane contents to the necessary region.

Like commands with their commandsets, it helps for viewpanes to be logically groupable. An application can provide a default grouping for related viewpanes by providing a parent viewpane, and/or it may allow the user to define his own groupings. An Abstract web browser, for example, could allow users to assemble arbitrary collections of page views into a custom multipane window. An example for a typical e-mail client would be:

CEmailClientViewpane::CEmailClientViewpane()
{
    this->SetOrientation(horizontal);
    this->Add(new CTreelistViewpane);    // folders
    this->Add(new CMessageDataViewpane); // subcontainer
}

CMessageDataViewpane::CMessageDataViewpane()
{
    this->SetOrientation(vertical);
    this->Add(new CMessagesListViewpane); // shows all messages in current folder
    this->Add(new CMessageViewpane);      // shows the current message
}

which at runtime would look like this:

Minimum and maximum bounds limit how small and how large a viewpane can become. If the bounds match, the pane cannot be resized. Top-level viewpanes (such as dialog boxes or document windows) are usually resizable. Containers cannot be sized smaller than their children will allow.

The measurement system is somewhat nontrivial. Normally one uses screen pixels, but this is awkward when using relative measures such as “make this window one-third the width of its parent”. In HTML, both percentages and pixels are allowed. If we encapsulate behavioral rules, we come across situations like “I want this pane to never be less than ten percent the height of its parent” but also “I need this pane to be exactly 640 x 480 pixels” because it is previewing, e.g., a television display. Pixels should only be used when absolutely necessary, because future display devices may easily have significantly greater resolution.

Stacking direction (orientation) controls how panes are ordered within a parent pane. The thin region between panes (whose thickness is user-definable) is used to resize the panes on either side, acting like a splitter bar (assuming the panes are resizable along the axis of pointer movement). In some layouts, such as dialogs, there are few active splitter bars. If neither pane is resizable, the splitter bar goes away or becomes inert.



The alignment of a pane controls how it is justified within its container. If the parent is stacking the children from left to right, then the alignments can be top, centered, or bottom. Otherwise they are left, centered, or right. Alignment has no meaning if a pane can be resized to fit its container.



Placement order controls which pane is placed first in a parent pane, and may match the host system's language reading direction. With Latin languages, the order is top-down, left-to-right. Arabic languages tend to run top-down but right-to-left.



Empty, invisible panes can be inserted to act as “springs”, which force other panes to align with the edges of the parent pane, or simply to keep panes separated some distance.

OS X's approach to subpane arrangement (with the HIToolbox) is to use positioning/scaling/binding attributes. The problem is what to do when a referenced viewpane is destroyed, and it is possible for one to make circular references. The advantage is that fewer container panes need to be used, and the interface tends to be more intuitive to how people arrange panes.

Most scalar datatypes are visualized with rigid or semi-rigid panes. A choice (enum) datatype, for example, is visualized with a combo box or menu selector, and has a fixed height and minimum width corresponding to the longest readable entry in its choicelist.

The ability to break a pane into two siblings with independant subcontent views and to merge such siblings back into a single pane is desirable, but we would need to add a “breakable” pane attribute.

The tab order determines how viewpanes obtain the input focus when the Tab key is pressed. Abstract looks for focusable viewpanes starting with the currently focused pane's immediate sibling in the forward ordered direction. If that sibling is a container, then the search continues with that container's first child, recursively if necessary. If the focused pane has no sibling in the forward direction, it carries on with the parent's sibling. If no such sibling exists, then the parent wraps around and tries its parent or its first child. The reverse logic is used if Shift-Tab is pressed.

An interesting problem concerning the layout of dialog widgets is how to nicely present named items. For example, consider the following window:

We normally want to treat each question item (Name, Age, etc.) as a widget entity. But to make the entry fields line up nicely left-aligned as in the above picture, we would need to place the names in one vertically oriented container and the fields in another. But this would cause a relational break between items and their names. If we wanted, say, to remove an item, we'd have to perform two removal operations. Clearly this is not conducive to being able to think about question items abstractly, which is our entire point.

The solution is to approach the problem as the implementation of a layout rule, which is:

When named items are placed in a container, we want their parts following their name to be aligned.
The alignment point must be at least the width of the longest name plus some suitable padding.

It's tempting at this point to say “Screw it -- let's offer lower-level interfaces so that people can position things however they want. Who's to say we can or even should try to predict all the ways in which viewpanes need to be placed?” But doing so means offering lots of low-level functions, and the burden we wished to alleviate -- letting programmers focus on the essentials -- is instead retained as in a traditional system. We gain portability but nothing else.

Implementing the rule is interesting because for an item to present itself properly, it must obtain information about other items. An item by itself need not bother with field alignment, but adding another item makes both items care.

Another issue is units. This governs the measurement system for user replies. Sometimes units are static, as in the above picture, other times they are variable and usually chosen with a combo box widget. We could treat units as a completely user-definable issue, making developers build higher-level composite widgets out of simpler ones, but that seems awfully traditional for something that is universal enough to be handled by the system. In fact, numbers rarely have meaning without units, and this is exactly the sort of common, low-level detail we would like to automate.

The Windows OS uses a visible main window which contains or owns all of the remaining windows, the Macintosh does not, and Linux could employ either scheme (although most Linux programs mimic Windows, since a global shared menubar is uncommon on that platform). Since an application's windows need an organizing container to track them, what we do is place the organizational features into a CViewpaneBase class, and derive CViewpane from it to add the remaining features we normally associate with windows (sizing, drawing, etc.). On OS X, we can make an invisible root viewpane derived from CViewpaneBase while on other systems we can derive from CViewpane. Since other objects may be drawn upon, we can also declare a third class or interface called IDrawable. A fourth interface also presents itself: IInteractable or CEventHandler, to give a viewpane the ability to respond to user events instead of being merely a presentation device.



Viewpanes and Windows

Viewpanes and windows are abstract two-dimensional Cartesian-addressable surfaces that embody some or all of these features:

How they are actually implemented to the user depends on the platform and the Data Visualizer. An application should not care about how its main window looks or whether a menu bar has been placed within it, where the toolbars are, etc. These are all presentation details. There might not even be a main window at all. In Abstract, a viewpane only has its client area.

An application can associate viewpanes by using the same logical grouping concept as used for dialog items. By indicating, e.g., that two viewpanes are permanent siblings, the Data Visualizer knows to offer facilities for letting the panes be stacked within a parent window, and to not let the user disassociate the two. If, for example, a user wants to group several views together into a single window frame, the application shouldn't have to know about that or implement it. Conversely, the same is true if the user wants to move some views onto a second monitor. Obviously, the Data Visualizer needs to persist view location/size data in a coherent manner and relate them to particular views/data sources.

Freeform mixing and matching of viewpanes into arbitrary groups may violate US Patent No. 5,546,528 (Adobe Systems Inc. “tabbed palettes invention”), although it appears the panes in question would have to have clickable tabs and overlap each other in a modeless property sheet manner within helper panes. The patent was unfortunately upheld in US court against Macromedia and was filed in 1994, leaving until 2014 to expire. As a US patent, however, it may apply only to operation of Abstract within the US or by American parties.

Then again, since Abstract is only a framework, legal obstacles may only apply to those writing applications. As long as a developer gained permission to use the patent, he could use Abstract's “flexible/collapsible tabbed windows” facility. To do this would require adding a flag to the framework letting it allow/deny the facility.

Viewpane events such as mouseclicks/drags are normally auto-translated to document space by a prespecified coordinate transformation object. This way, the application can think of user events as occurring in document space instead of worrying about window space.

Done properly, things like splittable/stackable views don't concern the application either. The View Manager should be able to offer these features independantly and have them work seamlessly as long as the application properly supports multiple views per data source.

Viewpanes are also used by GUI dialogs to handle data types known only to the application (see below).

A viewpane that can only show a subset of its datasource content automatically becomes decorated with navigation facilities such as scrollbars. Such viewpanes must respond to document navigation messages issued by the Data Visualizer.

A viewpane can have the attribute of presentation dimensionality (2D or 3D). In the 3D case, rendering calls follow the OpenGL API. In 2D, rendering follows a protocol similar to PostScript/GDI.

For complex presentations, it is better to treat drawing as the behavior of renderer classes which are tightly coupled to datasources, rather than thinking that the viewpane is drawing its content. One can also consider adding Draw or Render methods to datasources, but since there are many ways to draw the same data, the renderer paradigm is more favorable. Appearance classes are also handy metaphors for separating the appearance of data apart from the data itself, but still providing for appearances to be editable and persistable. Render classes and objects make report generation (printing) much easier as well.

Font and font glyph resources can differ across platforms. The well-designed viewpane adapts to variable font availability, but often it is convenient or artistic to fix a specific appearance. Some applications include their own font resources as one solution. The basic serif, sans serif, and monospace fonts and ISO 8859-1 charset represent the current common denominator (the fonts are usually mapped to Times New Roman, Arial/Helvetica, and Courier). The Dialog Visualizer automatically uses the preferred (or perhaps user-specified) system fonts for any dialog box visuals. On Windows and Mac this maps to, respectively, Tahoma 8pt and Lucida 12pt.



CViewpane

A viewpane is a visual representation of a datasource. It is instantianted when the datasource is asked to visualize itself. Since there can be different views of the same data, a datasource offers a list of view types and the ability to instantiate a view of one of those types. It also defines one of the view types as the default view type.

Once created, a view is stored in an internal viewlist structure maintained by Abstract. The view is destroyed when the user closes it or if the application decides to do so. Since Abstract subscribes to the datasource, the datasource's destruction automatically destroys all of its viewpanes.

The base viewpane class offers core drawing methods. Usually, a datasource will subclass to a specific viewpane type that is a friend of the datasource.

The data members inside a view govern its size, relative position within its container, etc. One could make a datasource to visualize this data, although of course this offers an infinite regress (views about views about...).

Viewpane creation normally occurs at the behest of the user, e.g., when opening a document or choosing a command that cannot execute without a dialog. In such cases, Abstract is directed to visualize the related datasource. In these cases, there is no logical containing view, so the created viewpane is automatically a top-level one (in Windows, there is of course the main window frame, but from the application's perspective, this window does not exist). Such viewpanes tend to be aggregates of subpanes. Abstract constructs the necessary platform-specific window and then relates the created viewpane with it.

In Windows, the viewpane hieararchy tends to naturally mirror the host's containment window list. In OS X, controls are their own entity class, so some extra work is needed to treat them as subwindows. In the pane class, a reference to the control would be stored instead of an HWND as it would be on Windows, and subpanes would explicitly store size/position data. To Carbon, its windows have a flat list of controls placed within them, but behind the scenes Abstract maintains the illusion of a hierarchy and of course, to the application, this is all it can know about.

Pointer events occurring to a viewpane are filtered through a spatial mapper to convert the pointer (mouse) coordinates to some meaningful position in datasource terms. A text editing widget, for example, wants to know what character position in the text stream was indicated. A bitmap editor has a more intuitive mapping, usually involving scaling and translation. The inverse transformation is also desirable when rendering the datasource.



Viewpane Updating

An instance of CViewpaneUpdateInfo is used when a viewpane needs to draw its contents. The most valuable member is the update rectangle, since this can save time by culling drawing operations that will not be forwarded to the display. All drawing performed by CViewpane methods intrinsically takes place in a pixel buffer which is then blitted to the display. On Mac OS X, buffering is intrinsic so the framework omits its own buffering. Another handy member is the “is resizing” indicator, which lets the viewpane draw a high-speed (but simpler) representation of the datasource to keep dynamic window resizing operations from stuttering.



The Pointer

In Abstract, the mouse is considered to be a driver for a higher-level object called the pointer, which is a Cartesian spatial entity manipulated by the user and which has certain other attributes (such as button states). It is possible that in the future, large desktop systems will allow multiple users to simultaneously work within a system, and if so, CPointerInfo will include information identifying which pointer is involved. It is also easier to support different devices driving the pointer, such as tablets, pens, mice, trackballs, etc.

In GUI systems, the mouse pointer has a visual representation (called the cursor) which is usually an arrow but which changes to other shapes to indicate other states. For example, when a program is busy the cursor becomes a wristwatch. In Abstract, it is better for the application or a viewpane to identify itself as busy and the system then chooses the optimal way to communicate that. In MFC, for example, one instances a CWaitCursor object whose lifetime makes the cursor appear as an hourglass, but it would be better to instance a CBusyState object. The underlying system could just as easily display a “busy” message in a status bar until the object destructed.



Useful Window Types

There are basically a few broad categories of viewpane:

The application's main window, dialog boxes, splitter bars, tooltips, embedded scrollbars, popup menus, etc. are examples of automatic viewpanes since they are automatically created and managed by Abstract itself, and accessibility to them may be limited or nonexistant. Items placed on dialog boxes and helper windows are widgets (predefined or custom).

Modeless dialog boxes do not exist (use helper windows). To show a progress bar and some status text for a lengthy operation, use a statusbar. A global statusbar can be placed in a helper window or local ones can be placed alongside related content viewpanes.

Helper windows are siblings of top-level datum viewpanes, because they often allow mutation of the associated datums. In Adobe Illustrator, for example, the floating Type window lets one alter the attributes of the current document's selected text objects. The reason helper windows are desirable is that having them as embedded children within a normal document view uses up more screen real estate than necessary. The only real drawback is that one cannot compare some datum attributes of multiple documents simultaneously -- switching to another document automatically clears out the old values from the helper window and replaces them with ones reflecting the selected document. Old versions of Quark Xpress actually had toolbars embedded into document windows, which felt awkward.

Helper windows don't always refer to document datums, of course -- they can just as easily manage application or view-specific entities. The main thing is, the helper windows are singletons regardless of how many datums exist. What would be nice is a way to associate a helper window to respond to changes in the active document or view. Basically, the concept of current view and current document should be formalized in an object of its own that such windows (or anything else) can listen to. Either that, or the windows need to listen to the app and act upon messages such as msg_viewactivate and msg_docactivate.

Helper windows should have their own specific CViewpane subclass, since we don't want to interpret their class any other way (e.g., like being direct children of the root window). A special and likely popular type of helper window is the document treelist, which allows document element access in textual drilldown fashion. The concept is well exemplified in WCS and most 3D modelers.

Top-level viewpanes automatically gain a title bar and can be moved about; these styles are inherent in the pane's “topness”. If one wants an SDI model, all it takes is to make only one such viewpane, and tell Abstract that the application only allows one top-level normal viewpane. This could also be a user preference, letting users decide when an app should deploy SDI or MDI behavior. The user normally can also explicitly create and close top-level viewpanes. What's implied here is that an application should always support multiple top-level viewpanes, since “SDI-ness” may be a user choice.

Modal dialogs are basically top-level container viewpanes that temporarily exist to answer some question the application has of the user. The event loop is trapped to funnel events mainly to the dialog until it is closed. The datasource is a structured question (basically a hierarchical collection of simpler datums) that the user can modify or view but not modify. The traditional OK and Cancel buttons are auto-generated and invisible to the program [ todo: and Apply also, but should that be optional? ]. Question datums whose type are unknown to Abstract (basically anything more complex than a bool, number, char, time, date, color, choice, string, etc.) require custom editors to be preregistered for those types.

Why have modal dialogs at all, when it is technically possible to have modeless property panels? The main advantages are that modal sessions a) focus attention onto the issue at hand and b) the visual representation of the issue disappears completely once it has been dealt with. It doesn't hang around afterwards taking up screen real estate and calling attention to itself in your peripheral vision. Modeless panels also make an application feel more complicated than it might actually be, simply because of the “bells and whistles” effect. If you've ever opened a typical 3D modeler and seen dozens of buttons and sliders and other widgets appear from the get-go, and felt intimidated, you know what we mean.

This is not to say that the strategy devolves to modal vs. modeless. It is more a matter of knowing when each tactic makes the most sense. Modal dialogs are great, for example, at handling once-in-a-blue-moon issues, critical questions, and things that occur exceptionally like whether to proceed with a slower operation because memory is low. Things that are not off the beaten path are better suited to modeless panels.

Some questions are simple enough (e.g., a single bool) that a Yes/No message box is more appropriate. If there is no question at all, the dialog is simply an alert or informational statement.

To create and open a dialog, one would do something like this:

    // class CQuestion : public CDatasource

    class MyQuestion : public CQuestion
    {
         MyQuestion() { this->Add(new NumberQuestion(String(“No. of apples”)); }
    };

    MyQuestion.Ask();
    if(MyQuestion.WasAnswered()) { // Respond to answer  }



Since there could be little underlying difference between dialogs and normal viewpanes, the only difference between a CQuestion and a CViewpane would be the Ask and WasAnswered methods. If we want to support Apply buttons, however, we need the CQuestion class to have a virtual Apply method that MyQuestion can optionally override (and Unapply, if Apply is used but the dialog is cancelled).

If the dialog is referring to document members, however, then ideally we should be able to pass a reference to those members as the datasource. We also want to package the answer as an edit, because it can be undone/redone.



More Dialog Stuff (moved from the main page)

Dialogs are traditionally implemented using dialog boxes, which are window subclasses. This works fine in a GUI environment, but it actually constricts one's options. What the application really needs is a way to (modally) ask the user for some information. How that question gets asked and answered should not really concern the program. In most systems, however, developers often spend much of their development time designing the appearance and behavior of dialog boxes.

An Abstract dialog basically requires a developer to structure the question he wants to ask the user, and to format the answer in usable form. Matters of appearance depend entirely on the runtime context of the computer, such as whether the interface is graphical or console-based, what the native language is, and so on. The particular widgets used to perform responses are decided based on the data types of the question parameters. If a question requires a yes/no (boolean) answer, the widget is a checkbox. Sets of questions can be logically grouped, and if so a GUI system will try to display those questions so that they appear together and in a preferred reading order. If the dialog contains an abundance of questions, the Dialog Visualizer can opt to scroll the dialog, generate a tabbed dialog, or use some other strategy that solves the problem.

There are more dialog types than simple question-and-answer sessions. Some dialogs are used to command the program. Others perform interactive viewing of the document. Some dialogs require special visual aids (e.g., Leveller's “Enlarge Canvas” dialog displays a 3 x 3 tile widget to let the user visually choose a bitmap enlargement method). This exception can be handled by treating such dialog elements as viewpanes whose data must be presented by the application. Basically, the system defers presentation/interaction of unknown data types to the application.

When datasources are flagged as static, the Dialog Visualizer can provide some powerful features, such as the ability to let users choose competing presentations for some data types. In a traditional dialog, one uses a text field to edit a number. Want spin buttons? They have to be added manually. Want to use a slider? Now the text field has to be ripped out and replaced, and the dialog box probably resized. In Abstract, these widget choices could be left entirely to the user and recorded for automatic reuse when the dialog is opened again. All the developer needs to do is describe the datasource as a number with exposed validation rules, such as range and precision.

The visual layout of GUI dialogs often requires the human touch. Automated methods can produce serviceable simple layouts but fail to achieve the elegance of design of which humans are capable. A solution is to have the Visualizer let users alter dialog layouts at will, at runtime, and record their designs.

String translation for multilingual systems is also problematic. There currently does not exist a comprehensive universal translation engine to convert developer language text into the host language. Instead, one needs to manually provide desired translations ahead of time in string resources. Abstract could supply premade translation resources for the most common English strings.

The choice of widget can sometimes be ambigious. For example, a number can be edited using a text field, a slider, or both. We could say that a text field is the default editing visualization of a number but a slider should be used for constrained numbers, i.e., those that have some bounding limit. But if a constrained number is a subtype, then we eventually need to define such subtypes for all number types. Worse, the constraint may not be generalizable. A string, for example, might need to disallow certain character codepoints or leading/trailing whitespace. A date or time might not allow certain temporal references, and so forth.

A better way to think of constraints is as validation processes. Any datasource, in fact, should have a function member derived from CDataValidator that references a validation function. For constrained numbers, this function simply checks the value against the desired bounds. For editing visualization, we need to have Abstract instance the correct widget based on the validator. It follows that there must be some public way to identify validators, if not by RTTI than by an explicit ID member. If one uses a generic number type, it is straightforward to visualize the correct widget.

Some types have intrinsic validation or constraints. Unsigned integers, counters and quantities, for example, cannot be negative. Absolute angles need only range from zero to 360. Surface normals cannot have negative Y components, etc.

Another shortcoming of traditional dialogs is that there is no explicit support for displaying strings containing presentation meanings such as bold, italic, etc. This is particularly acute in message boxes where sometimes it would be handy to emphasize particular words. My recommendation is to support HTML encoding in the string type and to have visualizers render such encoded text accordingly.

Radio buttons are basically an optional representation of the choice type, which normally uses a combo box. One interprets the matter as choosing whether all the choices should be visible or if only the current choice needs to be visible. A fully displayed combo box list, however, achieves the same result as a radio button group.



Win32 Implementation

I've chosen Windows MDI (Multiple Document Interface) to handle the way Win32 implements Abstract's viewpane management. The application itself is not aware of MDI (nor does it need to be). MDI is necessary because on Windows, it is the de facto standard way for multidocument applications to behave. There are other models (such as Adobe's) where the main difference lies in helper window docking behavior. There's no reason why Abstract couldn't offer multiple viewpane management models on Win32, except it's obviously more work to do so.

The root window is visible and corresponds to the application's main (or frame) window. The menu bar resides in this window.

The application's main window has a client area (actually implemented with a child window called the MDIClient) which holds any number of titled, moveable and overlappable document view windows, as if the app window was a miniature desktop. When a view window is maximized, it occupies the entire client area of the app window and its title bar is subsumed, making the app window look like a document window. The window sizing controls are also moved to the frame's menubar. Windows can also provide easy access to top-level views by adding commands to a specified menu. Most of this behavior comes for free with the Win32 API.

Application's are unaware that top-level viewpanes are MDI children. A private CViewpane::m_bIsDocWindow member flags whether this is the case or not, and the default WndProc for the viewpane processes Win32 messages accordingly. [ todo: probably better to make m_bIsDocWindow a IsTopLevel method and under Win32, just check to see if the parent HWND is the MDIClient ]

The CAppWindow class contains references to both the frame (m_hWnd) and the MDI client (m_hWndMDIclient).

Because the root window is visible, helper windows can be docked inside it (or perhaps alongside it too). Although Win32 does not readily provide this behavior, MFC shows a good implementation example and its behavior is fairly standard. In docked mode, a controlbar window is used to take up an area of the frame's client (and reduces the MDIClient accordingly) that spans either the width or the height of it depending on orientation; this keeps the MDIClient from having an irregular shape. The controlbar can hold multiple helper viewpanes. In an undocked state, a floating titled window is used to contain the controlbar inside its client area which in turn holds only one helper viewpane (which is what defines the size of the window in the first place). Or that's at least the way MFC does it. The docking behavior is fairly sophisticated in that controlbars might be attachable to any edge of the frame window and in either orientation, and can be shuffled amongst other docked controlbars. Docked helper viewpanes can also be freely moved from one docked controlbar to another, or create entirely new controlbars to contain them if docking by themselves. Docked controlbars likewise vanish if their last helper viewpane was moved out. The overall effect gives the illusion that only the helper viewpanes exist and that the frame window magically expands and shrinks its nonclient area.

Some Windows apps take the docking one step further and make the menubar itself a controlbar which by default is docked but can be undocked. We have that as an option, but some users find it questionable in practice, especially when a menubar becomes undocked by mistake. It is difficult to see the usefulness of a floating menu bar.

A pointer to the CViewpane object is stored as a userdata item in the HWND object. [ For this reason, we could omit the windowlist members from the base viewpane class and simply use Win32 windowlist calls to access parents/sliblings/children ]

The details of the MDI handling are in the viewpane and app modules.



OS X Implementation

Multiple top-level viewpanes on OS X reside in an invisible root viewpane, so the issue of docking helper viewpanes suggests the ability to dock them relative to each other and to the top-level viewpanes. Adobe handles the situation by letting palette windows (toolbars and other small floating windows) simply “snap” when near each other or near certain features of the parent window and avoid overlap. On Linux, some window managers have universal snapping: any window can snap to align nicely with any other, including the desktop.

OS X offers nestable viewpane support in 10.2 using HIView subclasses. Top-level windows, however, need to be created as traditional window objects (accessed via WindowRef handles).



Linux Implementation

Both visible and invisible root viewpanes have been implemented on Linux by other applications, although the standard would appear to lean towards the Macintosh way of doing things (invisible root viewpane). It would not surprise me at all if the Linux version of Abstract offered both styles.



Datums, Viewers, and Editors

Views present a classic problem that negates the benefits objection orientation tries to bring. The problem is this: for any given type of data, there can easily be many different ways to view and/or edit that data. If we accept that premise, that means we end up defining multiple viewers and editors, all of which need to be on “friendly” terms with the data type for performance reasons. This situation, in fact, is entirely similar to the problem applications have with documents: for any given document type, there are many programs that can display or edit it.

It would be nice to associate viewing and editing directly with a data type. The problem is that the data type must anticipate what the possible viewing and editing styles will be. At the very least, anyone implementing a new viewer or editor must access the definition for the data type. Purists object to this approach because a data type should be concerned solely with the value of its content and not on externals such as rendering and, by extension, rendered mutating.

A simple example is provided by an angular value type: one normally edits it with a dial control. But if you look at the parent class, which is a number, we could also edit angles using sliders or number box fields.

Those who like the data-centric model worry that, as the number of viewers/editors increases, changes to the data type cause a corresponding breakage in the viewers/editors. This is evident whenever a document gains new fields or data members: old programs must be upgraded to handle them. The relationship is classically “strongly coupled” and is a chief bane of software design, particularly because documents are also persistent.

If we borrow the filesystem metaphor where users can associate editors with filetypes, we could let Abstract application developers or users do the same with the editable types inside the application. This requires some additional runtime structures but is not conceptually difficult. The main thing is that we would need to standardize the definition of type editors, and not hardwire which ones are available to work on a type or datum. If we include the concept of a read-only editor, we can fold viewers and editors into the same concept. In this vein, there is no datum difference between a static text widget and an editable text widget; the underlying text is either read-only or read/write.



Rendering Protocols

Most of the time, drawing 2D graphics means blitting bitmaps. Vector graphics are required by programs that need to display text or vector shapes. The simplest rendering protocol would be to offer a pixel framebuffer and let application code fill it in however it wants. This of course would lead to the development of automation libraries (since no one wants to draw lines by coloring pixels each time), hence one or more standard vector protocols should be provided also. An application using its own vector library would of course have full portability, and computers are now fast enough to provide good software-only vector performance for a wide range of presentation needs.

Operating systems have richer 2D and 3D drawing support which makes portable protocols easier. For example, GDI supports Bezier splines, and Quartz supports PostScript-style spatial transforms.

Bitmap pixel manipulation presents a weak link in portability because different platforms use different bitmap pixel and scanline arrangements. Using a standard pixel format impacts performance because it must potentially be converted when blitted to the device framebuffer. We can use platform-specific pixel formats, and include descriptive metadata to applications (which is what OpenGL does). There is some performance loss as the application must conditionally switch to the necessary pixel processing code based on the metadata. One way around this is to encode pixel reads/writes as macros, with a different one for each platform. If we use 24-bit DIBSECTIONs on Win32, for example, the pixel write macro is:

WRITE_PIXEL(p, c) *p++ = c.b; *p++ = c.g; *p++ = c.r; // BGR order

while on OS X if we use 32-bit CGWorld pixmaps, the macro is:

WRITE_PIXEL(p, c) *p++ = c.r; *p++ = c.g; *p++ = c.b; p++; // RGB_ order

Which essentially means that we make an inline pixel access/mutation function.

For bitmap drawing, it's nice to have a choice of bit depths, at least 1/8/16/24/32. Richer bit depths or floating-point depths (per pixel or per-channel, or even ILM's 16-bpp “Half” floating point format) are useful for those performing composited processing or other intermediate calculations.

For vector drawing, there are several features that are desirable:



Some existing protocols are worth examining:



It's tempting to use a 3D protocol since 3D is a logical superset of 2D, and one also gains 3D lighting effects for free. It also simplifies the API for applications needing both 2D and 3D -- it's the same API for both.

Currently, the protocol is undecided. Abstract uses an IDrawable interface class to make the choice of protocol easily switchable in the viewpane class using conditional compilation. Expanding on that idea, the most flexibility could be gained by using “pluggable” protocols that are connected at runtime, although this understandbly goes against the idea of abstraction. My personal favorite is PostScript, as it's original form is the “RISC chip” of 2D APIs: it accomplishes a lot with as few functions as possible (e.g., all shapes can be constructed from paths).



Printing

It's a common tactic to make the rendering protocol identical for viewing and printing. Which seems very reasonable, if the drawing code is fairly involved and/or the printed version looks similar to what is on the screen. Desktop publishing and GUI word processing programs exemplify the need considerably, since people expect perfect WYSIWYG correspondance (in essence, their documents appear as virtual printouts -- print previews -- on screen). But there is no universal rule that says printed output must match or be produced in the same manner as visible output. I will admit, however, that since consistency is desirable, one should aim for as much similarity as possible. As monitor resolution is expected to increase and therefore progress closer to typeset quality (or at least the quality of the first laser printers), the correspondance will make even more sense.

Printing adds an additional wrinkle in that the total surface space is conceptually much longer, but usually along only one axis. While on a screen one pans content that is not fully visible, with a printer one prints as many pages are required. The physical break between pages also implies that a program takes care not to render items across page seams. Users also need to be able to choose which pages are to be printed, which means that the program must determine what content appears on the starting page. Due to margins and break avoidance, converting between document space and printer space is not a straightforward matter of just multiplying the page number by the page height.

In a perfect world, the system would fully automate printing by asking for viewpanes to be rendered and then converting their pixel content to the necessary pages. Since printers typically have much higher resolution, this would only work if the viewpanes were larger and appropriate scaling was factored in. Another approach would be to define data visualization as a generic task independant of final realworld representation, and then map the result of that task to the desired device.



Structure

In simple terms, one would just like to visualize data. But what does that really mean? To make data visible? Or to capture the visualization somewhere (which may be offscreen) and then displayed? Is there any real difference between an offscreen device context or bitmap and a viewpane? If a window is hidden, does that mean we can still draw on it and the drawing will be there for us to display later?

From a totally abstract perspective, rendering would be more OpenGL-like in terms of visual persistence (assuming you had double buffering enabled). It doesn't matter if the viewpane is actually visible, what you draw is stored somewhere. It might help to look back at the history of GUI development.

Older computers had neither the memory nor speed to buffer drawing (or at least, not universally). The original Macintosh would move the mouse cursor during vertical retrace intervals to avoid tearing. Unbuffered drawing was made workable by employing window invalidation and update concepts: if the data changed, or if an overlapping window moved and caused new regions of another window to become visible, the necessary regions would be added to an invalid region and an update event sent to the window, which would then redraw the data. If the program was slow at drawing, these updates would stall GUI responsiveness. A well-designed program would draw only what intersected the update region, or if not that then the region's bounding rectangle, also called the update rectangle. If the underlying datum changed, however, usually the whole window had to be redrawn, and this is true of course even with buffered approaches.

Apple has evolved OS X to try a universal buffering approach, which makes even more sense since they are using OpenGL. If I'm not mistaken, I believe QuickDraw and Quartz render calls simply modify a texture bitmap, which OpenGL then blits. If this is the case, buffering the GL context would be redundant. In any event, to answer our earlier question, offscreen bitmaps and viewpanes are equal.

In this case, updates not involving datum changes can be handled by the system (which is precisely what OS X does). Elsewhere, the application must at least blit from a buffer during processing of an update event. Even programs which can draw quickly often use buffering because it also looks much nicer -- one doesn't see any drawing occurring. Automatic buffering thus delivers a twofold win: it simplifies development and improves the visual experience.

Achieving this on Win32 requires letting base viewpane classes handle update events not related to datum change, which means we need to identify datum changes. The simplest way to do this is to use a broadcast/listener pattern to handle datum changes, and when they occur, invalidate not only the viewpane but also the buffer holding the datum's rendition. This way, whenever an interface invalidation occurs, the update logic can always default to just blitting from the buffer. As far as the viewpane subclass is concerned, update events never occur -- it becomes all about maintaining the buffer whenever the datum changes.



Content/View Relationship

A viewpane normally has some principal datum that holds the information that the viewpane is showing to the user. A clock applet, for example, could have a round viewpane showing the current time.

Interaction with the viewpane at the operating system level is concerned with coordinates in window space, while accesses and mutations to the content occur in datum space. While the former is in 2D or 3D, the latter may not necessarily be. A text document, for example, is a 1D array of characters (and higher-level items such as paragraphs) that wraps or breaks lines to make its presentation 2D. Obviously, some kind of coordinate mapping between these two spaces must be provided at the application level.

The mapping can also answer the question of whether the viewpane is currently showing all or part of the datum. If it can only show a part, then the system can not only provide scrolling facilities but also know the range and position of the scroll interface's elements.

The coordinate system of datum space is logically tightly defined by the datum's nature. A word processor, e.g., can have a complex structure describing the cursor position in a text stream. While character position is obvious, other auxiliary data might be preferable to store. A tree widget would map viewpane coordinates to a reference to a displayed object item.

Zooming is inherent in many visible datums. Bitmaps can enlarge pixels, word processors can increase and decrease their viewing level, etc. The spatial mapping system must take any such viewing scale into account. Rotation is rare but can also occur.



Child Viewpane Layout Algorithm

Getting a container viewpane to lay out its children is basically a space-filling problem that, due to the hierarchical pane structure, is recursively divided and conquered. The pseudocode is basically:

viewpane::layout()

    pos = 0

    if container layout is horizontal then
        slack = container width - sum(fixed child widths)
    else
        slack = container height - sum(fixed child heights)

    for all child panes
        if container layout is horizontal then
            set child height to min(container height, max child height)
            if child width is fixed then
                set child width to its fixed width
            else
                set child width to max(min child width, slack / count(panes remaining))
                limit child width to max child width
                slack -= child width
            endif
            move child to (pos, 0)
            add child width to pos
        else
            // vertical orientation, same logic as above but change axis
        end if
    next child pane

Note that we do not need to have layout() recursively call itself upon any child panes, since resizing a child pane inherently causes it to lay itself out. This is efficient, because laying out only needs to be performed when a pane is resized.



Simple Portability via the Framebuffer Approach

August 21/2004: Desktop computers are now so powerful that a great deal of portability can be achieved by simply implementing the GUI (the window manager) at the application level.

In this scheme, only top-level windows are created by the OS. Child windows are simply offscreen framebuffer regions managed by the library. They are composited together whenever the top-level window is invalidated. The framebuffers are stored in a traditional tree data structure.

Ideally, widget appearance and window decoration is customized for each platform, along with any platform-specific interface behavior. A prototype that implements a universal theme can be used first for testing.

Supporting both OpenGL and regular 2D content can be made easier by instancing framebuffers as OpenGL textures and drawing them using camera-facing orthographic projection (in fact, this is already what OS X does). The top-level window hosts an OpenGL context covering its entire area, and the framebuffer texture for that context is built by compositing the child framebuffers (which may or may not be textures, whichever is more efficient).



Copyright 2003-2004 Daylon Graphics Ltd.
Abstract is a trademark of Daylon Graphics Ltd.
Author(s): Ray Gardener (rayg@daylongraphics.com)
Location: http://www.daylongraphics.com/other/abstract/visuals.htm
Last updated: August 21, 2004