| Home -> JCNIX -> Java2D Rendering pipeline |
The following is some notes on the design of a rendering pipeline for the Java2D implementation I've been working on.
Graphics objects has properties that determine the effect of
drawing operations. The properties include color, paint, line
stroke, coordinate transforms, etc. The most suitable rendering
pipeline at any given time is based on the properties of the
graphics object. A graphics object with simple properties, such as
a line width of 1 pixel, no dashing, simple coordinate
translation, etc., suggests a simple rendering strategy that takes
advantage of the drawing primitives of the underlying native
graphics libraries. E.g. an X11 toolkit would use
XDrawLine, XDrawRectangle, etc., when
possible, rather than draw every pixel manually and then ship the
whole thing as an image to the X server. When the graphics object
has complex properties such as arbitrary affine transforms,
transparency, rendering hints, etc., it is often not possible to
support these properties directly in the native graphics
library. Not all native graphics libraries share the same
capabilities. E.g., an X11 supports multiple line widths and
certain kinds of line styles, but does not support affine
transforms and transparency. Some graphics backends have vastly
different characteristics than typical raster based screen output
backends such as X11. Consider for instance a Postscript backend
compared to an X11 backend.
All these differences must be consolidated so that all backends
can be operated in a uniform manner, through the
java.awt.Graphics2D API. The properties of a graphics object
determines the best rendering strategy, and that graphics objects
are not immutable, (meaning, the properties may change during the
lifetime of the graphics object). This means that the graphics
object must be prepared to switch rendering strategies. The
transition from one strategy to another must be seamless and
happen on the fly. There in no way of determining in advance
whether advanced Java2D rendering features will be called
upon. This means that it is not possible to designate advanced
rendering capabilities only for specific components. In terms of
X11, a specific paint method may require the rendering pipeline to
switch from server-side rasterization to client-side rasterization
and back again seamlessly as a response to requests such as
turning on and off anti-aliasing hints. All drawing that was
earlier rasterized on the server-side should be considered when
performing compositing on the client side.
The far front-end of the rendering pipeline consists of the
Graphics2D API. In the far back-end, lies the native
graphics libraries. In most cases the native graphics libraries
only have direct support for a subset of the properties of
Graphics2D. To make up missing features in the native
graphics libraries, the pipeline between the front-end and the
back-end need to translate drawing request to primitive operations
that are supported by the back-end. E.g. for X11, drawing a
straight line will translate to an XDrawLine, drawing
a bezier curve will trigger flattening of the curve and will
result in a call to XDrawLines.
This is the basic strategy for the rendering pipeline: Whenever a graphics property change occurs, that causes the current pipeline to be insufficient, amend or replace parts of the pipeline so that the pipeline will once again be able to translate requests to the set of primitives supported by the native graphics library.
Most graphics libraries share common subsets of functionality. To be able to reuse pieces of the rendering pipeline for several backends, we define interfaces that describe subsets of characteristics supported by the backends. A wrapper for the native library can implement several interfaces to describe its range of functionality.
Typically, most painting is done with a graphics object with simple properties. Unless one is using a complex Look & Feel, the painting of Swing components will never require affine transforms, alpha blending, non-rectangular clipping, etc. When graphics objects are created, they start off in a state where all the properties are simple. Most graphics objects experience only trivial property changes, and never leave this simple state. It is therefore wise to ensure that the rendering pipeline for this initial state is lean and as much as possible plugs directly into the backend.
The initial state for graphics object of most raster displays would call for two levels of indirection:
Graphics2D object --->
IntegerGraphicsState--->DirectRasterGraphics
The Graphics2D object is an instance of a concrete
implementation subclass of java.awt.Graphics2D
(Graphics2DImpl). This objects manages a state object
(State pattern, GOF book) that represents the current pipeline
configuration. The Graphics2D object forwards most of the
requests to the state object. The Graphics2D object
itself only administers properties that are not specific for a
certain state. It stores properties such as foreground and
background color so that the state object is not bothered with
get-requests for these properties.
IntegerGraphicsState is one of several graphics
state implementations. This graphics state is used when the
graphics object has simple properties, (coordinate translation
only, no transform) and the backend supports integer coordinates
(pixel based). For primitive paint operations, this object
translates the coordinates and forwards the request to the
backend. For requests to draw arbitrary shapes and paths, this
object translates the requests to primitive drawing operations
supported by the backend. IntegerGraphicsState is meant to support
the most common state of an graphics object. The degree of
functionality is roughly equivalent with the old
java.awt.Graphics API.
All graphics states is derived from a class
AbstractGraphicsState:
public abstract class AbstractGraphicsState implements Cloneable {
Graphics2DImpl frontend;
public void setFrontend(Graphics2DImpl frontend);
public void dispose();
// Graphics methods:
public abstract void setColor(Color color);
public abstract void setPaintMode();
public abstract void setXORMode(Color altColor);
public abstract void setFont(Font font);
public abstract FontMetrics getFontMetrics(Font font);
public abstract void setClip(Shape clip);
public abstract Shape getClip();
public abstract Rectangle getClipBounds();
public abstract void copyArea(int x, int y,
int width, int height,
int dx, int dy);
public abstract void drawLine(int x1, int y1,
int x2, int y2);
public abstract void fillRect(int x, int y,
int width, int height);
public abstract void clearRect(int x, int y,
int width, int height);
public abstract void drawRoundRect(int x, int y,
int width, int height,
int arcWidth, int arcHeight);
public abstract void fillRoundRect(int x, int y,
int width, int height,
int arcWidth, int arcHeight);
public abstract void drawOval(int x, int y,
int width, int height);
public abstract void fillOval(int x, int y,
int width, int height);
public abstract void drawArc(int x, int y,
int width, int height,
int startAngle, int arcAngle);
public abstract void fillArc(int x, int y,
int width, int height,
int startAngle, int arcAngle);
public abstract void drawPolyline(int[] xPoints, int[] yPoints, int nPoints);
public abstract void drawPolygon(int[] xPoints, int[] yPoints, int nPoints);
public abstract void fillPolygon(int[] xPoints, int[] yPoints, int nPoints);
public abstract boolean drawImage(Image image, int x, int y,
ImageObserver observer);
// Graphics2D methods:
public abstract void draw(Shape shape);
public abstract void fill(Shape shape);
public abstract boolean hit(Rectangle rect, Shape text,
boolean onStroke);
public abstract void drawString(String text, int x, int y);
public abstract void drawString(String text,
float x, float y);
public abstract void translate(int x, int y);
public abstract void translate(double tx, double ty);
public abstract void rotate(double theta);
public abstract void rotate(double theta,
double x, double y);
public abstract void scale(double scaleX, double scaleY);
public abstract void shear(double shearX, double shearY);
}
DirectRasterGraphics is an interface that
describes a common denominator between most raster based graphics
devices. A backend object (such as XGraphics, or
GDKGraphics) implements this interface. The backend
object may also implement additional interfaces to indicate that
it supports more advanced drawing operations.
public interface DirectRasterGraphics extends Cloneable {
public void dispose();
public void setColor(Color color);
public void setPaintMode();
public void setXORMode(Color altColor);
public void setFont(Font font);
public FontMetrics getFontMetrics(Font font);
public void setClip(Shape clip);
public void copyArea(int x, int y, int width, int height,
int dx, int dy);
public void drawLine(int x1, int y1, int x2, int y2);
public void drawRect(int x, int y, int width, int height);
public void fillRect(int x, int y, int width, int height);
public void drawArc(int x, int y, int width, int height,
int startAngle, int arcAngle);
public void fillArc(int x, int y, int width, int height,
int startAngle, int arcAngle);
public void drawPolyline(int[] xPoints, int[] yPoints, int nPoints);
public void drawPolygon(int[] xPoints, int[] yPoints, int nPoints);
public void fillPolygon(int[] xPoints, int[] yPoints, int nPoints);
public void drawString(String str, int x, int y);
public boolean drawImage(Image image, int x, int y,
ImageObserver observer);
public MappedRaster mapRaster(Rectangle bounds);
public void unmapRaster(MappedRaster mappedRaster);
}
A change in the graphics properties occurs when one of the
set-methods of the graphics object is called. This change may or
may not trigger a replumbing of the graphics pipeline. The
set-request is forwarded from the Graphics2D object to
the graphics state object. Depending on the current pipeline
state, it may be forwarded further, or handled within the state
object. If the graphics state determines that it currently cannot
support the new graphics property, it throws an
PipelineChangeException.
The exception is caught by the graphics object implementation and delivered to a Plumber implementation. A plumber is a device specific object that is responsible for reconnecting pipe segments (i.e. rewiring objects) so that the current set of properties are supported. The exception object that is thrown may hint to which changes that must be done on the pipeline. The plumber object may choose to take these hints into consideration. The exact nature of the information exchange between the code that throws the exception and the plumber object, is left open. The plumber object is custom tailored for each backend.
The compositing capabilities of the backend are often insufficient. The backend may not support alpha blending, or may not support some other special compositing rule. This means that compositing must sometimes be done within the rendering pipeline. The general compositing operation consists of combining new color and alpha values with existing color values on the drawing surface, to find the new color values for the drawing surface. The way the values are combined, determines what kind of compositing operation that is performed. The default compositing operation is alpha compositing.
In order to perform alpha compositing and other compositing
operations, we need access to the color values of the imagery that
has already been drawn on the drawing surface. The
DirectRasterGraphics interface must therefore contain
methods that makes it possible to gain access to the pixel values
of the drawing surface. The methods are modeled after the POSIX
mmap() and munmap() functions. But,
instead of mapping and unmapping portions of data from a file
descriptor to memory, the methods in
DirectRasterGraphics maps and unmaps portions of the
drawing surface to data arrays within writable raster objects.
A call to mapRaster() will return a writable
raster object, encapsulating the image data of the drawing surface
in the requested domain. The data encapsulated by this raster
object can be modified using the WritableRaster API,
or the data buffers can be retrieved from the raster, so that the
data arrays can be manipulated directly. When the raster image has
been modified as desired, the data can be resynchronized with the
drawing surface by calling mapRaster().
As with mmap() and munmap() the
methods may work by direct manipulation of shared memory,
(i.e. the raster object directly wraps the actual image data of
the drawing surface), or may make a private copy that is resynched
when the raster is unmapped. The backend may choose to implement
either mechanism, and the pipeline code should not care what
mechanism is actually used. This design allows us to make full use
of speedups such as X shared memory extentions when available.