Session Code CLI 341 Using Graphics Composition And

Session Code: CLI 341 Using Graphics, Composition, And Animation In Your Longhorn Applications Greg Schechter Software Architect Microsoft Corporation gregsc@microsoft. com

Login. Screen

What’s Behind The Demos? What’s demo? What’s platform? Thin demos, rich platform Powerful programming model for media Rich set of graphics, imaging, multimedia, and text primitives Powerful ways of composing those primitives Animation support throughout Resolution independence But…before we get there…

Other Architectural Tenets Very aggressive use of the GPU High frame rate (~ monitor refresh rates) Low CPU utilization 2 -D shapes through the 3 -D pipeline App writer doesn’t tune for the hardware Software fallback always available Printing Support Remotability Support Coming – Rich 3 D Support Not discussing these items particularly in this session (see CLI 340)

Presentation Data Avalon UI Object Media Adaptive UI Engine Designer Services Personalization and Profiling Services Hardware Rendering Controls Membership and Security Services Media Services Software Rendering and Sinks Base Operating System Services CLR Application Deployment Engine (Click-Once) GDI/GDI+ DDI Input Manager Memory Manager Global Audio Engine Audio Drivers Power Manager Direct. X Graphics Mini port Config Manager Kernel Group Calendar Media Document … Memory Manager Data. Set Data Model Relationships Security Reference Monitor LPC Facility Loader Security Make the connection System Services Transaction Real-Time Activities Signaling Federation Serialization Demand Activation and Protocol Health Network Services Storage Virtual Shadow Copy Service Identity & Security System PNRP File Replication Service Virtual Disk Service Filter Redirectors Distributed File System Manager Transacted NTFS Universal Data Format FAT 16/32 Native SIP Wi. Fi TCP Listener UDP Listener IPC Listener Internet Connection Firewall Management Protocols Filter Engine Cache Manager IO Manager SCSI/FC Hardware Abstraction Layer Routing Message Encoder Network Class Library Code Execution Transaction Coordinator Logging Service Collaboration History Eventing Communications Manager (Port) Providers Extensions People and Groups Queuing Channel Security Transport Channels (IPC, HTTP, TCP…) SQL XML Items Policy Engine Channels (Datagram, Reliable, Peer, …) Hosting Layer Kernel Transaction Manager Messaging Services Connectivity Object. Spaces Base Class Libraries Lightweight Transactions Process Manager Schemas People File. System Services (Meta. Data. Handlers. . ) Transactions Graphics drivers Service TPC, UDP IPV 4, IPV 6 IPSEC Device Drivers 802. 3 802. 11 . . QOS HTTP Listener Kernel Mode Plug and Play Window Manager Info. Agent (Preference. Rules. . ) Collaboration Framework Animation and Media Capture and Composition Processing Sourcing Indigo ADO. NET XML Synchronization (Win. FS, Win 32. . ) Page/Site Composition Desktop Composition Engine T/SQL Services Application Services Presentation Object Manager Win. FS ASP. NET Desktop Window Controls Interop Manager Engine Desktop Services Communication Models Document Windows Forms

Avalon Document Desktop Services UI Media Desktop Window Manager Presentation Object Manager Desktop Composition Engine Media Services Animation and Composition Media Processing Capture and Sourcing Hardware Rendering Software Rendering And Sinks Base Operating System Services CLR Direct X Graphics GDI/GDI+ Window Manager Global Audio Engine DDI Input Manager Audio Drivers Base Services Kernel Hardware Abstraction Layer Make the connection Graphics Drivers Direct. X Graphics Mini port

Tools Client Application Model Avalon Windows Forms System. Windows. Forms Web & Service Application Model Data Systems Application Model Win FS ASP. NET / Indigo Yukon System. Data. Sql. Server System. Storage System. Web Mobile PC & Devices Application Model Compact Framework Command Line System. Console Mobile PC Optimized System. Windows. Forms NT Service System. Windows System. Service. Process System. Search System. Windows UI Element Explorer Documents Media Controls Text Element Controls Side. Bar Navigation System. Windows. Forms Relevance Panel Design Data. Set Sql. Types Mapping Sql. XML Object. Spaces Web. Controls Control Adaptors Print Dialog Html. Controls Design Mobile. Controls Recognition Synthesis System. Natural. Language. Services System. Remoting System. Runtime. Remoting Query Activities Oracle. Client Schema System. Web. Services Description Discovery Contact Media Location Audio Message Video Document Images Event Protocols Transport Queue Port Pub. Sub Channel Router Service Policy System. Net System. Web Personalization Query System. Message. Bus Peer Group System. Xml Serialization Uddi Ole. Db. Client Relationship Xpath Transient. Data. Session Signaling. Session Media Core Schema Active Directory Object. Space System. Speech System. Drawing Real. Time. Endpoint System. Directory. Services Web. Service Item System. Collaboration System. Discovery Odbc. Client System. Storage System. Web. UI Page System. Help System. Messaging Sql. Client Logging Forms Design Monitoring Control Notification Shape System. Data Annotations Animation Dialogs Shapes Ink Communication Data Presentation Caching Session. State Http. Web. Request Network. Information Ftp. Web. Listener Sockets Ssl. Client. Stream Cache Web. Client Fundamentals Security Base & Application Services System. Timers System. Globalization System. Serialization System. Threading System. Text System. Design System. IO Ports System. Collections Serialization Compiler. Services System. Component. Model System. Reflection Interop. Services Configuration System. Web. Configuration System. Security Generic System. Code. Dom System. Runtime System. Windows. Trust. Management System. Enterprise. Services System. Transactions System. Web. Security System. Message Bus. Security Authorization Access. Control Policy Principal Cryptography Token System. Web System. Message. Bus. Configuration Administration System. Configuration Management Permissions Credentials Deployment/Management System. Resources System. Management System. Deployment System. Diagnostics

Agenda Developer Profiles Shapes Resolution Independence Graphical Resources Animation Drawing. Context The Visual class

Levels Of API UI/Content Designer Framework Element Shapes Image Video Custom Control Drawing. Context Visual API Typical App Author Control Developer Graphics-heavy App Developer

Levels Of API Framework Elements – Shapes, Text, Image, Video Integrates with property, layout, and styling systems Custom Control Drawing – Drawing. Context Two means of expression: XAML Markup and Code Exposed primarily to Control authors Visual API Simplest level of programmability, yet very composable Imperative feeling API – Draw. Line, Draw. Rectangle, Draw. Text, Low-level programmability Use Framework Elements(brushes, until proven Uses the same “resources” paths, otherwise etc. ) used everywhere Push. Opacity Access to full graphical hierarchy else Applications may pull from multiple levels No hierarchical structure More for expert-level developer with deep involvement in graphical aspects of application Bedrock for the Avalon Presentation Framework (controls, panels, styling, shapes, input, etc. )

Markup – Shapes, Image And Video <Ellipse Radius. X="80" Radius. Y="40" Fill="Orange" /> Framework Elements Integrate with property system for property inheritance, databinding, styling Participate in layout XAML Markup = Object Model

Shapes: Geometry + attributes Rectangle, Ellipse, Line, Path, Polygon, Polyline, Glyphs Image and video Controls that draw images and video Text – control that renders text strings Includes very rich typographical support Canvas – absolute positioning container Shapes & arbitrary Controls

Properties On Shapes And Controls Fill Stroke and stroke properties Composition effects Opacity Coming – Image Effects and Blend modes Clipping

Rectangle (From Demo) <Rectangle Fill="Orange" Stroke="Yellow" Stroke. Thickness="5" Radius. X="10" Radius. Y="10" Height="100" Width="100" />

Defining Shapes On A Canvas

Markup = Object Model The markup compiler knows nothing about Rectangle XAML <Rectangle Fill="Orange" Stroke="Yellow" Stroke. Thickness="5" Radius. X="10" Radius. Y="10" Height="100" Width="100" /> Visual Basic. NET Dim R as New Rectangle R. Fill = Brushes. Orange R. Stroke = Brushes. Yellow R. Stroke. Thickness = New Length(5. 0) R. Radius. X = New Length(10. 0) R. Radius. Y = New Length(10. 0) R. Height = New Length(100. 0) R. Width = New Length(100. 0)

Code and Zoom

Resolution Independence What’s the opportunity/problem? 204 ppi monitors are new, but they’re here (~4. 5 x today’s typical 96 ppi) 3840 x 2400; 22” diagonal; visit booth! 150 ppi laptops are here now 15. 4” screen; $1, 450 ~2. 5 x pixel density vs. 96 ppi Applications are typically written using pixel-oriented units That leads to problems…

96 ppi Versus 204 ppi

Resolution Independence In Longhorn works in “device independent pixels” Measures of 1/96 th of an inch Thus, a drawing on a 204 ppi monitor will be the same physical size as on a 96 ppi monitor But it will be much more detailed and crisp Use vector graphics to define your interfaces! Expanded Color Support sc. RGB supported through the entire pipeline, supporting wide gamut and high dynamic range Color components specified as floats at 32 bpc s. RGB (8 bpc, 2. 2 gamma) still always available

Graphical Resources Examples above use simple property values – colors, lengths Avalon extends these properties to be very rich Example abstract graphical resources Brush – for filling an area Transform – for scaling, rotating, positioning, skewing Geometry – for 2 -D contours and paths Image. Source – for raster data

Brushes Defines how to fill an area Brush (abstract base class) Solid. Color. Brush Gradient. Brush Linear. Gradient. Brush Radial. Gradient. Brush Tile. Brush Image. Brush Drawing. Brush Video. Brush (coming) Visual. Brush (coming) Can contain animate values

A Variety Of Static Brushes

Brushes In Markup General pattern Compact syntax <Rectangle Fill="Orange" /> Relies on Brush Type. Converter to translate from “Orange”. More general syntax <Rectangle …> <Rectangle. Fill> <Solid. Color. Brush Color="Orange" /> </Rectangle. Fill> </Rectangle> Suggests the pattern for all other rich resource usage

Linear. Gradient. Brush <Rectangle …> <Rectangle. Fill> <Linear. Gradient. Brush Start. Point="0, . 75" End. Point="1, . 25"> <Linear. Gradient. Brush. Gradient. Stops> <Gradient. Stop Offset="0. 0" Color="Cyan" /> <Gradient. Stop Offset="0. 5" Color="Magenta" /> <Gradient. Stop Offset="1. 0" Color="Yellow" /> </Linear. Gradient. Brush. Gradient. Stops> </Linear. Gradient. Brush> </Rectangle. Fill> </Rectangle> This is the key to rich visual composability! Composability? Abstracting out a concept and being able to apply it in another context, and do so repeatedly, building richer and richer apps.

Linear. Gradient. Brush In Code Markup <Linear. Gradient. Brush Start. Point="0. 0, . 75" End. Point="1. 0, . 25"> <Linear. Gradient. Brush. Gradient. Stops> <Gradient. Stop Offset="0. 0" Color="Cyan" /> <Gradient. Stop Offset="0. 5" Color="Magenta" /> <Gradient. Stop Offset="1. 0" Color="Yellow" /> </Linear. Gradient. Brush. Gradient. Stops> </Linear. Gradient. Brush> C# Linear. Gradient. Brush lgb = new Linear. Gradient. Brush(); lgb. Start. Point = new Point(0. 0, 0. 75); lgb. End. Point = new Point(1. 0, 0. 25); lgb. Gradient. Stops. Add(new Gradient. Stop(Colors. Cyan, 0. 0)); lgb. Gradient. Stops. Add(new Gradient. Stop(Colors. Magenta, 0. 5)); lgb. Gradient. Stops. Add(new Gradient. Stop(Colors. Yellow, 1. 0));

Applying A Cool Brush To An Element

Animation – A Fundamental Service Animation supported on everything we’ve seen Shapes – positions, sizes, stroke thicknesses, borders Arbitrary controls – positions, sizes, clips, transforms, etc. Graphical Resources – they’re intrinsically animate E. g. , Brushes may be constructed with animate inputs (Gradient. Stop color, for instance) Anything that this brush “paints” is now animate User-defined properties on user-defined controls!

Animate Shapes And Brushes

Animation – Example Markup Ellipse with Animating Radius. X <Ellipse Fill="Green" Center. X="120" Center. Y="120" Radius. Y="80" Radius. X="90" Stroke="Black" Stroke. Thickness="5"> <Ellipse. Radius. X> <Length. Animation. Collection> <Length. Animation From="90" To="180" Auto. Reverse="true" Duration="2. 0" Repeat. Duration="Indefinite" /> </Length. Animation. Collection> </Ellipse. Radius. X> </Ellipse>

Animation What’s provided? Typed Double. Animation, Point. Animation, Color. Animation, etc. Timed Duration, Speed, Begin time, auto-reverse, repeating, acceleration, deceleration Programmable Applications can start, stop, seek, pause, etc. animations Key Framed Support discrete From/To values, Key Frames, and Key Splines Advanced concepts Hierarchical – animations may be “parented” to other animations/timelines Composing – simultaneous animations can “compose” and do the right thing Coming: Integration with Styles and Interaction

Animation We do the heavy lifting Applications should not build their own frame loop/animation system Time-based (not frame-based) == scalable System samples and processes animations appropriately Some animations are run on the asynchronous composition thread, and are remotable Apps can provide their own classes of animations to plug into the system System can monitor power usage to dial back frame rate (not in the current bits)

Custom Control Drawing Control implementers will typically use “control composition”, and not do explicit drawing. CLI 301 – “Building Rich, Interactive Controls” But, when they do custom drawing, implement On. Render(Drawing. Context ctx) Drawing “instructions” are issued through Drawing. Context Draw. Rectangle(Brush, Pen, Rect) Draw. Geometry(Brush, Pen, Geometry) Draw. Image, Draw. Line, Draw. Ellipse, Draw. Text, Draw. Video Push. Transform, Push. Opacity, Push. Clip, Pop

Drawing. Context And Control. On. Render() These “instructions” are retained Resolution independent rendering (metafile, scenegraph, displaylist) May contain animations When the control needs to draw, it walks these instructions, and issues them On. Render() gets called when underlying model for the control changes Not when the control needs to be redrawn Important for remoting, for animation, etc. System calls application back far less often.

Visuals Primary unit of construction for the Longhorn “Presentation Core” Everything “visible” in the “Presentation Framework” builds upon Visuals are arranged in a Visual Tree that represent 2 D composition; Each Visual has Visual children, and optionally has “render content” Rendering of the tree occurs by rendering the content, then traversing the children Hit Testing and Animation fundamentals are here

Top-Level Objects Generally, Avalon owns the frame loop, and takes care of target surface management, rendering loop, ticking the animations, etc. There are important scenarios where the Avalon system cannot own the frame loop (for instance, rendering to bitmaps for encoding) When apps want to go somewhere other than the screen. “Top-Level Objects” exist to meet this need; for very advanced users, but provide Explicit rendering to Image. Data rasters Control over ticking the animations Ability to import and wrap other datastructures to participate in Avalon graphs A “Visual. Manager” to mediate these operations Not available in the PDC bits

Summary Very integrated and diverse forms of visual media Extreme composability of that media Animation supported throughout Deep, natural integration with all of Avalon Resolution independence Different levels to program to

Community Resources Get Your Questions Answered! More PDC Sessions CLI 340: Part 1: Overview CLI 342: Part 3: Audio, Video, Imaging (next) CLI 300: Building Apps With Controls and Dialogs (happened) CLI 301: Building Rich, Interactive Controls (happened) CLI 303: Application Model Fundamentals CLI 304: New UI Possibilities CLI 308: Visual Theming Client Lounge: Concourse Foyer Ask The Experts: Tuesday 7 P. M. – 9 P. M. in Hall G, H Newsgroup: microsoft. public. windows. developer. winfx. avalon Longhorn Dev Center: http: //msdn. microsoft. com/longhorn PDC Weblogs: http: //pdcbloggers. net Now! Questions? Comments?

© 2003 -2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.