An Extensible Automated World VSX Shell GAX DSL

An Extensible & Automated World VSX, Shell, GAX, DSL, Factories…oh my!! Santosh Benjamin santosh. benjamin@yahoo. com http: //santoshbenjamin. wordpress. com

Agenda VSX 101 Guidance Automation Domain Specific Languages Software Factories Roadmap Resources

VSX – ‘Am I Bothered? ’ IDE – Vast, feature rich – A long journey from the old days (pre VS 6. 0) – Extensibility is crucial Not possible to fulfil every single developer need For Developers – Extensibility allows you to fit the tool (IDE) to the process For ISVs (The Catherine Tate Show) – Extensibility allows you to make the tool (IDE) your platform …therefore – We should be bothered !!!

VSX Options Packages Macros VS User Customization Developer Reach Cost of integration Level of integration Add-Ins

Visual Studio Templates Project & Item templates – – Reusable and customizable project and item stubs “New Project” and “Add New Item” dialog boxes Can be used to provide starter kits Accelerate the development process Easy deployment: just a. zip file – Content files: source code files, embedded resources, project files, . . . – Metadata stored in a “. vstemplate” file (XML) – Copy to a predefined folder (local or on the network) DEMO – Out of the box templates & locations – Creating and customizing a template Limitations – Capabilities are limited (templates are “dumb”) – Can be extended using IWizard interface to make them a little more dynamic

The VS Automation Model Automation object model contains >140 objects, allowing control of the IDE from a solution level all the way to a single parameter of a single function The Legacy – Old components never die – they only get wrapped in layers and buried deep The COM has not gone away In the old days the common interface was IUnknown (…and for most of us it is still not known )

Macros can leverage nearly all of the features of the. NET Framework, as well as most of the automation model Macros accessed via Tools | Macros have some limitations – – – Macros can only be written in VB Only source code distribution – no binaries Macros cannot be used to implement new tool windows, commands, Tools Options pages

Add-Ins also leverage the automation model. Add-ins can implement new tool windows, commands, Tools Options pages Seamless addition into the IDE Choice of language (VB/ C# / C++) Simple registration using XML ( no more COM registration ) However, Add-ins cannot implement new document types, new project types, new debug engines, etc. which require a package

Wizards In VS, wizards generally ask a series of questions that solicit input from a user, and then use the results to generate code. Do not always have UI. Can be programmed to invisibly generate code behind the scenes. 3 Types – – – New Project wizards —to generate new code for a particular type of project, . Add New Item wizards —to add new items, such as Web forms, text files, HTML pages, XML pages, and so forth, to a project. Custom wizards— Called directly from add-ins, macros, or other types of code. They may or may not display a UI. In either case, they generate code. This type of wizard is used the least often. IDTWizard – the ‘original’ one. Implemented using vsz files, HTML and script code IWizard – new kid on the block in VS 05 and VS 08. C# and VB project systems primarily use this. VSTemplates are usually extended using IWizard.

Packages Some users need more power and flexibility than what they get via macros, add-ins and wizards. They may need custom editors and so on. To achieve this level of integration, we need to write packages Actually the VB, VC#, VC++ project systems are implemented as packages on top of the basic “IDE shell”. VS Packages are created using the Visual Studio SDK in any language

Package Extensibility Points IDE – Project systems, Languages, Editors, Tool Windows, –. NET Designers, Menus & Commands, Toolbox Integration, Tools Options Page, Property Browser – Help Integration, Setup and Deployment Debugger – Custom Debug Engine, Expression Evaluators , Type Visualizers Source Control – Project systems enabled to participate in source control – Source Control Provider ( Plug-in or Package ) Data Sources – Design-time Data Source Drag-n-Drop – Data Source Provider …. and more

Visual Studio SDK A developer suite, framework /platform that allows you to achiever finer control and deeper access into the IDE With this, we now get access to the same interfaces and internals of the platform as the teams who write the IDE itself. Key Concepts – Experimental Hive The SDK creates the separate registry hive to do prototyping work Samples auto-register in Experimental Hive The new SDK enables a “reset all” for the experimental hive – DLK and PLK Developers use the Developer License Key, provided with the SDK Shipping requires Package Load Key If you distribute your package to other developers without a PLK, then they need to have the SDK to run it. – Setup and Deployment Final product needs to author a setup and deployment project that handles: – Presenting your company EULA – Install & Registration

Package Architecture Your Managed Package or MPF 8. 0 Your Native Package Interop Assemblies Native Interfaces Visual Studio 2005 is the platform Native Interfaces provide extensibility points Interop Assemblies enable managed code access SDK Provides – – Native Interface IDLs Interop Assemblies Samples, Doc, templates, etc Package Wizard

The Visual Studio Shell A streamlined IDE on which you can build tools etc for specific markets (like Eclipse). The shell has always been there (in VS 2005 it was referred to as Premier Partner Edition) but not as a separate product with its own identity as it is now. Integrated Mode: – Optimized for programming languages (e. g. ) Iron Python) – Applications built on the integrated Shell will automatically merge with any other editions of Visual Studio installed on the same machine. Isolated Mode – Optimized for specialised tools (e. g. ) Storyboard Designer – Applications built with the isolated Shell will run side-by-side with any other editions of Visual Studio installed on the same machine.

Extensibility Map - Summary Type Benefits Limitations Customization IDE / VS Templates Simple Cannot control IDE, can only customize Macros Full control of IDE –leverage automation model and framework Cannot add features Provided in source • Only written in VB Add-ins Full control of IDE Compiled for performance and protection of source Simple registration Can implement : Cannot implement: • new document types, • new project types, • new debug engines, • new tool windows, • commands, • Tools Options pages Packages Full control of IDE Compiled for performance and protection of source Full ability to add new features Complex development model

VSX In Action - Demos Macro Add-In with Wizard SDK – Tool Window

Agenda VSX 101 Guidance Automation Domain Specific Languages Software Factories Roadmap Resources

Guidance Automation 101 – What’s the big deal? We all have good intentions – We want a consistent way of developing software – We want our teams to share and follow standards and good practices So what’s the problem? Doh! Is that the architecture standards document? !!! ‘Size’ is only 1 problem. Arguably the more pressing problem is that its passive. Pages and pages of ‘guidelines’ And we all know how enthusiastic we are about reading through tons of info to get to the part we want (especially if it was written by someone else ) What we need is a way to make all this carefully (sometimes painstakingly) collected material available at the point of need and even better, make it actually “executable”.

Enter Guidance Automation The Aim – – – Make reusable code and pattern assets available in Visual Studio Integrate reusable code into applications Guide developers through complex procedures Built and used by Microsoft patterns & practices – Built upon Visual Studio SDK – Used extensively in their various “Software Factories”

Guidance Life Cycle Only the package author needs GAT. The package user (usually a developer) only needs GAX to run the package and produce source code. The end product of all this is source code. The end user only sees a web app or winforms app and doesn’t care how it was built. There are NO runtime library requirements for the end user.

GAT – The Key Elements Recipes automate activities that developers would usually perform manually, often by following a series of instructions. Actions are atomic units of work called in a defined sequence by recipes. The sequence is specified in the recipe definition. “T 4” templates A Text Template Transformation Toolkit (T 4) template consists of a combination of text and scriptlets (expressions in VB/C# evaluated at runtime) Wizards are value gathering strategies used to gather values from recipe arguments. . Type Converters Type converters validate the value of a field and convert them from their user interface representation to a type representation VS Templates GAT allows you to associate these with recipes to make solution/project/item generation very parameterised

Demo – GAT In Action Using the Web Service Software Factory (Dec 2006 Refresh) Inspecting the internals of the Service Factory

Building A Guidance Package So how would you build one of those things? A Guidance Package is a VS solution with a specific layout and some specific artefacts in it. It is not trivial to build and easy to get wrong. To make things easier we have a “Meta Package” – a “package”

Building a GP - Demo Using the out of the box Meta Package Using the Clarius Meta-factory (part of Software Factories Toolkit) – Enhances the VS out of the box GP with an enhanced structure & compilation settings, – more context driven recipes and visual elements – Some advanced techniques such as GAX Scripting – Provides a much better authoring experience

VSX, SDK and GAT VSX (vs. ) GAX – Without VSX there would be no GAT – GAX/GAT is a vertical application of VSX for a specific purpose – With VSX you can build tools & products that have nothing to do with GAX/GAT. . but then while building the tools GAT can help!! GAT meets the SDK – SDK has a steep learning curve – GAX driven tooling allows a streamlined approach to developing with the SDK – Examples VSSDKAssist Factory – allows us to create, configure, deploy and test various items such as Commands, Services, Tool. Windows, Custom projects etc VSShell. Assist Factory – allows us to create, configure and test shell setup projects, load keys, splash screens etc

Some Best Practices for GAT* Use GAT when – You have a specific problem with a specific solution that Is automatable Must be executed multiple times Has a development time or risk that is too high to do manually – You can’t use “dumb” Visual Studio Templates – You want to prepare for building a Software Factory Do not use GAT when – You will use the Guidance Package only once – ever – You can’t/don’t want to invest time in the learning curve – You just think it’s cool * Jelle Druyts – Deep Dive into GAT

Authoring Guidance Packages * There is a learning curve * – Do not involve everyone – Make 1 person/team responsible for GAT – Once the knowledge is there, investment pays off quickly Don’t try to build everything at once * – Start automating simple scenarios – Incrementally increase complexity – Add templates, logic and guidance along the way Borrow / Reuse / steal !! – Don’t reinvent the wheel – Most of the factories have source code and that’s available for non commercial reuse in your own stuff (check the license carefully) – At the very least you can see how someone else has implemented a similar idea to yours. – The Guidance Extensions Library (GEL) contains a collection of recipes, actions etc used by P&P Factories and made available for us to use * Jelle Druyts – Deep Dive into GAT

Agenda VSX 101 Guidance Automation Domain Specific Languages Software Factories Roadmap Resources

Domain Specific Languages Domain-Specific Languages (DSLs) typically are small, highly focused languages used to model and solve some clearly identifiable problems which an analyst, architect, developer, tester, or system administrator must handle. Some examples of traditional DSLs: – SQL for data manipulation, – XSD for XML document-structure definition, etc How does it apply to us? – Most teams can talk about their software in very ‘team specific lingo’ – using specific names for recurring types of components, patterns. – The area the team works in is their “domain” and the terminology forms their “language”. – There is now technology available in this area for us to build on.

DSL’s – The Visual Element Just being able to use a DSL will help teams get more consistency in system design and implementation. A DSL does not have to be visual. However, being able to put a ‘face’ to these things (if possible) helps in understanding and communicating the system better Some teams use custom Visio stencils and some go a step further and use the Visio automation model to generate code from the stencils, generate documentation and so on. But there is no common framework around Visio code generation nor a platform to make things easier This is where “DSL Tools for Visual Studio” comes in

DSL Tools For VS – Domain-Specific Language Tools allow Visual Studio 2005 developers to create their own graphical designers and code generation tools like the ones you find in Visual Studio today, such as the Class Designer, Application Designer etc. – Using DSL Tools, designers can specify the domain model (semantics) for a new language; create a custom graphical editor that uses a domain-specific diagram notation (syntax); and construct custom text templates that use models created using the graphical editor, to generate source code and other files.

DSL’s examples…(1) Service Factory Modelling Edition – Service Contract Model, Data Contract Model, Host Model

DSL examples…(2) VS Team Architect – Application Designer , Logical Data Center Designer VS Team Developer – Class Designer Biztalk – Orchestration Designer, Pipeline Component Designer WF – Workflow Designer

DSL vs UML is very general purpose. It can be considered as a DSL but the domain would be the whole of software development. The focus of a typical DSL is much narrower. It picks on specific verticals and (sometimes) specific types of applications in that vertical Some might argue that you could achieve similar results with “UML Stereotypes” but that’s quite hard and tooling is not easily available to translate those stereotyped models into your specific code AND back again Steve Cook , a leading DSL proponent, was on the IBM board that worked on the standardisation of UML so he is definitely an authority on what it can and cant do However UML hasn’t gone away and it isn’t going away. The Class Designer extends the familiar UML notation to accommodate things that are unique to. NET classes (attributes etc). Some new tools in “Rosario” will allow architects to reverse engineer code into sequence diagrams and other visualizations that will have many UML elements in it.

Agenda VSX 101 Guidance Automation Domain Specific Languages Software Factories Roadmap Resources

What is a Software Factory? “A Software Factory is a software product line that configures extensible tools, processes and The ‘easy’ (Pn. P)template based on a software factory schema to automate content using adefinition software factory the and maintenance of variants of anof archetypical product assets. by adapting, “Adevelopment software factory is a structured collection related software When a assembling and configuring framework-based components. ” software factory is installed in a development environment, it helps architects and Jack Greenfield, Keith Short, Software Factories developers predictably and efficiently create high-quality instances of specific types of applications. ” The ‘Official’ Definition

In more concrete terms…. * A ‘Tool’, to automate the assembly and configuration of solutions addressing a specific, understood problem domain Runs within, and integrates with, Microsoft Visual Studio Uses models, designers, editors, wizards, etc. , to provide abstractions of the solution domain Outputs mostly generated solution (product) that may require manual customizing to complete Also contains executables, readables, reusables. * Build Your Own Software Factories – Jezz Santos, Edward Bakker

What would we build it with ? * Asset Tool/Technology Automated Action GAX/GAT recipe, with wizard Code/artifact Templates Text Templating Engine Project or Solution Template Visual Studio template with GAX/GAT recipes Domain Specific Language DSL Toolkit Logical Application & System Design Visual Studio Team Edition for Architects (SDM SDK) • GAX/GAT available from patterns & practices : All other tools available in Visual * Build. Studio Your. SDK Own Software Factories – Jezz Santos, Edward Bakker

Its like an umbrella… This SF approach brings in all the good stuff – – Patterns Guides Guidance Automation Domain Specific Languages . . and puts them all under one roof

Demo Service Factory Modelling Edition – Service Contract Model – Data Contract Model – Host Model

Factories. . Things to remember * Solutions that are going to be modelled in the factory should be built before the factory itself is built. Keep in mind that the factory-building process will take two to three times longer than a one-off product such as building a Web service from scratch. Finally, keep in mind that ROI will not come until the factory has been put to use at least three times. * Build Your Own Software Factories – Jezz Santos, Edward Bakker

Demo …. outcome So what have we seen? – The DSL approach provides a higher level of abstraction – This itself gives the Service Factory more value than its previous incarnation which could be considered as “code generation on steroids”.

Factories…. Just the beginning What we have seen isn’t all there is to software factories… in fact its just the beginning The P&P software factories (esp Service Factory) are now moving in the right direction with the DSL abstraction over code generation and the incontext guidance But in reality we are only scratching the surface of Software Factories as envisioned by Jack Greenfield and others. There advanced things like the “Software Factory Schema” , Software Product Lines and so on.

Any feelings of “déjà vu”…? A common criticism is that this like the 80’s all over again with respect to CASE Tools As with UML , the key players here have “been there, done that”. Software Factories now takes a more modular approach than the top down process heavy approach that was seen with the CASE tools. You can pick and choose what suits your development approach best and the tooling does NOT try to dictate the process. Examples: Even in the area of “layered” VS solutions, the WSSF allows you to have a single project for everything if you don’t want 8 separate assemblies. Again, you can use just the service interface models and do everything else in your own way , or just use the Repository Factory to control the data layer and do the rest yourself. So no, its not CASE all over again. But if you’re a die-hard cynic , perhaps it would be best to wait and watch!!

Agenda VSX 101 Guidance Automation Domain Specific Languages Software Factories Roadmap Resources

Roadmap Visual Studio (& Team System) – In Rosario, Hawaii and beyond , Factories are going to be baked into the product. Edward Bakker has some posts on how this manifests itself in Rosario – “Team Factories” – from the initiation of the project on TFS, through to work item allocation, code generation, testing and maintenance. Service Factory – Guidance to encompass Biztalk, WCF, WF, Astoria etc (will start as documented guidance and may eventually become full fledged factories) – REST & RSS capabilities – Leverage Metadata repositories

Agenda VSX 101 Guidance Automation Domain Specific Languages Software Factories Roadmap Resources

Resources …(1) Books – Software Factories (2004) - Jack Greenfield, Keith Short – Working With Visual Studio 2005 – Johnson, Skibo, Young – Professional Visual Studio Extensibility (2008) - Keyvan Nayeri VSX Articles – VS Templates and Starter Kits – Developing Visual Studio Wizards (IWizard) GAX/GAT Articles – Introduction to GAT (MSDN) – Guidance Automation Series (Jelle Druyts) Blogs (Hall Of Fame) – VSX: Carlos Quintero – SF: Jezz Santos , Edward Bakker , Jelle Druyts , – Clarius Blogs : Pablo Galiano, Victor Garcia, Daniel Cazzulino Newsgroups – MSDN Forums (VSX, GAT)

Resources …(2) Webcasts & Videos – – – Deep Dive into GAT – Jelle Druyts MSDN Belux Packaging Design and Architecture Guidance Introduction to Service Factory Build Your Own Software Factory (EMEA Spotlight Tech. Ed 07) Visual Studio DSL Tools – Stuart Kent Software – GAX (Feb 08) and GAT (Feb 08) Factories – – – Web Service SF , Web Client SF, Repository Factory Smart Client SF, Ent Lib 3. 1 Application Block Software Factory Biztalk Solution Factory Misc – My blog – starting a series on GAT shortly

Summary Visual Studio is highly extensible and can now justifiably be considered more than just an IDE , it is a tools platform. This is a good time to start getting into the field of VS extensibility. Guidance Automation leverages the extensibility of VS to allow us to – Leverage common best practices in an executable form – provide ‘active’ guidance to development teams DSL Tools provide us a platform to build visual languages and link into code generation etc Software Factories bring everything together to help us along the road to a higher level of consistency and accuracy in software development.

Q&A