VENDOR LANDSCAPE Review the PPM Solution Vendor Evaluation

VENDOR LANDSCAPE Review the PPM Solution Vendor Evaluation Info-Tech Research Group 1

Review Info-Tech’s Vendor Landscape of the PPM solution market to identify vendors that meet your requirements 2. 1 Vendors Evaluated Each vendor in this landscape was evaluated based on its features, product considerations, and vendor considerations. Each vendor was profiled using these evaluations and, based on its performance, qualified and placed in specific use-case scenarios. Info-Tech Research Group 2

2. 1 These vendors were included due to consideration of their market share, mind share, and platform coverage Vendors included in this report provide a comprehensive, innovative, and functional solution for managing project portfolios. Included in this Vendor Landscape: Clarizen: A collaborative PPM solution built on an enterprise-grade platform. Eclipse PPM (Upland Software): Offers a product family capable of assisting businesses at all stages of growth. Innotas: Saa. S specialists in project portfolio management (PPM) and application portfolio management (APM). Liquid. Planner: Committed to designing a tool that accommodates the real-world nature of portfolio management. Project Insight (Metafuse Inc. ): Leveraging PPM software and service experience to optimize the PMO. Microsoft Project Server 2013: Integrating well-established enterprise product offerings into a PMO specific tool. Planisware: E. U. -based and globally-oriented PPM vendor with a strong R&D customer base. PLANTA Portfolio (PLANTA Projektmanagement-Systeme Gmb. H): Experienced and established vendor in the E. U. Sciforma: Helping PMOs manage modern portfolios containing both traditional and agile project methodologies. Team. Dynamix: Higher-ed specialist with a solution other non-profit or government organizations may find useful. Workfront (formerly At. Task): Assisting PMOs regardless of the nature of the project portfolio (marketing, IT, etc. ). Info-Tech Research Group 3

2. 1 Table Stakes represent the minimum standard; without these, a product doesn’t even get reviewed The Table Stakes Feature: Project Management What it is: Traditional scheduling and tracking functionality Project Portfolio Management Multi-project overview with comprehensive, highlevel reporting on progress and cost Resource Management Ability to display which resources are available or over-assigned and to assign projects/tasks to specific resources Timesheet Management Ability to log time spent on projects in real-time or retroactively Document Management Offers check in/check out versioning controls on documents Lifecycle Management Continuity between projects through management of request ideas to the completion of a project with unique workflows per division What does this mean? The products assessed in this Vendor Landscape meet, at the very least, the requirements outlined as Table Stakes. Many of the vendors go above and beyond the outlined Table Stakes, some even do so in multiple categories. This section aims to highlight the products’ capabilities in excess of the criteria listed here. Info-Tech Insight If Table Stakes are all you need from your PPM solution, the only true differentiator for the organization is price. Otherwise, dig deeper to find the best price to value for your needs. Info-Tech Research Group 4

2. 1 Advanced Features are the capabilities that allow for granular differentiation of market players and use-case performance Scoring Methodology Info-Tech scored each vendor’s features on a cumulative four-point scale. Zero points are awarded to features that are deemed absent or unsatisfactory, one point is assigned to features that are partially present, two points are assigned to features that require an extra purchase in the vendor’s product portfolio or through a third party, three points are assigned to features that are fully present and native to the solution, and four points are assigned to the best-ofbreed native feature. Feature: What we looked for: Manage Reactive Work Ability to request, assign, and track ad hoc work that happens outside of the project scope Zero-Adoption Capability The solution maintains the ability to conduct portfolio-level project planning and status reporting without team-level adoption of timesheet and task-data maintenance Agile Support Functionality tailored to agile project management methodologies (e. g. Kanban boards, burn-down charts, etc. ). End-User Portal The solution includes a portal that enables individuals, internal or external to the organization, to access reports or dashboards without a full licence or profile What-If Planning Ability to test scenarios and review implications of schedule and resource changes; includes the option to implement or cancel all hypothetical scenarios Workflow Management Ability to submit, manage, and be notified of work requests, approvals, and assignments Mobile Support Mobile application or, at least, web functions optimized for mobile For an explanation of how Advanced Features are determined, see Information Presentation – Feature Ranks (Stoplights) in the Appendix. Info-Tech Research Group 5

2. 1 Vendor scoring focused on overall product attributes and vendor performance in the market Scoring Methodology Product Evaluation Features Info-Tech Research Group scored each vendor’s overall product attributes, capabilities, and market performance. Usability The end-user and administrative interfaces are intuitive and offer streamlined workflow. Affordability Implementing and operating the solution is affordable given the technology. Architecture Multiple deployment options, platform support, and integration capabilities are available. Features are scored individually as mentioned in the previous slide. The scores are then modified by the individual scores of the vendor across the product and vendor performance features. Usability, overall affordability of the product, and the technical features of the product are considered, and scored on a five-point scale. The score for each vendor will fall between worst and best in class. The vendor’s performance in the market is evaluated across four dimensions on a five-point scale. Where the vendor places on the scale is determined by factual information, industry position, and information provided by customer references, and/or available from public sources. Vendor Evaluation Features Viability Vendor is profitable, knowledgeable, and will be around for the long term. Focus Vendor is committed to the space and has a future product and portfolio roadmap. Reach Vendor offers global coverage and is able to sell and provide post-sales support. Sales Vendor channel partnering, sales strategies, and process allow for flexible product acquisition. Info-Tech Research Group 6

2. 1 Balance individual strengths to find the best fit for your enterprise Vendor Performance Product Vendor Overall Usability Afford. Arch. Overall Viability Focus Reach Channel 3 4 2 3 3 3 4 4 4 3 2 3 3 3 4 4 3 3 4 2 3 2 3 4 4 3 3 2 3 3 3 4 4 4 3 4 2 2 2 4 3 3 3 4 2 3 3 4 4 4 4 3 3 4 2 3 3 3 3 3 4 3 2 2 4 4 0 2 0 3 3 2 3 4 0 3 Clarizen Eclipse PPM Innotas Liquid. Planner MS Project Server 2013 Planisware PLANTA Portfolio Project Insight Sciforma Team. Dynamix Workfront Legend = Exemplary = Good = Adequate = Inadequate = Poor *The vendor declined to provide pricing and publically available pricing could not be found For an explanation of how the Info-Tech Harvey Balls are calculated, see Information Presentation – Criteria Scores (Harvey Balls) in the Appendix. Info-Tech Research Group 7

2. 1 Balance individual strengths to find the best fit for your enterprise Feature Performance Clarizen Eclipse PPM Innotas Liquid. Planner MS Project Server 2013 Planisware PLANTA Portfolio Project Insight Sciforma Team. Dynamix For an explanation of how Advanced Features are determined, see Information Presentation – Feature Ranks (Stoplights) in the Appendix. Legend Workfront = Feature is best in its class = Feature is fully present in its native solution = Feature is present at additional cost = Feature is partially present = Feature is absent Info-Tech Research Group 8

USE CASE 1 Mid-Market: Moderate PPM Maturity Info-Tech Research Group 9

2. 1 Drill Down: Mid-Market – Moderate PPM Maturity Choose a solution to help you establish a strong PPM foundation. • Info-Tech believes that some PPM solutions are more adaptable and appropriate for mid-market organizations that have not yet arrived at full PPM maturity. • Mid-market organizations with moderate PPM maturity need solutions with features that allow them to: o Manage and track ad hoc work (project or nonproject) o Carry out core portfolio management activities (e. g. reporting and resource planning) even if there is low adoption of the solution at the project manager or project worker level • PPM solutions (on their own) will not help you mature your PPM processes. Info-Tech Insight If your organization has low PPM maturity, then you may want to avoid purchasing a commercial PPM tool and instead focus your resources on developing your core PPM processes to a moderate level of maturity. Until these are in place (and adopted) it will be difficult to determine which solution will be the best fit. Consider a non-commercial solution like the one featured in Info-Tech’s Grow Your Own PPM Solution. Indicators of a strong alignment with the mid-market and moderate PPM maturity use case: ü The IT department has 100+ people but is not yet a large enterprise (see the large enterprise use-case below for more details). ü Basic PPM processes are in place (but more governance and standardization are still required) ü There is general awareness of highlevel project status across the portfolio ü Leadership views reports and dashboards on an ad hoc or asneeded basis (but a scheduled formal review cadence may not yet exist) ü Long-term resource planning is nonexistent or limited ü (PPM Maturity Levels 2 or 3) Maturity Level 5: Optimized Maturity Level 4: Managed Maturity Level 3: Defined Maturity Level 2: Repeatable Maturity Level 1: Ad Hoc Establish your PPM maturity level using Info-Tech’s PPM Solution Use-Case Fit Assessment Tool. For a more detailed discussion about PPM Maturity see our publication Establish an Effective PMO for IT (especially slides 41 to 76). Info-Tech Research Group 10

Feature weightings for the mid-market and moderate PPM maturity use-case scenario Feature Weightings Core Features Manage Reactive Work 2. 1 Project workers in mid-sized organizations likely report to multiple managers (functional and project) and have non-specialized roles. Lower PPM maturity means a lower capability for longterm resource planning at the portfolio level. It may also mean intake processes are focused solely on large-scale projects, leaving smaller projects to be handled by individual project managers/workers on an ad hoc basis. Organizations in this category must be able to track all kinds of ad hoc work (project or nonproject) that use up resources. Mobile Support Workflow Management Manage Reactive Work 10% End-User Portal Agile Support 5% 5% 5% 45% 30% Zero-Adoption Capability Fine-grained task-level planning and timetracking may impede PPM adoption and slow down PPM maturity in the long term. Being able to carry out core portfolio management exercises with minimal adoption of the tool at the project manager and project worker level is a strong advantage in this use case. Zero-Adoption Capability Additional Features Mobile Support Agile Support End-User Portal Workflow Management Info-Tech Research Group 11

Vendor considerations for the mid-market and moderate PPM maturity use-case scenario 2. 1 Product Evaluation Features Usability Like all software, usability can help drive adoption of the solution. Affordability Budget is an important factor for mid-market organizations. Architecture The ability to easily integrate with existing software is important for businesses with limited budgets and IT resources. Vendor Evaluation Features Viability Compared to other factors, we believe viability is relatively less important for the mid-market. Focus Significant weight is placed on this category to sort out solutions designed and marketed primarily for large enterprises from those designed and marketed primarily to mid-market enterprises. Reach Support is an important factor for this group but, on the whole, support is not a major differentiator in this market. Sales Partnerships and channels are not a significant factor since vendors focused in the mid-market handle sales, implementation, and support directly. Info-Tech Research Group 12

2. 1 Vendor performance for the mid-market and moderate PPM maturity use-case scenario Mid-Market: Moderate PPM Maturity Best Overall Value Award Sciforma Team. Dynamix: Exceptional ability to manage reactive work combined with its ability to offer core portfolio management functionality – even when adoption of the solution is less than optimal at the project worker/manager level – placed it high in this use case. Leaders in Mid-Market: Moderate PPM Maturity • • Sciforma Eclipse PPM Players in the Mid-Market and Moderate PPM Maturity Scenario • • Clarizen Liquid. Planner Project Insight Microsoft Project Server 2013 Info-Tech Research Group 13

Use-Case Performance Vendor performance for the mid-market and moderate PPM maturity use-case scenario 2. 1 Vendors Info-Tech Research Group 14

Value Index for the mid-market and moderate PPM maturity scenario What is a Value Score? The Value Score indexes each vendor’s product offering and business strength relative to its price point. It does not indicate vendor ranking. 2. 1 On a relative basis, Sciforma maintained the highest Info-Tech Value Score. TM of the vendor group for this use-case scenario. Vendors were indexed against Sciforma’s performance to provide a complete, relative view of their product offerings. Vendors that score high offer more bang-for-the-buck (e. g. features, usability, stability) than the average vendor, while the inverse is true for those that score lower. Price-conscious enterprises may wish to give the Value Score more consideration than those who are more focused on specific vendor/product attributes. *Vendors who scored 0 declined to provide pricing and publicly available pricing could not be found. For an explanation of how Price is determined, see Information Presentation – Price Evaluation in the Appendix. For an explanation of how the Info-Tech Value Index is calculated, see Information Presentation – Value Index in the Appendix. Info-Tech Research Group 15

USE CASE 2 Mid-Market: High PPM Maturity Info-Tech Research Group 16

2. 1 Drill Down: Mid-Market – High PPM Maturity Choose a solution that will allow you to use your PPM processes to create an IT project portfolio generating organization-wide value. • Mature PPM processes provide full value only if they are widely adopted at all levels of the organization. While organizations with mature portfolio processes in place will likely get more out of their solution, there is also the risk that reports and dashboards could be corrupted if project-worker or project-manger adoption of the solution breaks down. The usability of the solution and the ability to manage the portfolio independent of full adoption at the project worker/manger level is still a factor to be considered. • As portfolio governance is established, portfolio managers will turn to PPM solutions to help with resource allocation, resource planning, and producing portfolio-level reports to enable business leaders to make strategic decisions that will shape the business. • Portfolio maturity creates the need to plan and aggregate projects carried out in a wide variety of methodologies (agile or traditional) in a single portfolio. Solutions that offer agile support and integrate agile projects into the overall portfolio will be an asset for mature PMOs. Indicators of a strong alignment with the mid-market and high PPM maturity use-case: ü The IT department has 100+ people but is not yet a large enterprise (see the large enterprise use-case below). ü Standard PPM processes are in place at all levels of the organization (from business leaders down to project workers). ü Projects and resources are regularly planned based on realistic and accurate reports of project demand project supply. ü Projects are approved and prioritized according to the organization’s strategic goals. ü Project portfolio reports and dashboards play an integral role in major decisions made by those in leadership positions. ü (PPM Maturity Levels 4 or 5) Maturity Level 5: Optimized Maturity Level 4: Managed Maturity Level 3: Defined Maturity Level 2: Repeatable Maturity Level 1: Ad Hoc Establish your PPM maturity level using Info-Tech’s PPM Solution Use-Case Fit Assessment Tool. For a more detailed discussion about PPM Maturity see our publication Establish an Effective PMO for IT (especially slides 41 to 76). Info-Tech Research Group 17

2. 1 Feature weightings for the mid-market and high PPM maturity use-case scenario Feature Weightings Core Features Manage Reactive Work Zero-Adoption Capability Agile Support Dedicating resources exclusively to planned project work is often an unattainable luxury for many IT departments (especially smaller ones). Tracking reactive work will still be a concern for high-maturity mid-market PMOs. Maintaining the discipline of task-level input into the solution can still be a struggle even for mature PMOs that have clearer processes and more defined roles. Having a tool that can still offer PPM value even when adoption drops is still an important consideration. Mature PMOs tend to have the appetite and capability to include all projects into the portfolio regardless of the methodology used to manage them. Mobile Support Manage Reactive Work 15% Workflow Management 20% 10% What-if Planning 5% 20% 15% Zero-Adoption Capability 15% End-User Portal Agile Support End-User Portal High PPM maturity means a high level of leadership engagement. The ability to provide easily accessible and high-quality reports to key stakeholders (licensed or non-licensed) is highly beneficial for portfolio managers in these organizations. Additional Features What-If Planning Mobile Support Workflow Management Info-Tech Research Group 18

Vendor considerations for the mid-market and high PPM maturity use-case scenario 2. 1 Product Evaluation Features Usability Like all software, usability can help drive adoption of the solution. Affordability Budget is an important factor for mid-market organizations. Architecture More portfolio governance may involve working with more non-PPM applications that could be beneficially integrated and may result in more storage requirements given greater data input. Vendor Evaluation Features Viability Compared to other factors, we believe viability is relatively less important for the mid-market. Focus Significant weight is placed on this category to sort out solutions designed and marketed primarily for large enterprises from those designed and marketed primarily to mid-market enterprises. Reach While support is still important, we have ranked it lower here to shift priority to architecture concerns. Sales Partnerships and channels are not a significant factor since vendors focused on the mid-market handle sales, implementation, and support directly. Info-Tech Research Group 19

Vendor performance for the mid-market and high PPM maturity use-case scenario 2. 1 Best Overall Value Award Mid-Market: High PPM Maturity Sciforma: This solution scored high in almost all aspects. On the whole, it offered the strongest balance of all of the features combined. Leaders in Mid-Market: High PPM Maturity • Team. Dynamix • Project Insight Players in the Mid-Market and High PPM Maturity Scenario • • Clarizen Eclipse PPM Innotas Liquid. Planner Info-Tech Research Group 20

Use-Case Performance Vendor performance for the mid-market and high PPM maturity use-case scenario 2. 1 Vendors Info-Tech Research Group 21

Value Index for the mid-market and high PPM maturity scenario What is a Value Score? The Value Score indexes each vendor’s product offering and business strength relative to its price point. It does not indicate vendor ranking. 2. 1 On a relative basis, Sciforma maintained the highest Info-Tech Value Score. TM of the vendor group for this use-case scenario. Vendors were indexed against Sciforma’s performance to provide a complete, relative view of their product offerings. Vendors that score high offer more bang-for-the-buck (e. g. features, usability, stability) than the average vendor, while the inverse is true for those that score lower. Price-conscious enterprises may wish to give the Value Score more consideration than those who are more focused on specific vendor/product attributes. *Vendors who scored 0 declined to provide pricing and publicly available pricing could not be found. For an explanation of how Price is determined, see Information Presentation – Price Evaluation in the Appendix. For an explanation of how the Info-Tech Value Index is calculated, see Information Presentation – Value Index in the Appendix. Info-Tech Research Group 22

USE CASE 3 Large Enterprise Info-Tech Research Group 23

2. 1 Drill Down: Large Enterprise Large enterprise project portfolios call for comprehensive PPM solutions. • Choose a solution that allows you to align a large number of diverse projects with strategic goals that span the entire organization. • Select the best tool for reconciling complex project demands across large, diverse, and cross-functional resource pools. • Enable the PMO to produce perpetually accurate and up-to-date reports and dashboards that can be distributed and customized to a wide variety of decision makers (individuals or committees). Indicators of a strong alignment with the large enterprise use-case: ü The organization’s IT department has more than 1, 000 people in it. ü Individual business units have IT resources dedicated to them. ü The organization is implementing or maintaining enterprise-grade applications. ü A “portfolio hierarchy” exists: the project portfolio is large and complex enough to be divided into sub-portfolios. ü Project updates, reports, and dashboards for major projects will have to be distributed to many people and to multiple committees (e. g. steering committees, budget committees, compliance and risk committees, etc. ). ü Internal project resources are located in multiple locations. ü Large projects that require external project resources are not uncommon. Info-Tech Research Group 24

2. 1 PPM maturity and large enterprise PPM solutions PPM maturity does not determine the kind of PPM solution that is appropriate, but it will be a major factor in determining how you configure and implement your solution. • Given the amount of resources required to configure and implement an enterprise grade PPM solution, large organizations will want to select a solution that they can grow into (instead of replace). • Many large enterprises struggle to achieve and maintain widespread and consistent adoption of a PPM solution across the organization regardless of their level of PPM maturity. Maturity Level 5: Optimized Maturity Level 4: Managed Maturity Level 3: Defined While PPM maturity is not a major factor during the selection stage for large enterprises, it is something that will be important for implementation. Large organizations will want to determine their PPM maturity level in order to understand how they will need to configure and roll out their solution. Maturity Level 2: Repeatable Maturity Level 1: Ad Hoc Establish your PPM maturity level using Info-Tech’s PPM Solution Use-Case Fit Assessment Tool. For a more detailed discussion about PPM Maturity see our publication Establish an Effective PMO for IT (especially slides 41 to 76). Info-Tech Research Group 25

2. 1 Feature weightings for the large enterprise use-case scenario Feature Weightings Core Features What-If Planning Agile Support Workflow Management A distinguishing feature of a large enterprise project portfolio is the sheer volume and complexity of projects and project resources. The ability to save and compare proposed resource allocation scenarios is a valuable feature for portfolio managers tasked with prioritizing, coordinating, and allocating resources across a large project portfolio. This feature is especially valuable if portfolio managers must present alternatives to decision makers (whether individuals or committees). Larger-enterprise project portfolios will be diverse. A tool that can accommodate and aggregate projects governed by various project management frameworks is an asset. Large enterprise portfolios have a large number of project requests, all of which need to be approved or denied in a timely fashion. The ability to submit, manage, and be notified of work requests, approvals, and assignments is especially important here. Mobile Support Manage Reactive Work Zero-Adoption Capability Workflow Management 10% 5% 10% 15% 20% 30% Agile Support 10% End-User Portal What-if Planning Additional Features Zero-Adoption Capability Mobile Support Manage Reactive Work End-User Portal Info-Tech Research Group 26

Vendor considerations for the large enterprise use-case scenario 2. 1 Product Evaluation Features Usability Ease of use will help large organizations train numerous (and constantly changing) users and admins and drive long-term adoption of the solution. Affordability While all organizations are budget conscious, large enterprises expect to pay enterprise prices for enterprise-grade solutions. Architecture Integration with existing enterprise applications is key. The quantity and frequency of data input makes architecture options important. Vendor Evaluation Features Viability Given the investment, vendor viability is key. Focus Significant weight is placed on this category to sort out solutions designed and marketed primarily for large enterprises from those designed and marketed primarily to mid-market enterprises. Reach While global reach can be an asset for large enterprises (especially multi-national corporations) vendor support does not rank high in this use case. Sales Partnerships and channeling are more important here than in the mid-market, but are not core considerations. Info-Tech Research Group 27

Vendor performance for the large enterprise use-case scenario Large Enterprise 2. 1 Best Overall Value Award Sciforma Planisware: This solution combined best-in-class what-if planning with strong agile support and strong workflow planning. Its mobile support also contributed to its top rank in this use case. Leaders in Large Enterprise • Sciforma • Clarizen Players in the Large Enterprise scenario • • Microsoft Project Server 2013 PLANTA Team. Dynamix Workfront Info-Tech Research Group 28

Use-Case Performance Vendor performance for the large enterprise use-case scenario 2. 1 Vendors Info-Tech Research Group 29

2. 1 Value Index for the large enterprise scenario What is a Value Score? The Value Score indexes each vendor’s product offering and business strength relative to its price point. It does not indicate vendor ranking. On a relative basis, Sciforma maintained the highest Info-Tech Value Score. TM of the vendor group for this use-case scenario. Vendors were indexed against Sciforma’s performance to provide a complete, relative view of their product offerings. Vendors that score high offer more bang-for-the-buck (e. g. features, usability, stability) than the average vendor, while the inverse is true for those that score lower. Price-conscious enterprises may wish to give the Value Score more consideration than those who are more focused on specific vendor/product attributes. *Vendors who scored 0 declined to provide pricing and publicly available pricing could not be found. For an explanation of how Price is determined, see Information Presentation – Price Evaluation in the Appendix. For an explanation of how the Info-Tech Value Index is calculated, see Information Presentation – Value Index in the Appendix. Info-Tech Research Group 30

VENDOR LANDSCAPE Vendor Profiles and Scoring Info-Tech Research Group 31

Use the information in the PPM solution Vendor Landscape analysis to streamline your own vendor analysis process 2. 1 This section of the Vendor Landscape includes the profiles and scoring for each vendor against the evaluation framework previously outlined. Vendor Profiles • Include an overview for each company. • Identify the strengths and weaknesses of the product and vendor. • Identify the three-year TCO of the vendor’s solution (based on a ten-tiered model). Vendor Scoring Use the Harvey Ball scoring of vendor and product considerations to assess alignment with your own requirements. Review the use-case scenarios relevant to your organization’s Use. Case-Fit Assessment results to identify a vendor’s fit to your organization's PPM solution needs. (See the following slide for further clarification on the use-case assessment scoring process. ) Review the stoplight scoring of advanced features to identify the functional capabilities of vendors. Info-Tech Research Group 32

2. 1 Clarizen Product Clarizen OVERVIEW Employees 200+ Headquarters San Mateo, CA Website clarizen. com Clarizen’s collaborative work management solution, brings together cross-company project management, configurable workflow automation, in-context collaboration and a tailored, role-based experience, all built on an enterprise-grade platform. Founded 2006 Presence Privately Held STRENGTHS 3 year TCO for this solution falls into pricing tier 7, between $100, 000 and $250, 000 $1 Pricing provided by vendor. $1 M+ • Clarizen is a genuinely global vendor. Its product is designed to work with multiple languages and currencies. It also has a strong global presence with regional headquarters in four countries. • Balances client needs by providing both a variety of useful out-ofthe-box reports to ease implementation yet still allowing for a broad range of client customization and configurability to match to an organization’s unique business processes and workflows. • Full-featured collaboration interface includes: timelines, discussion threads, hashtags, and mentions. Internal and external users can link their unstructured conversations to actual work items and take action directly from real-time discussions. CHALLENGES • Limited capacity to carry out what-if planning. Info-Tech Research Group 33

2. 1 Clarizen Info-Tech Recommends Vendor Product Usability Afford. Arch. Overall Viability Focus Reach Sales 3 4 3 4 4 4 Mid-Market Moderate Maturity Features Value Index Scenario Performance Use Overall Mid-Market High Maturity Player 4 4 3 out of 7 5 5 4 out of 7 Manage Zero-Adoption Agile Support Reactive Work Capability 4 Large Enterprise 4 4 Leader End-User Portal What-If Planning Workflow Management Mobile Support 4 4 An appropriate PPM solution for a variety of sizes and levels of maturity. Organizations that are looking for a vendor with a strong focus on collaboration may be particularly interested in this solution. Info-Tech Research Group 34

2. 1 Upland Software: Eclipse PPM Product Eclipse PPM OVERVIEW Employees 362 Headquarters Austin, TX Website eclipseppm. com Founded 2002 (Upland purchased the product from Solution Q in 2014) Upland Software offers a comprehensive range of management products including three PPM products (each PPM product offering different levels of complexity, scalability, and configurability). Organizations looking to integrate professional services software and financial software with project portfolio may want to consider Upland. Presence Nasdaq: UPLD 3 year TCO for this solution falls into pricing tier 5, between $25, 000 and $50, 000 STRENGTHS • Eclipse offers a well-organized, attractive, and user-friendly experience. • Eclipse is designed to be an out-of-box ready PPM tool requiring minimal customization and faster implementation. This can be an advantage for organizations that need to implement a PPM solution quickly and do not require much customization to integrate existing PPM processes and workflows into the solution. CHALLENGES $1 Pricing derived provided by vendor. $1 M+ • The solution does not currently provide any functionality tailored to project managers and workers using Agile methodologies. • While non-licenced project stakeholders can submit requests through the tool, they cannot directly access reports or dashboards in the solution. Info-Tech Research Group 35

2. 1 Upland Software: Eclipse PPM Info-Tech Recommends Vendor Product Afford. Arch. Overall Viability Focus Reach Sales 4 4 4 2 2 4 2 3 0 Mid-Market, Moderate Maturity Mid-Market, High Maturity Leader Player 3 6 3 out of 7 2 2 out of 7 Features Value Index Use Usability Scenario Performance Overall Manage Zero-Adoption Agile Support Reactive Work Capability 4 4 4 We recommend this solution for midmarket organizations with moderate PPM maturity who are looking to get a PPM solution implanted in a shorter timeframe. Its out-of-box capability will be an asset for organizations in this situation. End-User Portal What-If Planning Workflow Management Mobile Support 4 4 Info-Tech Research Group 36

2. 1 Innotas Product Innotas PPM OVERVIEW Employees N/A Headquarters San Francisco, CA Website innotas. com Founded 2000 Innotas provides software solutions for two kinds of portfolio management: application portfolio management (APM) and project portfolio management (PPM). It also offers a supplemental product for Agile Portfolio Management. Customers looking to manage various and diverse portfolios in their IT departments will want to consider Innotas’ PPM solution. Presence Privately Held STRENGTHS 3 year TCO for this solution falls into pricing tier 7, between $100, 000 and $250, 000 • Report and dashboard production is user friendly. The ability to automatically refresh and resend reports removes some of the manual work usually required to maintain accurate portfolio visibility for core stakeholders. • Innotas offers a “predictive portfolio analysis (PPA)” feature that is designed to help optimize portfolios by accurately predicting what you can get done with the resources available. PPA is designed to automatically generate scenarios based on variables that are critical to your organization thereby reducing manual analysis. CHALLENGES $1 Pricing derived from public information. $1 M+ • Until recently, Innotas’ mobile interface has offered limited functionality and user experiences (relative to its competitors). Innotas recently announced an update to its mobile features (Sep. 2015) so customers will want to ensure that previous mobile shortcomings have actually been addressed on the most recent releases. Info-Tech Research Group 37

2. 1 Innotas Info-Tech Recommends Vendor Product Afford. Arch. Overall Viability Focus Reach Sales 3 4 2 3 3 2 2 Use Usability Mid-Market High Maturity Scenario Performance Overall Player 7 Value Index Features Organizations that require a high level of functionality on mobile devices should assess this solution carefully. The vendor has indicated they will be releasing improvements in the mobile functionality, so organizations should be sure to include mobile functionality in any RFP and demo. out of 7 6 out of 7 Manage Zero-Adoption Agile Support Reactive Work Capability 4 4 4 Organizations in the mid-market category will want to consider Innotas. However, some of the advanced features, such as Innotas’ predictive portfolio analysis may require a higher level of PPM maturity to ensure the degree of sustained user adoption required to maintain these functions. End-User Portal What-If Planning Workflow Management Mobile Support 4 4 Info-Tech Research Group 38

2. 1 Liquid. Planner Product Liquid. Planner OVERVIEW Employees 70 Headquarters Seattle, WA Website liquidplanner. com Liquid. Planner was designed to capture the highly fluid nature of project portfolio management. It predicts a range of likely completion dates and its scheduling engine automatically calculates dates and adjusts them in real-time across the portfolio as actuals are inputted. Founded 2007 STRENGTHS Presence Privately Held • Project co-ordination and resource management are all based upon estimated ranges (best to worst case) instead of artificially specific and static dates/times. This provides portfolio managers and key stakeholders a more accurate (i. e. less artificially precise) view when making decisions about the future. • The tool offers user-friendly timing features to help workers enter up-to-date and accurate project actuals. 3 year TCO for this solution falls into pricing tier 5, between $25, 000 and $50, 000 $1 Pricing provided by vendor. $1 M+ CHALLENGES • Liquid. Planner’s ability to distribute changes across the portfolio instantaneously using predictive analytics is one of its most important distinguishing features but could also offer issues for organizations that are unable to ensure long-term adoption of widespread task-level input across the portfolio. • Its ability to shift allocations across the portfolio may require managers to ensure that its automated moves match the organization’s actual (and shifting) priorities. It also requires all project workers and managers to keep all actuals up to date and accurate to preserve portfolio-level functionality. Info-Tech Research Group 39

2. 1 Liquid. Planner Info-Tech Recommends Vendor Product Afford. Arch. Overall Viability Focus Reach Sales 4 3 2 0 Mid-Market Moderate Maturity Mid-Market High Maturity Player 5 5 5 out of 7 3 3 out of 7 Mid-market organizations that are serious about task-level project planning and time tracking should consider Liquid. Planner. Full functionality in Liquid. Planner may require prioritizing your projects. Organizations that are struggling with project prioritization during intake or that struggle to re-prioritize as projects are in progress may want to be cautious about some of its more advanced resource management features. Features Value Index Use Usability Scenario Performance Overall Info-Tech Research Group 40

2. 1 Microsoft: Project Server 2013 Product Project Server OVERVIEW Employees 117, 000 Headquarters Redmond, WA Website https: //products. office. com/enus/project/enterprise-projectserver Founded Microsoft: 1975 Project Server: 2000 Presence NASDAQ: MSFT A project management software powerhouse since the introduction of MS Project in the early 1980 s and Project Server in the early 2000 s. Project Server 2013 is an on-site PPM solution that builds upon an organization’s existing enterprise-grade software (e. g. Share. Point and Exchange) and directly integrates with its widely popular project management software (MS Project) to offer comprehensive portfolio-level functionality in a single program. (Microsoft’s Saa. S offering – Project Online – was not considered here: Project Server was assessed as a self-standing PPM product. ) STRENGTHS 3 year TCO for this solution falls into pricing tier 7, between $100, 000 and $250, 000 $1 Pricing derived from public information. $1 M+ • Access to Microsoft’s far-reaching support and partnering network. • Microsoft’s strong market presence means Project Server 2013’s interface will be familiar and intuitive for many project managers and project workers. CHALLENGES • Full functionality requires enterprise versions of Share. Point and Exchange and it also requires subscriptions to MS Project Professional. The cost of acquiring, implementing, integrating, and maintaining these various prerequisites should be factored into decision making by organizations that do not yet have them. • Microsoft has only recently moved towards Saa. S-based project and portfolio management software. This option currently seems to be marketed as a separate product – Project Online. Info-Tech Research Group 41

2. 1 Microsoft: Project Server 2013 Info-Tech Recommends Vendor Product Afford. Arch. Overall Viability Focus Reach Sales 2 2 2 3 4 4 3 Mid-Market Moderate Maturity Large Enterprise Player 6 4 out of 7 6 6 out of 7 Features Value Index Use Usability Scenario Performance Overall Manage Zero-Adoption Agile Support Reactive Work Capability 4 4 4 While some mid-market organizations may want to consider Project Server 2013 for their PPM solution, it is our opinion that this solution will work best for large enterprise organizations that have already invested in the other enterprise-grade Microsoft products and services that are needed to get full functionality out of this tool. Given Microsoft’s established presence in the market and its strong presence in heavily regulated market verticals, organizations with strict compliance requirements (e. g. health, education, or pharmaceutical) may want to include this solution on their list of prospects. End-User Portal What-If Planning Workflow Management Mobile Support 4 4 Info-Tech Research Group 42

2. 1 Planisware Product Planisware OVERVIEW Employees 180 Headquarters Châtillon, France Website planisware. com Planisware is a global organization based out of the E. U. that offers a PPM solution appropriate for the global marketplace. Planisware’s experience working with its clients engaged in top-level R&D shows through in the solution’s design and functionality. Founded 1995 Presence Privately Held STRENGTHS 3 year TCO for this solution falls into pricing tier 7, between $100, 000 and $250, 000 • The vendor offers strong multi-lingual support and its product is designed to work in multiple languages and currencies. This will be an advantage for organizations operating in a multi-national context. • Planisware offered one of the strongest capabilities for carrying out what-if scenarios. The ability to save and compare scenarios sideby-side will be a welcome feature for portfolio managers charged with allocating and prioritizing a large pool of resources across large numbers of projects. CHALLENGES $1 $1 M+ • Planisware is a good option for organizations with higher PPM maturity; however, organizations with lower PPM maturity will need to ensure that Planisware is implemented incrementally in order to avoid being overwhelmed. Pricing provided by vendor. Info-Tech Research Group 43

2. 1 Planisware Info-Tech Recommends Vendor Product Arch. Overall Viability Focus Reach Sales 3 3 3 4 2 4 3 Use Afford. Large Enterprise Scenario Performance Usability Champion Value Index Overall 5 Planisware should be seriously considered by large organizations looking for an enterprise-grade solution. 1 1 out of 7 Features out of 7 Info-Tech Research Group 44

2. 1 PLANTA Projektmanagement-Systeme Gmb. H: PLANTA Portfolio Product PLANTA Portfolio OVERVIEW Employees 57 Headquarters Karlsruhe, Germany Website planta. de PLANTA produces an enterprise-grade PPM solution. It is based in the E. U. and is used in a wide range of industries (e. g. IT, manufacturing, financial, and R&D) worldwide. Founded 1980 STRENGTHS Presence Privately Held • PLANTA Portfolio offers a variety of dashboards and reports to represent resource allocations, project status, etc. • PLANTA offers two kinds of licenses: in addition to the typical user -based licence, it offers a planning object licence where businesses pay based on the number of tasks in the solution. This may offer an attractive alternative to organizations that have a large number of project workers but a more limited number of projects in their portfolio. This option could also prove beneficial for organizations that face significant gaps between projects (if there are no planning objects to enter, there is no fee to pay). 3 year TCO for this solution falls into pricing tier 7, between $100, 000 and $250, 000 CHALLENGES $1 $1 M+ Pricing provided by vendor and public information. • This solution currently offers only minimal mobile functionality, but PLANTA advises this is in development and improved mobile support should be available in the first half of 2016. • Complex resource planning may be made more difficult than with other solutions due to the lack of the ability to save or compare allocation scenarios within the tool itself. Info-Tech Research Group 45

2. 1 PLANTA Projektmanagement-Systeme Gmb. H: PLANTA Portfolio Vendor Product Info-Tech Recommends Afford. Arch. Overall Viability Focus Reach Sales 3 3 2 4 3 2 2 Use Usability Large Enterprise Scenario Performance Overall Player Planta’s unique object-based licencing (where licences are assigned to projects/tasks instead of people/roles) may be an interesting option for organizations that predominantly work on large projects (instead of a multitude of smaller ones) or for organizations that have significant downtime between major projects. 7 out of 7 Value Index Features The majority of PLANTA’s customers opt for an on-premises deployment. Organizations looking for this option should include Planta in their considerations. 3 out of 7 Manage Zero-Adoption Agile Support Reactive Work Capability 4 4 4 End-User Portal What-If Planning Workflow Management Mobile Support 4 4 Info-Tech Research Group 46

2. 1 Metafuse: Project Insight Product Project Insight OVERVIEW Employees N/A Headquarters Irvine, CA Website projectinsight. net Project Insight claims to be one of the first multi-tenant PPM solutions aimed at mid-market organizations, and it continues to provide a strong offering to this market. Founded Metafuse: 1996 Project Insight: 2001 Presence Privately Held STRENGTHS 3 year TCO for this solution falls into pricing tier 7, between $100, 000 and $250, 000 • While Project Insight offers fine-grained task planning and reporting, a team may also use high-level project metadata for portfolio-level reporting, even if they choose not to fully utilize these project management functionalities. • Project Insight offers strong mobile support. Mobile access to the whole solution provides users the same functionality they would have on a desktop via any device they choose (phones or tablets). • Project Insight is customizable without programming, coding or complex configuration, making it very flexible and adaptable to a variety of business processes. CHALLENGES $1 Pricing provided by vendor. $1 M+ • Currently, non-project tasks (i. e. tasks not directly associated with an existing project in Project Insight) that are entered into the to-do list are not automatically represented in resource utilization. This could result in over-assigning work to project resources unless time blocked off for non-project work is perpetually adjusted to match actual non-project commitments. Info-Tech Research Group 47

2. 1 Metafuse: Project Insight Info-Tech Recommends Vendor Product Afford. Arch. Overall Viability Focus Reach Sales 3 3 3 4 3 3 3 2 3 Mid-Market Moderate Maturity Mid-Market High Maturity Player Leader 7 3 out of 7 7 7 out of 7 Project Insight appears to be particularly suited to organizations in the mid-market and high PPM maturity category. Organizations that are looking for strong performance in mobile functionality should strongly consider Project Insight. Its performance in this category was best in class. We received strong positive feedback about its mobile performance from an organization that had the majority of its technicians using the tool in the field on mobile devices. Features Value Index Use Usability Scenario Performance Overall Info-Tech Research Group 48

2. 1 Sciforma Product Sciforma OVERVIEW Employees 150 Headquarters Los Gatos, CA Website sciforma. com Founded 1982 Sciforma is an experienced and globally focused PPM solution provider and offers a solution that can help portfolio managers allocate resources and align projects with strategic goals across the enterprise. Its clean and modern user interface pairs well with its functionality developed over decades of experience. It also offers trans-national support and functionality. Presence Privately Held STRENGTHS • Offers strong support for Agile project management methodologies. It also offers the ability to integrate various project management methodologies across the portfolio into a comprehensive overview. • Offers strong mobile support. • Read-only access is always free, making it possible for key project or portfolio stakeholders to directly access reports and dashboards. 3 year TCO for this solution falls into pricing tier 6, between $50, 000 and $100, 000 CHALLENGES $1 Pricing provided by vendor. $1 M+ • Sciforma recently (2015) transitioned from a Java version to an HTML 5 version. The transition will surely be welcomed by existing and potential customers. We have not heard of any issues with this major release, but (as with any major version release) customers should consider the possibility of a series of updates and fixes during the implementation stage as the new version matures and the vendor responds to clients’ needs and opinions. Info-Tech Research Group 49

2. 1 Sciforma Info-Tech Recommends Vendor Product Usability Afford. Arch. Overall Viability Focus Reach Sales 4 4 4 3 4 4 Mid-Market Moderate Maturity Mid-Market High Maturity Large Enterprise Leader Champion Leader 2 1 2 2 out of 7 1 out of 7 2 out of 7 1 1 1 out of 7 Sciforma scored well in every Info-Tech use case and it also scored well in the value index for each use case. It is our opinion that Sciforma offers one of the most balanced PPM solutions. This solution may not be appropriate for all organizations, but it should be considered seriously by organizations regardless of their size or maturity. Features Value Index Scenario Performance Use Overall Info-Tech Research Group 50

2. 1 Team. Dynamix Product Team. Dynamix OVERVIEW Employees N/A Headquarters Columbus, OH Website teamdynamix. com Founded 2001 Team. Dynamix provides a tool that fully integrates project portfolio management and service management functionality. Team. Dynamix has established a strong specialization in the higher-education market, but organizations (especially those that are non-profit or educational) may find many of their PPM needs are addressed by this solution. Presence Privately Held STRENGTHS 3 year TCO for this solution falls into pricing tier 7, between $100, 000 and $250, 000 • The integration of service management functionality and PPM functionality helps project workers and managers to represent both project (planned pro-active) and non-project (ad hoc reactive) work in the tool. This fosters more complete and accurate analytics, project capacity awareness and resource planning across the portfolio. • Clients of Team. Dynamix have provided some of the most positive feedback regarding customer support and their overall relations with the vendor. CHALLENGES $1 Pricing provided by vendor. $1 M+ • Team. Dynamix has a strong market presence in higher education and they have developed a strong reputation and support community in that field. While they have clients in other industries (professional associations, non-profits, etc. ), Team. Dynamix’s solution and support have not yet been widely implemented in other spheres. Info-Tech Research Group 51

2. 1 Team. Dynamix Info-Tech Recommends Vendor Product Afford. Arch. Overall Viability Focus Reach Sales 4 4 2 3 3 2 0 Mid-Market Moderate Maturity Mid-Market High Maturity Champion Leader 1 2 1 out of 7 5 2 out of 7 4 4 2 out of 7 Features Value Index Use Usability Scenario Performance Overall Large Enterprise Player Team. Dynamix was a strong performer in both mid-market use cases. The extent of its combination of project portfolio management and service management was not matched by any other vendor in this field. This vendor has a strong presence and good reputation in the highereducation market and so organizations associated with higher-ed should probably take a serious look at Team. Dynamix. Organizations in other markets (especially those involved in public service or not-for-profit sectors) may want to follow-up with this vendor. Info-Tech Research Group 52

2. 1 Workfront Product Workfront (previously At. Task) OVERVIEW Employees 660 Headquarters Lehi, UT Website workfront. com Founded 2001: as At. Task 2015: Rebranded to Workfront As a vendor, Workfront focuses on helping project workers and managers work more effectively in order to bring about a more ordered workplace and work–life balance. Workfront changed its name in 2015 to reflect the addition of enterprise portfolio-level functionality that had been added to its task-level project management functionality over the past decade. Presence Privately Held 3 year TCO for this solution falls into pricing tier 7, between $100, 000 and $250, 000 STRENGTHS • Workfront provides reports and dashboards that are user friendly for project or portfolio stakeholders who may not be (and may not have to be) familiar with the finer details of project portfolio management. • Workfront offers strong Agile project management methodology support (sprints, burndowns, etc. ) and also allows PMOs to produce portfolio-level reports that encompass projects carried out in various methodologies (e. g. Agile and Waterfall). CHALLENGES $1 Pricing provided by vendor. $1 M+ • While Workfront offered some of the most impressive end-user reporting and dashboards, it does not offer direct in-solution access to such users unless the organization purchases end-user portal access at an additional cost. Indirect access is possible through file exporting (e. g. PDFs) or posting reports to public URLs. Info-Tech Research Group 53

2. 1 Workfront Info-Tech Recommends Vendor Product Afford. Arch. Overall Viability Focus Reach Sales 3 3 3 3 4 3 Use Usability Large Enterprise Scenario Performance Overall Player 6 out of 7 Value Index Features Workfront should be considered by large enterprises that have teams using Agile project management methodologies and/or that need to track time spent on project work. 7 out of 7 Manage Zero-Adoption Agile Support Reactive Work Capability 4 4 4 End-User Portal What-If Planning Workflow Management Mobile Support 4 4 Info-Tech Research Group 54

Phase 2, Step 2: Shortlist vendors in the PPM solution market 2. 1 Analyze vendors in the PPM solution market 2. 2 Shortlist vendors in the PPM solution market 2. 3 Select your PPM solution This step will walk you through the following activities: This step involves the following participants: • Review Info-Tech’s three use cases and identify which use case most accurately describes your organization’s situation • Portfolio manager/PMO director • Procurement officer • PPM solution procurement project team members Outcomes of this step • Shortlist of vendors for your PPM solution Info-Tech Research Group 55

Evaluate alternative PPM vendors not included in Info. Tech’s Vendor Landscape 2. 2 Info-Tech only evaluated a portion of vendors in the PPM market. In order for a vendor to be included in this Landscape, the company must meet three baseline criteria: • Our clients must be talking about the solution • Our analysts must believe the solution will play well within the evaluation • Vendor must meet table stake criteria Below is a list of some notable vendors in the space that did not meet all of Info-Tech’s inclusion requirements. Additional Vendors in the PPM Solution Market Info-Tech Research Group 56

Phase 2, Step 3: Select your PPM solution 2. 1 Analyze vendors in the PPM solution market 2. 2 2. 3 Shortlist vendors in the PPM solution market Select your PPM solution This step will walk you through the following activities: This step involves the following participants: • Prioritizing your solution requirements based on your business, architecture, and performance needs • Preparing a PPM solution request for proposal (RFP) • Preparing a Vendor RFP Response Template • Evaluating and scoring vendors based on RFP responses • Conduct vendor demonstrations • Conduct client reference interviews • Scoring and evaluating vendors • Selecting your PPM solution based on the previous activities listed here • • PPM solution project manager PPM solution project team members Portfolio manager(s) Procurement officer (or another appropriate representative from the financial department) • At least one project manager representative • At least one project worker representative Outcomes of this step • • PPM solution request for proposal (RFP) Template RFP response to send to vendors Vendor scorecard PPM solution selection Info-Tech Research Group 57

2. 3 Determine your PPM solution procurement strategy Critical Points and Checks in Your Procurement Info-Tech’s 15 -Step Procurement Process • Follow your own organization’s procurement procedures to ensure that you adhere to your organization’s policies. • Based on your organization’s policies, identify if you are going to conduct a private or public RFP process. o If your RFP will contain sensitive information, use a private RFP process that is directed to specific vendors in order to protect the proprietary practices of your business. Use Info-Tech's Procurement Process to ensure that your PPM solution selection is properly planned and executed. INFO-TECH OPPORTUNITY 1. Initiate procurement. 2. Select procurement manager. 3. Prepare for procurement; check that prerequisites are met. 4. Select appropriate procurement vehicle. 5. Assemble procurement teams. 6. Create procurement project plan. 7. Identify and notify vendors about procurement. 8. Configure procurement process. 9. Gather requirements. 10. Prioritize requirements. 11. Build the procurement documentation package. 12. Issue the procurement. 13. Evaluate proposals. 14. Recommend a vendor. 15. Present to management. If your organization lacks a clear procurement process, refer to Info-Tech's Optimize IT Procurement research to help construct a formal process for selecting application technology. Much of your procurement process should already be outlined from your charter and initial project structuring. In this stage of the process, focus on the successful completion of steps 7 -15. If you are still not sure of a vendor’s capabilities, we recommend sending a Request for Information (RFI) before proceeding with an RFP. Info-Tech Research Group 58

2. 3 Prioritize your solution requirements based on your business, architecture, and performance needs Review Shortlist Evaluate 1. Identify Your Requirements Select INPUT Brief Completed in Phase 1, Step 4 Use the findings being collected in the Requirements Workbook and related materials to define clear requirements around your organization’s desired PPM solution. 2. Prioritize Your Requirements • Identify the significance of each requirement for your solution evaluation. • Info-Tech recommends identifying features and requirements as mandatory, important, or optional. • Control the number of mandatory requirements you document. Too many mandatory requirements could create an unrealistic framework for evaluating solutions. 3. Create a Requirements Package • Consolidate your identified requirements into one list, removing redundancies and conflicts. • Categorize the requirements based on their priority and nature. • Requirements Workbook and requirements gathering findings OUTPUT • Full documentation of requirements for the RFP and solution evaluation process Use this requirements package as you evaluate vendors and create your RFP for shortlisted vendors. No solution will meet 100% of your requirements. Control the number of mandatory requirements you place in your procurement process to ensure that vendors that are the best fit for your organization are not eliminated unnecessarily. Info-Tech Research Group 59

2. 3 Create an RFP to submit to vendors PPM Solution RFP Template INPUT Activity: Interpreting the Results Leverage Info-Tech’s PPM Solution RFP Template to convey your desired suite requirements to vendors and outline the proposal and procurement steps set by your organization. Build Your RFP 1 Outline the organization's procurement instructions for vendors (Sections 1, 3, and 5). 2 Input the requirements package created in Phase 1, Step 4, into your RFP (Section 4). 3 Create a scenario overview to provide vendors an opportunity to give an estimated price. Approval Process Each organization has a unique procurement process; follow your own organization’s process as you submit your RFPs to vendors. 4 Ensure compliance with your organization's standards and gain approval for submitting your RFP. Info-Tech RFP Table of Contents 1. Statement of Work 2. General Information 3. Proposal Preparation Instructions 4. Scope of Work, Specifications, and Requirements 5. Vendor Qualifications and References 6. Budget and Estimated Pricing 7. Vendor Certification • Requirements package (5. 1) • Organization’s procurement procedures OUTPUT • RFP Materials • Whiteboard and markers Participants • Project manager • Core project team Info-Tech Research Group 60

Standardize the potential responses from vendors and streamline your evaluation with a response template 2. 3 PPM Solution Vendor Response Template Adjust the scope and content of the PPM Solution Vendor Response Template to fit your PPM solution procurement process and vendor requirements. Section Why is this section important? About the Vendor This is where the vendor will describe itself and prove its organizational viability. Understanding of the Challenge Methodology Demonstrates understanding of the problem is the first step in being able to provide a solution. Proposed Solution Describes how the vendor will address the challenge. This is a very important section as it will articulate what you will receive from the vendor as a solution. Provides an overview of the project management methodology, phases of the project, what will be delivered, and when. Provides evidence of prior experience with delivering similar projects for similar clients. Project Management, Plan, and Timeline Vendor Qualifications References Value Added Services Requirements Shows that there is a proven methodology to approach and solve the challenge. Provides contact information for individuals/organizations for which the vendor has worked and who can vouch for the experience and success of working with this vendor. Remember, this could lead to a long-term relationship. It’s not only about what you need now, but also what you may need in the future. Confirmation from the vendor as to which requirements it can meet and how it will meet them. Info-Tech Research Group 61

2. 3 Evaluate the RFPs you receive with a clear scoring process PPM Solution Evaluation and Scoring Tool Review Shortlist Select How to use this tool Evaluate Brief Activity 5. 3 Build a fair evaluation framework that evaluates vendor solutions against a set criteria rather than relative comparisons. Instructions + A 1. Have members of the PPM solution evaluation team review the RFP responses given by vendors. 2. Input vendor solution information into the PPM Solution Evaluation & RFP Scoring Tool. • Review the feature list and select where each feature is mandatory, desirable, or not applicable. • Select if each feature has been met by the vendor RFP response. • Enter the costing information provided by each vendor. • Determine the relative importance of the features, architecture, and support. Tool Output • Costing • Overall score • Evaluation notes and comments 3. Analyze the vendors against your identified evaluation framework. 4. Identify vendors with whom you wish to arrange vendor briefings. 5. Contact vendors and arrange briefings. Info-Tech Research Group 62

2. 3 Vendor product demonstration PPM Solution Vendor Demo Script Demo Invite vendors to come onsite to demonstrate the product and to answer questions. Use a demo script to help identify how a vendor’s solution will fit your organization’s particular business capability needs. Make sure the solution will work for your business Provide the vendor with some usage patterns for the PPM solution tool in preparation for the vendor demo. Challenge the vendors in the demo by asking them to do the following (for example): • Carry out two what-if scenarios for resource allocations. Provide the following information in your PPM Solution Vendor Demo Script: • Visually represent how much of the portfolio’s resources are utilized for project vs. non-project work. • Usage for different groups • The key reports the PPM solution needs to produce • The key tasks the PPM solution needs to support (e. g. resource allocation) • Produce an automatic alert for any project that has not been updated in the last 7 days. INFO-TECH ACTIVITY 1. Create a demo script that will be sent to vendors that outlines PPM solution usage patterns from your organization. 2. Construct the demo script with your PPM solution evaluation team, providing both prompts for the vendor to display the capabilities and some sample data for the vendor to model. • Produce a project capacity forecast for the next two quarters that takes into account all projects carried out in both agile methodologies and traditional methodologies. Review INPUT OUTPUT • Requirements package • Use-case results Shortlist Evaluate • Onsite demo Select Brief Info-Tech Research Group 63

Conduct client reference interviews to identify how other organizations have successfully used the vendor’s solution Reference Checking 2. 3 Request references from the vendors. Have conversations with the references to make sure the vendors deliver what they promise. Vendors are inevitably going to provide references that will give positive feedback, but don’t be afraid to dig into the interviews to understand some of the limitations related to the solution. • Even if a vendor is great for one client, that doesn’t necessarily mean it will fit for you. Ask the vendor to provide references to organizations in your own or a similar industry, or someone who has automated similar business processes or outlined similar expectations. If you are looking to include a high level of customization in your PPM solution, pay particular attention to this step and the client responses, as these will help you understand how easy a vendor is to work with. Make the most of your client reference interviews by preparing your questions in advance and following a specific script. • Use these reference calls as an opportunity to gain a more accurate understanding of the quality of the vendor’s service support and professional services. Info-Tech Insight Some references may be reluctant to say anything negative about their vendor. An indirect approach may be useful in such cases. For instance, instead of asking “What didn’t you like about [PPM solution]? ” try asking “How would you have prepared differently for implementing [PPM solution] knowing what you know now? ” or “What issues do you advise we look out for when using [PPM solution]? ”. Have vendors supply client references to confirm the value of their implemented solution. Info-Tech Research Group 64

Use vendor RFPs and demos to select the PPM solution that best fits your organization’s needs 2. 3 PPM Solution Vendor Evaluation & Scoring Tool: Tab 5, Overall Score Don’t just choose the vendor who gave the best presentation; select the vendor who meets your functional requirements and organizational needs. Category Review Shortlist Evaluate Select Brief Weight Vendor 1 Vendor 2 Vendor 3 Vendor 4 PPM Features 60% 75% 80% 90% Architecture 25% 55% 60% 90% Support 15% 10% 70% 60% 95% Total Score 100% 60% 74% 80% 91% Use your objective evaluation to select a vendor to recommend to management for procurement. Don’t automatically decide to go with the highest score; validate that the vendor is someone you can envision working with for the long term. • Select a vendor based not only on their evaluation performance, but also on your belief that you could form a lasting and supportive relationship with them. • Integration needs are dynamic, not static. Find a PPM solution tool and vendor that have strong capabilities and will fit with the application and integration plans of the business. • In many cases, you will require professional services together with your PPM solution purchase to make sure you have some guidance in the initial development and your own staff are trained properly. Following the identification of your selected solution, submit your recommendation to the organization’s management or evaluation team for final approval. Info-Tech Research Group 65

PHASE 3 Plan the PPM Solution Implementation Select and Implement a PPM Solution Info-Tech Research Group, Inc. is a global leader in providing IT research and advice. Info-Tech’s products and services combine actionable insight and relevant advice with ready-to-use tools and templates that cover the full spectrum of IT concerns. © 1997 -2016 Info-Tech Research Group Inc. Info-Tech Research Group 66

Phase 3: Plan the PPM solution implementation 3. 1 Implementation considerations Estimated Timeline: 6 -12 months 3. 2 Evaluate your project metrics Info-Tech Insight Implementations are about establishing sustainable longterm adoption of the PPM solution and not merely about deploying software. Organizational change management is necessary for any PPM solution implementation. See Adopt Organizational Change Management Best Practices to Maximize Project Benefits for additional guidance. Major Milestones Reached Key Activities Completed • Plan for implementation and expected go-live date • PPM Solution Implementation Plan • Governance Plan • Change Control Methods Outcome from this phase Plans for implementing the selected PPM solution tool Info-Tech Research Group 67

Phase 3, Step 1: Implementation considerations 3. 1 Implementation considerations 3. 2 Evaluate your project metrics This step will walk you through the following activities: This step involves the following participants: • Identify important tasks that should be completed or avoided in order to ensure a successful implementation for each of the following implementation stages: plan, design, build, test, and roll-out. • Since the implementation of most commercial PPM solutions will require utilizing professional services (usually arranged through the vendor), this section will not provide detailed instructions on the implementation process. The aim is to identify some useful tips that will help you plan out your implementation with your implementation team. • • • Project managers and workers Portfolio managers Applications director, applications managers, or SMEs IT architects Business directors and managers Business analysts Relevant business leaders (e. g. CIO, CEO) PPM solution project team Vendor implementation team Outcomes of this step • Insights into some core stages of the implementation process Info-Tech Research Group 68

Identify the resourcing plan for your PPM solution implementation 3. 1 There are three main ways to implement your PPM platform 1 Implement in-house by own staff 2 Implement using a combination of your own staff and professional services from an implementer 3 Implement using professional services Instructions 1. Select your implementation team and identify the number of external consultants/amount of support that will be required to implement your selected PPM solution. Consider your in-house skills, timeline considerations, integration environment complexity, and cost constraints as you make your resourcing plan. 2. Document the roles and expectations related to each step of the implementation. (Use the work breakdown structure and your selected vendor’s materials as a reference to guide your planning). Info-Tech Insight Always pay attention to implementation trade-offs: Adopting a DIY implementation attitude could draw out your implementation timeline and increase the likelihood of errors. Carefully consider your integration environment to determine the resourcing and external support required. Opting for professional services may result in a shorter implementation period and fewer errors, but may also deny your IT staff the opportunity to develop the skills necessary to maintain and configure the solution in the future. What is your priority? Faster? Cheaper? Better Quality? Build your resourcing team to address the technical competencies required to implement a PPM Solution. Consider the following: Prioritize members from your core selection team. They will have a strong insight into the tool and its envisioned position in the organization. General Roles • Integration specialists • Solution/enterprise architects • QA engineer • IT service management team External Roles • Vendor’s implementation team/professional services • Systems integrator Info-Tech Research Group 69

3. 1 Create your PPM solution implementation plan Participants 1 2 3 Plan the implementation Design the framework Build the framework 4 Test the solution 5 Roll out the solution Use Info-Tech’s 5 -step implementation process to guide your own planning. For incremental or iterative roll-outs, it will be necessary to return to step 2 and move down the list again for each module of the solution. Project Manager Project Sponsor Enterprise Architect(s) Solution Architect(s) Integration Specialist Organizational Change Management and/or Training Specialist Outcomes from this section • A project description document • A work breakdown structure INFO-TECH Templates Use a PPM Implementation Work Breakdown Structure to guide the configuration and implementation of your solution. Info-Tech Research Group 70

3. 1 Outline a PPM solution implementation plan PPM Implementation Work Breakdown Structure Activity As a group, discuss each of the following implementation plan phases for your PPM solution implementation project: plan, design, build, test, and roll-out. Consider the major challenges and opportunities that will present themselves at each stage and develop an initial strategy to proactively address these as the implementation plan is more fully worked out in the weeks and months ahead. Instructions 1 Brainstorm opportunities and challenges for each of the five stages. Small groups can discuss each stage successively together as a group. Larger groups can break up into 5 sub-groups where each sub-group brainstorms just one challenge and then shares their thoughts with the group. 2 Identify 1 or 2 of the most important challenges and 1 or 2 of the most important opportunities for each phase. 3 Record all of the shortlisted challenges and opportunities and ensure that this list is constantly reviewed as each stage is planned out in more detail in the months to follow. INFO-TECH Templates While outlining your implementation plan, you may find it useful to review and start filling out the implementation portions of the PPM Implementation Work Breakdown Structure to guide the configuration and implementation of your solution. After the selection process is over, you will want to revisit and refine the WBS sections relevant to the implementation stage of the project. INPUT • The vendor’s RFP response for the solution chosen • The requirements workbook OUTPUT • High-level outline for the implementation of your selected PPM solution Materials • PPM Requirements Gathering Tool • PPM Implementation Work Breakdown Structure Participants • PPM solution project team • Portfolio manager • Software implementation SME Info-Tech Research Group 71

3. 1 Plan the implementation Allocate resource time to the implementation 1 2 Plan the implementation All PPM solutions require accurate data to drive their functionality and to produce reports/dashboards. IT resources (project workers and project managers) will have to dedicate part of their workday to inputting and maintaining this data. This time will have to be included along with other non-project work in all IT resources’ schedules and subtracted from their project work capacity. If workers are not allocated time to maintain the data and if regular data maintenance is not enforced, the implementation will fail. 3 Establish clear expectations and processes for maintaining accurate and up-to-date portfolio data in the solution 4 Establishing sustainable adoption of a PPM solution at the projectworker level is a significant hurdle that many organizations cannot overcome. 5 Be sure that your implementation directly addresses the following issues: • Mandatory data input cadence (recommend daily) • Implement a user-friendly method for basic project data input at the project worker and project manager levels (utilize integrations where possible). • Implement some form of automatic alert indicating when data has gone stale (i. e. a project has not been updated, cancelled, or closed in x days). Info-Tech Research Group 72

3. 1 Plan the implementation Set short-term goals that are TANGIBLE 1 2 3 4 Plan the implementation Set short-term goals that aretangible and easy to measure. Focus on a small set of reports that are needed by keydecision makers to get things done. The production and delivery of a credible report is a clear and important milestone. Include the regular production of PPM reports in your PPM solution implementation plan. This will provide you short-term goals that are: • Measurable: The soundness of the report and level of stakeholder satisfaction can be assessed in a fairly direct way. • Valuable: This approach will generate value very early in the implementation, maintaining sponsor support and driving buy-in and adoption down the line. – Jim Morris, Consultant 5 Info-Tech Research Group 73

3. 1 Design the framework Establish formal processes and workflows FIRST, and then design how these will be represented in the solution 1 2 3 4 5 Design the framework • PPM solutions will not solve issues surrounding PPM processes or PPM workflows. Processes and workflows must be documented and mapped out before the solution is implemented. • Plan time for discussion about PPM processes and workflows and be prepared to resolve differences of opinion. Integrating the organization’s processes and workflows into the solution may force formality and standardization where there was not any before. PPM solutions bring visibility to the portfolio and sometimes what people initially see will be uncomfortable. • Good PPM solutions allow you to adjust the solution to your PPM process; while you may need to refine some processes during implementation, you should not have to adjust PPM processes to fit the PPM solution. Be firm with the vendor about processes and workflows: the vendor should be able to configure/customize their PPM solution to match your workflows. Info-Tech Research Group 74

3. 1 Design the framework Key processes, workflows, and criteria to design into the PPM solution framework 1 2 Design the framework 3 4 5 • List of project categories (ideally, the categories should be mutually exclusive and collectively exhaustive): e. g. transformative projects (any project size that introduces significant organizational/commercial changes), conservative projects (large projects that maintain business -as-usual, and discretionary projects (small projects that maintain business-as-usual). • Project Intake and Prioritization Scorecard • Complete project workflow encompassing all stakeholders • Key performance metrics and necessary reports/dashboards. • RACI chart for the project portfolio and project management User experience • Consult with each executive user and a representative for every other kind of user to find out: o What functionality they need o What information they need o What data they need to input • Consider the kinds of interfaces already present in the organization’s application environment and consider what each group is most comfortable with. Info-Tech Research Group 75

3. 1 Build the framework Consider resourcing trade-offs carefully • Consider the costs and benefits of each option for building your PPM solution framework. 1 Option 2 3 4 5 Benefit Cost Build the framework using your own IT staff • Your own IT staff will have direct and detailed knowledge of the framework • Your own staff will have the knowledge and skills to adapt the framework in the future • Your IT staff will likely have much less experience building PPM solution frameworks than a vendorapproved professional consultant • More inefficiencies and errors Implement by professional services • Experienced and efficient professionals will be more likely to carry out desired configurations and customizations • Expensive • Future dependence on the vendor or professional consultant for adaptation of the framework Implement using a combination of your own staff and professional services from an implementer • Your own IT staff will gain some direct knowledge and skill • A professional implementer may be able to avoid or proactively correct errors with the build • Work and authority relationships will have to be established between internal IT staff and external consultants • Partial dependence on external resources (advice and warranty work) Build the framework Info-Tech Research Group 76

3. 1 Test the solution User experience 1 2 Include qualitative assessments throughout the roll-out. Qualitative assessments: • Provide clear indicators of (dis)satisfaction for each stakeholder group at each stage • Foster stakeholder engagement at all stages • Help ensure that implementation adjustments are primarily based on how well the tool serves each stakeholder group and not merely on meeting technical requirements Not everything that can be counted counts, and not everything that counts can be counted. (Source: William Bruce Cameron, Informal Sociology, 1963, p. 13. ) 3 Remember to test successful and sustainable adoption and not merely technical efficiency. 4 5 Test the solution • Incremental roll-outs of the solution will help you more effectively isolate the sources of non-adoption or incomplete data o The more users and the more data, the harder it is to identify what is missing when things go wrong o Do not proceed to the next stage of the roll-out until satisfactory test results are attained at each stage • For each failed test ask: o What prevented the data from being processed (correctly)? - Lack of time to input data? - Lack of understanding about how to input data? - Fear of visibility by IT staff? - Faulty tool? Info-Tech Research Group 77

3. 1 Roll out the solution 1 2 3 4 5 For us, one of the challenges was cultural change because[the PPM tool] exposes a lot. You know when a project manager is not updating; that is toughbecause now you can see everything. When things are in spreadsheets, project managers haveto share it with you but now it's availablein real time. The challenge is getting over that cultural change. – Senior VP, Enablement, Software Industry Roll out the solution Anticipate and plan for organizational change • Implementing a commercial PPM solution for the first time will introduce unprecedented visibility into your organization’s project work. • Ensure that all stakeholders understand the visibility the tool will provide and that everyone has a clear expectation of what they will be expected to do in the solution. Build adoption on adoption • Roll out the PPM solution to those stakeholders you have reason to believe are most likely to adopt it long term. • Every organizational change will encounter “resisters. ” Avoid rolling out the PPM solution to these users as long as possible. • Once the PPM solution reaches a threshold tipping point of adoption, there will be added sources of motivation for those resisting adoption to comply: o Others are benefitting from the PPM solution (potentially giving them an edge on recognition, remuneration, or promotion) o Social pressure (or peer pressure) will make continued resistance or skeptical less attractive or influential • For more detailed advice on managing organizational change see Info-Tech’s Adopt Organizational Change Management Best Practices to Maximize Project Benefits blueprint. Info-Tech Research Group 78

3. 1 Roll out the solution 1 2 3 4 From an Info-Tech interview: Q: Why start implementing the tool at the portfolio level first and then move to implementing it at the project management level after? A: Our real priority was to establish and support the leadership governance teams. It was important for us to get organized and determine what projects were in the queue and what we were actively working on. We wanted to provide our leadership team some exposure into our portfolio and help them prioritize the work so we could ensure we were working on the right things to support the business. We already had tools and templates that we could use [for project management] so we thought the biggest priority was to build the portfolio. The next phase would then be to determine how we could better use this tool for managing our projects. – Portfolio manager at a major metropolitan hospital 5 Roll out the solution Build momentum by starting at the top and then rolling down the solution from there. • While data must be collected at the lowest levels of the PPM hierarchy (project workers and project managers), PPM solutions offer the most value to those at the highest levels: the CIO, business leaders, decision makers, etc. • Ensure that the first PPM solution outputs (reports and dashboards) are aimed at the highest levels. Provide key decision makers the basic information they need to make decisions on and prioritize projects. • If your company distinguishes between transformative IT projects and regular IT projects, start with the transformative ones. These projects will likely be long, complex, and involve many resources. They will also likely have high executive-level visibility. • Starting at the top also ensures that key project sponsors see the value of the PPM solution early on. Info-Tech recommends an incremental roll-out of the PPM solution. It may be useful to discuss options for an incremental roll-out with your vendor. There may be ways for the admin to disable certain functions, interfaces, or permissions during the initial phases of the implementation. Info-Tech Research Group 79

Phase 3, Step 2: Evaluate your project metrics 3. 1 Implementation considerations 3. 2 Evaluate your project metrics This step will walk you through the following activities: This step involves the following participants: • Compare key PPM solution metrics after the implementation with the baseline metrics established at the start of the project • Evaluate stakeholder satisfaction through stakeholder interviews • Isolate remaining PPM issues that could not be solved by the PPM solution in order to inform or refine a broader PPM strategy • PPM solution project manager • Project Portfolio Manager • Key project portfolio stakeholders o Each relevant executive stakeholder o Representative(s) of each relevant non-executive stakeholder group Outcomes of this step • Measured value for the PPM solution project • A focused and refined list of PPM opportunities for the organization’s post-implementation PPM strategy Info-Tech Research Group 80

Track metrics throughout the project to keep stakeholders informed As the project nears completion: 1. Capacity awareness should improve. 2. Supply management should become more efficient. 3. Decisions related to project demand management (intake and prioritization) should be more informed, objective, and defensible. 4. Key project portfolio stakeholders (especially business leaders, project managers, and project workers) should be more satisfied with the level of service and support provided by the PMO. Capacity Awareness Supply Management Demand Management Project Success Info-Tech Research Group 81

3. 2 Avoid narrow definitions of success While being on time, on budget, and in scope are all important components of success for any project, they are not the only factors. Ensure that your success criteria include both quantitative and qualitative benefits and quantitative and qualitative metrics. Jerry Manas (a successful PPM author) argues that “the primary reason most projects fail is because the focus of the execution and measurement of projects is too narrow and inwardly directed. ” According to Manas, a project’s success can be evaluated according to: q Client satisfaction q Delivered value q Nobody is harmed (physically or mentally) (Source: Jerry Manas. Service Oriented Project Management (SOPM): Whitepaper. The Marengo Group, LLC. Pages 7 -8. http: //marengogroup. com/wpcontent/uploads/2009/10/Article. Manas. SOPM. pdf ) Info-Tech Research Group 82

3. 2 Interview key stakeholders • Don’t give into the temptation to assess the implementation project solely on considerations of cost, time, and scope. • Schedule a 15 -25 minute interview with each executive stakeholder and with a representative of each of the other stakeholder groups and focus on the following questions: o What percent of the tasks that the PMO asks you to do on a regular basis does the tool make easier to do? o What is the biggest obstacle to using the PPM solution? o What had you hoped the solution would be able to do but you have not seen it do yet? • Sometimes problems will be identified that no PPM solution could ever solve (on its own). • Do not disregard these. These issues may identify gaps in PPM processes or PM training that need to be addressed in a larger PMO strategy. Info-Tech Research Group 83

3. 2 Create a post-project evaluation plan for your PPM solution implementation project PPM Solution Success Metrics Workbook Activity The first part of the exercise will establish an evaluation schedule. For the second part of the exercise you will identify which individuals or roles are responsible for evaluating the PPM solution implementation project at each of its major phases and/or milestones. INPUT • PPM Solution Success Metrics Workbook OUTPUT Instructions 1 Based on the PPM solution project metrics that were selected in Phase 1, establish concrete milestones for evaluation. Choose deadlines for the production or reports and dates to re-evaluate PPM performance once the solution is implemented. 2 Identify which people (or which roles) will be responsible and accountable for the evaluation process. It will also outline which stakeholders need to be consulted and informed at each stage of the evaluation. (Construct a RACI chart to complete and record the results of this exercise. ) • Plan for evaluating the PPM solution that outlines major milestones, responsibilities, and accountabilities Materials • PPM Solution Requirements Gathering Tool • PPM Solution Work Breakdown Structure Participants • PPM solution project team • Portfolio manager • Software implementation SME Info-Tech Research Group 84

Appendices Info-Tech Research Group 85

Vendor participation Vendors who briefed with Info-Tech Research Group • • • Clarizen Innotas Liquid. Planner Metafuse (Project Insight) Planisware • • • PLANTA Sciforma Team. Dynamix Upland (Eclipse. PPM) Workfront Info-Tech Research Group 86

Additional resource support Our research benefitted greatly from the participation of 14 individuals who volunteered to be client references for vendors featured in our vendor landscape. They provided useful information about the realworld performance of the PPM solutions discussed here and the level of service and support provided by vendors. They also provided valuable insights into the general process of selecting and implementing any PPM solution. The candor and honesty of these interviewees was impressive and deeply appreciated. Understandably, these client references asked that we maintain their anonymity in order to allow them to speak freely about how they use their PPM solutions and about the performance of those solutions. The pool of client references was diverse: • Some examples of the roles interviewed – one CEO, portfolio managers, project managers, and training specialists. • We interviewed users in the following industries: o Aviation o Communications infrastructure o Education o Financial o Healthcare o Manufacturing o Non-profit o Research and design Info-Tech Research Group 87

Bibliography Ajjan, Haya. “Understanding differences between adopters and nonadopters of information technology project portfolio management. ” International Journal of Information Technology & Decision Making 12. 6 (2013): 1151 – 1174. Manas, Jerry. Service Oriented Project Management (SOPM): Whitepaper. The Marengo Group, LLC. Pages 7 -8. http: //marengogroup. com/wp-content/uploads/2009/10/Article. Manas. SOPM. pdf Merkhofer, Lee. “White Paper: Project Portfolio Management Tools: Which Approach is Best? , Part 7: The Best Approach. ” Priority System: Lee Merkhofer Consulting. 2014. p. 36. http: //prioritysystem. com/PDF/toolscomplete. pdf PMI’s Pulse of the Profession: In-depth Report: Portfolio Management. Project Management Institute. 2013. p. 3. https: //www. pmi. org/~/media/PDF/Research/PMI-Portfolio-Management. ashx Info-Tech Research Group 88

Vendor Landscape Analysis Appendices Info-Tech Research Group 89

Vendor Landscape Methodology: Overview Info-Tech’s Vendor Landscapes are research materials that review a particular IT market space, evaluating the strengths and abilities of both the products available in that space, as well as the vendors of those products. These materials are created by a team of dedicated analysts operating under the direction of a senior subject matter expert over a period of several weeks. Evaluations weigh selected vendors and their products (collectively “solutions”) on the following eight criteria to determine overall standing: • Features: The presence of advanced and market-differentiating capabilities. • User Interface: The intuitiveness, power, and integrated nature of administrative consoles and client software components. • Affordability: The three-year total cost of ownership of the solution; flexibility of the pricing and discounting structure. • Architecture: The degree of integration with the vendor’s other tools, flexibility of deployment, and breadth of platform applicability. • Viability: The stability of the company as measured by its history in the market, the size of its client base, and its percentage of growth. • Focus: The commitment to both the market space, as well as to the various sized clients (small, mid-sized, and enterprise clients). • Reach: The ability of the vendor to support its products on a global scale. • Sales: The structure of the sales process and the measure of the size of the vendor’s channel and industry partners. Evaluated solutions within scenarios are visually represented by a Pathway to Success, based off a linear graph using above scoring methods: • Use-case scenarios are decided upon based on analyst expertise and experience with Info-Tech clients. • Use-case scenarios are defined through feature requirements, predetermined by analyst expertise. • Placement within scenario rankings consists of features being evaluated against the other scoring criteria. Info-Tech’s Vendor Landscapes are researched and produced according to a strictly adhered to process that includes the following steps: • Vendor/product selection • Information gathering • Vendor/product scoring • Information presentation • Fact checking • Publication This document outlines how each of these steps is conducted. Info-Tech Research Group 90

Vendor Landscape Methodology: Vendor/Product Selection & Information Gathering Info-Tech works closely with its client base to solicit guidance in terms of understanding the vendors with whom clients wish to work and the products that they wish evaluated; this demand pool forms the basis of the vendor selection process for Vendor Landscapes. Balancing this demand, Info-Tech also relies upon the deep subject matter expertise and market awareness of its Senior Analysts to ensure that appropriate solutions are included in the evaluation. As an aspect of that expertise and awareness, Info-Tech’s analysts may, at their discretion, determine the specific capabilities that are required of the products under evaluation, and include in the Vendor Landscape only those solutions that meet all specified requirements. Information on vendors and products is gathered in a number of ways via a number of channels. Initially, a request package is submitted to vendors to solicit information on a broad range of topics. The request package includes: • A detailed survey. • A pricing scenario (see Vendor Landscape Methodology: Price Evaluation and Pricing Scenario, below). • A request for reference clients. • A request for a briefing and, where applicable, guided product demonstration. These request packages are distributed approximately eight weeks prior to the initiation of the actual research project to allow vendors ample time to consolidate the required information and schedule appropriate resources. During the course of the research project, briefings and demonstrations are scheduled (generally for one hour each session, though more time is scheduled as required) to allow the analyst team to discuss the information provided in the survey, validate vendor claims, and gain direct exposure to the evaluated products. Additionally, an end-user survey is circulated to Info-Tech’s client base and vendor-supplied reference accounts are interviewed to solicit their feedback on their experiences with the evaluated solutions and with the vendors of those solutions. These materials are supplemented by a thorough review of all product briefs, technical manuals, and publicly available marketing materials about the product, as well as about the vendor itself. Refusal by a vendor to supply completed surveys or submit to participation in briefings and demonstrations does not eliminate a vendor from inclusion in the evaluation. Where analyst and client input has determined that a vendor belongs in a particular evaluation, it will be evaluated as best as possible based on publicly available materials only. As these materials are not as comprehensive as a survey, briefing, and demonstration, the possibility exists that the evaluation may not be as thorough or accurate. Since Info-Tech includes vendors regardless of vendor participation, it is always in the vendor’s best interest to participate fully. All information is recorded and catalogued, as required, to facilitate scoring and for future reference. Info-Tech Research Group 91

Vendor Landscape Methodology: Scoring Once all information has been gathered and evaluated for all vendors and products, the analyst team moves to scoring. All scoring is performed at the same time so as to ensure as much consistency as possible. Each criterion is scored on a ten-point scale, though the manner of scoring for criteria differs slightly: • Features is scored via Cumulative Scoring. • Affordability is scored via Scalar Scoring. • All other criteria are scored via Base 5 Scoring. Cumulative Scoring is on a four-point scale. Zero points are awarded to features that are deemed absent or unsatisfactory, one point is assigned to features that are partially present, two points are assigned to features that require an extra purchase in the vendor’s product portfolio or through a third party, three points are assigned to features that are fully present and native to the solution, and four points are assigned to the best-of-breed native feature. The assigned points are summed and normalized to a value out of ten. For example, if a particular Vendor Landscape evaluates eight specific features in the Feature Criteria, the summed score out of eight for each evaluated product would be multiplied by 1. 25 to yield a value out of ten to represent in a Harvey Ball format. In Scalar Scoring, a score of ten is assigned to the lowest cost solution, and a score of one is assigned to the highest cost solution. All other solutions are assigned a mathematically-determined score based on their proximity to / distance from these two endpoints. For example, in an evaluation of three solutions, where the middle cost solution is closer to the low end of the pricing scale it will receive a higher score, and where it is closer to the high end of the pricing scale it will receive a lower score; depending on proximity to the high or low price it is entirely possible that it could receive either ten points (if it is very close to the lowest price) or one point (if it is very close to the highest price). Where pricing cannot be determined (vendor does not supply price and public sources do not exist), a score of 0 is automatically assigned. In Base 5 scoring a number of sub-criteria are specified for each criterion (for example, Longevity, Market Presence, and Financials are subcriteria of the Viability criterion), and each one is scored on the following scale: 5 - The product/vendor is exemplary in this area (nothing could be done to improve the status). 4 - The product/vendor is good in this area (small changes could be made that would move things to the next level). 3 - The product/vendor is adequate in this area (small changes would make it good, more significant changes required to be exemplary). 2 - The product/vendor is poor in this area (this is a notable weakness and significant work is required). 1 - The product/vendor fails in this area (this is a glaring oversight and a serious impediment to adoption). The assigned points are summed and normalized to a value out of ten as explained in Cumulative Scoring above. Scores out of ten, known as Raw scores, are transposed as is into Info-Tech’s Vendor Landscape Shortlist Tool, which automatically determines Vendor Landscape positioning (see Vendor Landscape Methodology: Information Presentation – Vendor Landscape, below), Criteria Score (see Vendor Landscape Methodology: Information Presentation – Criteria Score, below), and Value Index (see Vendor Landscape Methodology: Information Presentation – Value Index, below). Info-Tech Research Group 92

Vendor Landscape Methodology: Information Presentation – Criteria Scores (Harvey Balls) Info-Tech’s criteria scores are visual representations of the absolute score assigned to each individual criterion, as well as of the calculated overall vendor and product scores. The visual representation used is Harvey Balls are calculated as follows: 1. Raw scores are transposed into the Info-Tech Vendor Landscape Shortlist Tool (for information on how raw scores are determined, see Vendor Landscape Methodology: Scoring, above). 2. Each individual criterion raw score is multiplied by a pre-assigned weighting factor for the Vendor Landscape in question. Weighting factors are determined prior to the evaluation process, based on the expertise of the Senior or Lead Research Analyst, to eliminate any possibility of bias. Weighting factors are expressed as a percentage, such that the sum of the weighting factors for the vendor criteria (Viability, Strategy, Reach, Channel) is 100%, and the sum of the product criteria (Features, Usability, Affordability, Architecture) is 100%. 3. A sum-product of the weighted vendor criteria scores and of the weighted product criteria scores is calculated to yield an overall vendor score and an overall product score. 4. Both overall vendor score / overall product score, as well as individual criterion raw scores are converted from a scale of one to ten to Harvey Ball scores on a scale of zero to four, where exceptional performance results in a score of four and poor performance results in a score of zero. 5. Harvey Ball scores are converted to Harvey Balls as follows: • A score of four becomes a full Harvey Ball. • A score of three becomes a three-quarter full Harvey Ball. • A score of two becomes a half-full Harvey Ball. • A score of one becomes a one-quarter full Harvey Ball. • A score of zero becomes an empty Harvey Ball. 6. Harvey Balls are plotted by solution in a chart where rows represent individual solutions and columns represent overall vendor / overall product, as well as individual criteria. Solutions are ordered in the chart alphabetically by vendor name. Harvey Balls Overall Harvey Balls represent weighted aggregates. Product Vendor Overall Usability Afford. Arch. Overall Viability Focus Reach Channel 2 2 0 2 3 3 4 3 2 Criteria Harvey Balls represent individual raw scores. Info-Tech Research Group 93

Vendor Landscape Methodology: Use-Case Scoring Within each Vendor Landscape a set of use-case scenarios are created by the analysts by considering the different outcomes and purposes related to the technology being evaluated. To generate the custom use-case vendor performances, the feature and Harvey Ball scoring performed in the Vendor Landscapes are set with custom weighting configurations. Use-Case Vendor Performance Calculations Each product has a vendor multiplier calculated based on its weighted performance, considering the different criteria scored in the Harvey Ball evaluations. To calculate each vendor’s performance, the advanced feature scores are multiplied against the weighting for the feature in the use-case scenario’s configuration. The weighted advanced feature score is then multiplied against the vendor multiplier. The sum of each vendor’s total weighted advanced features is calculated. This sum is used to identify the vendor’s qualification and relative rank within the use case. Vendors who qualified for each use-case scenario are ranked from first to last in a weighted bar graph based on the features considered. Each use case’s feature weightings and vendor/product weighting configurations are displayed within the body of slide deck. Info-Tech Research Group 94

Vendor Landscape Methodology: Information Presentation – Feature Ranks (Stoplights) Advanced features are determined by analyst expertise, leveraging information gained from conversations with clients. Advanced features chosen as part of the evaluation are representative of what Info-Tech clients have indicated are of importance to their vendor solution. Advanced features are evaluated through a series of partial marks, dedicated to whether the solution performs all aspects of the Info-Tech definition of the feature and whether the feature is provided within the solution. Analysts hold the right to determine individual, unique scoring criteria for each evaluation. If a feature does not meet the criteria, Info-Tech holds the right to score the feature accordingly. Stoplight Legend Use cases use features as a baseline of the inclusion and scoring criteria. Feature is best in class 4 Feature is fully present and native to the solution 4 Feature is available at an additional cost 4 Feature is partially present 4 Feature is not available or unsatisfactory Info-Tech Research Group 95

Vendor Landscape Methodology: Information Presentation – Value Index Info-Tech’s Value Index is an indexed ranking of solution value per dollar as determined by the raw scores assigned to each criteria (for information on how raw scores are determined, see Vendor Landscape Methodology: Scoring, above). Value scores are calculated as follows: 1. The TCO Affordability criterion is removed from the Affordability score and the remaining product score criteria (Features, Usability, Architecture). Affordability scoring is adjusted with the TCO weighting distributed in proportion to the use case’s weighting for Affordability. Weighting is adjusted as to retain the same weightings relative to one another, while still summing to 100%. 2. An adjusted multiplier is determined for each vendor using the recalculated Affordability scoring. 3. The multiplier vendor score and vendor’s weighted feature score (based on the use-case scenario’s weightings), are summed. This sum is multiplied by the TCO raw score to yield an interim Value Score for each solution. 4. All interim Value Scores are then indexed to the highest performing solution by dividing each interim Value Score by the highest interim Value Score. This results in a Value Score of 100 for the top solution and an indexed Value Score relative to the 100 for each alternate solution. 5. Solutions are plotted according to Value Score, with the highest score plotted first, and all remaining scores plotted in descending numerical order. Where pricing is not provided by the vendor and public sources of information cannot be found, an Affordability raw score of zero is assigned. Since multiplication by zero results in a product of zero, those solutions for which pricing cannot be determined receive a Value Score of zero. Since Info-Tech assigns a score of zero where pricing is not available, it is always in the vendor’s best interest to provide accurate and up-to-date pricing. In the event that insufficient pricing is available to accurately calculate a Value Index, Info-Tech will omit it from the Vendor Landscape. Value Index Vendors are arranged in order of Value Score. The Value Score each solution achieved is displayed, and so is the average score. Average Score: 52 100 80 40 30 10 A B C D E Those solutions that are ranked as Champions are differentiated for point of reference. Info-Tech Research Group 96

Vendor Landscape Methodology: Information Presentation – Price Evaluation Info-Tech’s Price Evaluation is a tiered representation of the three-year Total Cost of Ownership (TCO) of a proposed solution. Info-Tech uses this method of communicating pricing information to provide high-level budgetary guidance to its end-user clients while respecting the privacy of the vendors with whom it works. The solution TCO is calculated and then represented as belonging to one of ten pricing tiers. Pricing tiers are as follows: 1. Between $1 and $2, 500 2. Between $2, 500 and $5, 000 3. Between $5, 000 and $10, 000 4. Between $10, 000 and $25, 000 5. Between $25, 000 and $50, 000 6. Between $50, 000 and $100, 000 7. Between $100, 000 and $250, 000 8. Between $250, 000 and $500, 000 9. Between $500, 000 and $1, 000 10. Greater than $1, 000 Where pricing is not provided, Info-Tech makes use of publicly available sources of information to determine a price. As these sources are not official price lists, the possibility exists that they may be inaccurate or outdated, and so the source of the pricing information is provided. Since Info-Tech publishes pricing information regardless of vendor participation, it is always in the vendor’s best interest to supply accurate and up to date information. Info-Tech’s Price Evaluations are based on pre-defined pricing scenarios (see Product Pricing Scenario, below) to ensure a comparison that is as close as possible between evaluated solutions. Pricing scenarios describe a sample business and solicit guidance as to the appropriate product/service mix required to deliver the specified functionality, the list price for those tools/services, as well as three full years of maintenance and support. Price Evaluation Call-out bubble indicates within which price tier the three-year TCO for the solution falls, provides the brackets of that price tier, and links to the graphical representation. 3 year TCO for this solution falls into pricing tier 6, between $50, 000 and $100, 000 $1 $1 M+ Pricing solicited from public sources Scale along the bottom indicates that the graphic as a whole represents a price scale with a range of $1 to $1 M+, while the notation indicates whether the pricing was supplied by the vendor or derived from public sources. Info-Tech Research Group 97

Vendor Landscape Methodology: Information Presentation – Vendor Awards At the conclusion of all analyses, Info-Tech presents awards to exceptional solutions in three distinct categories. Award presentation is discretionary; not all awards are extended subsequent to each Vendor Landscape and it is entirely possible, though unlikely, that no awards may be presented. Awards categories are as follows: • Champion Awards are presented to the top performing solution in a particular use-case scenario. As a result, only one Champion Award is given for each use case, and the entire Vendor Landscape will have the same number of Champion Awards as the number of evaluated use cases. Vendor Awards for Use-Case Performance Info-Tech’s Champion Award is presented to solutions that placed first in an use-case scenario within the Vendor Landscape. • Leader Awards are presented to top performing solutions for each use-case scenario. Depending on the use-case scenario and the number of solutions being evaluated, a variable number of leader awards will be given. This number is at the discretion of the analysts, but is generally placed at two, and given to the solutions ranking second and third respectively for the use case. • Best Overall Value Awards are presented to the solution for each use-case scenario that ranked the highest in the Info-Tech Value Index for each evaluated scenario (see Vendor Landscape Methodology: Information Presentation – Value Index, above). If insufficient pricing information is made available for the evaluated solutions, such that a Value Index cannot be calculated, no Best Overall Value Award will be presented. Only one Best Overall Value Award is available for each use-case scenario. Info-Tech Leader Award is given to solutions who placed in the top segment of a use-case scenario. Info-Tech’s Best Overall Value Award is presented to the solution within each use-case scenario with the highest Value Index score. Info-Tech Research Group 98

Vendor Landscape Methodology: Fact Check & Publication Info-Tech takes the factual accuracy of its Vendor Landscapes, and indeed of all of its published content, very seriously. To ensure the utmost accuracy in its Vendor Landscapes, we invite all vendors of evaluated solutions (whether the vendor elected to provide a survey and/or participate in a briefing or not) to participate in a process of fact check. Once the research project is complete and the materials are deemed to be in a publication ready state, excerpts of the material specific to each vendor’s solution are provided to the vendor. Info-Tech only provides material specific to the individual vendor’s solution for review encompassing the following: • All written review materials of the vendor and the vendor’s product that comprise the evaluated solution. • Info-Tech’s Criteria Scores / Harvey Balls detailing the individual and overall vendor / product scores assigned. • Info-Tech’s Feature Rank / stoplights detailing the individual feature scores of the evaluated product. • Info-Tech’s Raw Pricing for the vendor either as received from the vendor or as collected from publicly available sources. • Info-Tech’s Scenario ranking for all considered scenarios for the evaluated solution. Info-Tech does not provide the following: • Info-Tech’s Vendor Landscape placement of the evaluated solution. • Info-Tech’s Value Score for the evaluated solution. • End-user feedback gathered during the research project. • Info-Tech’s overall recommendation in regard to the evaluated solution. Info-Tech provides a one-week window for each vendor to provide written feedback. Feedback must be corroborated (be provided with supporting evidence), and where it does, feedback that addresses factual errors or omissions is adopted fully, while feedback that addresses opinions is taken under consideration. The assigned analyst team makes all appropriate edits and supplies an edited copy of the materials to the vendor within one week for final review. Should a vendor still have concerns or objections at that time, they are invited to a conversation, initially via email, but as required and deemed appropriate by Info-Tech, subsequently via telephone, to ensure common understanding of the concerns. Where concerns relate to ongoing factual errors or omissions, they are corrected under the supervision of Info-Tech’s Vendor Relations personnel. Where concerns relate to ongoing differences of opinion, they are again taken under consideration with neither explicit not implicit indication of adoption. Publication of materials is scheduled to occur within the six weeks following the completion of the research project, but does not occur until the fact check process has come to conclusion, and under no circumstances are “pre-publication” copies of any materials made available to any client. Info-Tech Research Group 99

Pricing Scenario Info-Tech Research Group is providing each vendor with a common pricing scenario to enable normalized scoring of affordability, calculation of Value Index rankings, and identification of the appropriate solution pricing tier as displayed on each vendor scorecard. The pricing scenario functionality applies to at least one of the use cases. Please indicate if your pricing would be significantly different if your products were used for any of the other use cases being considered. Vendors are asked to provide list costs for PPM software licensing to address the needs of a reference organization described in the pricing scenario. Please price out the lowest possible 3 -year total cost of ownership (TCO) including list prices for software and licensing fees to meet the requirements of the following scenario. Three-year total acquisition costs will be normalized to produce the affordability raw scores and calculate Value Index ratings for each solution. (Pricing scenarios descriptions on the next slide. ) Info-Tech Research Group 100

Pricing Scenario (Cont. ) A project-driven organization is looking to implement your PPM solution. The expected solution capabilities are as follows: Please provide a price estimate for both or either scenario (enterprise and/or mid-market), depending on your target segment: Enterprise • 1, 000 primary/full users servicing a population of 10, 000 who need to have access to project and portfolio reports through an end-user intranet/portal. ◦ License + maintenance and/or Saa. S service fees for the first three years of the deal, including typical professional services/consulting costs related to the deployment and any typical engagements over the first three years. ◦ ◦ If the solution being quoted is a Saa. S solution, the cost of additional storage should be included, if applicable. Top-level support services should include all implementation support, ongoing support, and documentation. Mid-Market • 100 primary/full users servicing a population of 1, 000 who need to have access to project and portfolio reports through an end-user intranet/portal. ◦ License + maintenance and/or Saa. S service fees for the first three years of the deal, including typical professional services/consulting costs related to the deployment and any typical engagements over the first three years. ◦ ◦ If the solution being quoted is a Saa. S solution, the cost of additional storage should be included, if applicable. Top-level support services should include all implementation support, ongoing support, and documentation. Info-Tech Research Group 101