Tag Collector review Fabian Lambert Jrme Fulachier Jrme
Tag Collector review Fabian Lambert, Jérôme Fulachier, Jérôme Odier LPSC Grenoble, France October 3, 2020 Tag Collector Review, October 6 2013, CERN 1
Tag Collector review Review date November 6, 2013 9: 30 – 12: 30 (304 -1 -007) https: //indico. cern. ch/conference. Display. py? conf. Id=279611 Aim of the meeting : TC 3 working group Inventory main TC users use cases • Identify what need improvements in TC 2 Classify use cases • Try to design a draft of a menu structure fitting user needs Prioritization of functionalities development • Establish a road map of functions that should appear in TC 3 interface plans Developed in parallel to TC 2 interface Should be functional and testable for each functionality deployed October 3, 2020 Tag Collector Review, October 6 2013, CERN 2
Tag Collector history 2001 -2004 Tag Collector 1 PHP web application • Monolithic releases (no branches, no groups) 2004 - 2013 Tag Collector 2 Java web application using AMI framework underlying software 2004 • Container based releases, group dependency, PV lock system • PV dependencies checked against CMT req. file using CVS plugin 2008 • Leaf based releases, containers are not tagged anymore 2009 • Migration to SVN, new TC SVN plugin 2010 • Tag approval system, no more direct PV operations in releases • Bundle system 2013 Tag Collector 3 New GUI, redesign of all TC users interfaces October 3, 2020 Tag Collector Review, October 6 2013, CERN 3
Why Tag Collector 3 ? Two main reasons For TC users • Some interfaces are unused • Some interfaces (approvals…) are tricky to use • Some functionalities need to be reorganized For TC web application • AGILE methodology – Continuous delivery to users (short delivery cycles) – Decoupling graphical interface from server treatment – Be able to do test driven developments for the new interface • Benefit from modern and powerful technologies (almost standard) – Bootstrap CSS – JQuery Javascript – XHTML 5 • Improving application usability, reliability and evolutivity October 3, 2020 Tag Collector Review, October 6 2013, CERN 4
TC 2 workflow Tag Collector Web Interface Tag Collector Server User request HTTP request HTML form HTTP response User fill the form HTTP request Form Command HTML page form Command (DB operation) XML output HTML page result October 3, 2020 HTTP response Tag Collector Review, October 6 2013, CERN HTML page result To avoid Generate HTML (java) Generate XML (java) To avoid Transform XSL (java) 5
TC 3 workflow Tag Collector Web Interface User request TC app. HTML init. User actions TC app. HTML updates October 3, 2020 Tag Collector Server HTTP request HTML page TC app. HTTP response AJAX requests AJAX responses Tag Collector Review, October 6 2013, CERN Commands (DB operations) XML/JSON output Generate XML/JSON (java) 6
TC 3 interface structure Main menu bar Login Optional title bar Breadcrumb Left menu depending on the main menu context Main Application Structure draft for the new TC interface October 3, 2020 Tag Collector Review, October 6 2013, CERN 7
Test bench environment TC 3 new interface will use Javascript, HTML 5 and AJAX requests to the server Test bench environment Karma, http: //karma-runner. github. io • Test Runner for Java. Script Jasmine, http: //pivotal. github. io/jasmine • Tests driven development framework for Javascript Sinon. JS, http: //sinonjs. org • Standalone test spies, stubs and mocks for Java. Script Continuous integration server Jenkins, http: //jenkins-ci. org/ • Continuous deployment and tests (1 commit = 1 code analysis) Code quality analysis Sonar. Qube, http: //www. sonarqube. org/ • Analysis of compliance to quality rules for Javascript and Java • Technical Debt October 3, 2020 Tag Collector Review, October 6 2013, CERN 8
Tag Collector ontology Project -> Hidden Project Coordinator -> Administartor Group-> Project User Group Coordinator -> Project Coordinator Package Manager Package Developer Release Coordinator Package context Package Version approval October 3, 2020 Tag Collector Review, October 6 2013, CERN 9
Tag Collector survey Keep in mind that "difficult" is relative to experience. October 3, 2020 Tag Collector Review, October 6 2013, CERN 10
TC survey generalities How much time per day are you spending on TC ? What is your TC usage profile ? Project Coordinator 13% 12% Release Coordinator 69% Less than 10 min. 44% Between 10 min. and 30 min. More then 30 min. 44% Package Manager 94% 0% 20% 40% 60% 80% No answer 100% Time spent per day on TC have to be decreased Need GUI improvement Need a review of role workflows October 3, 2020 Tag Collector Review, October 6 2013, CERN 11
Package Managers October 3, 2020 Tag Collector Review, October 6 2013, CERN 12
Package Managers Packages per managers (all users) Packages per managers (experts) 13% 23% 31% One package 1 package between 2 and 10 packages Less than 10 packages 53% more then 10 packages 33% More than 10 packages No answer 46% 2 type of Package Managers Most have less than 10 packages Experts have often more than 10 packages • Need to simplify access to large number of packages October 3, 2020 Tag Collector Review, October 6 2013, CERN 13
Packages / Versions ? Declare a new package is ? 20% Declare a new package version (collect a tag) is ? 20% Easy Moderately easy 12% Easy 27% Moderately easy Difficult Nightmare 27% 33% No answer Difficult 7% Nightmare 7% Ask the expert (Emil ? ) to do it. Automatically get the list of available packages in SVN to avoid typos in the package name. After a few months not having declared new packages you forget what was the trick. Finding branch packages under which a new leaf package needs to go is difficult in most cases. A simple selector of branch packages like the one in the tag collection page would be most helpful. 47% No answer Updating the tag of a package that's already in a release is quite simple from the developer home page. But collecting a "new" package into let's say Atlas. Production is something that needs to be taught explicitly to all new developers, as doing it is anything but intuitive. Define a common context for package and package version ? October 3, 2020 Tag Collector Review, October 6 2013, CERN 14
Approvals Request an approval for my package version is ? 7% Getting information about my package version approval status is ? 7% 7% Easy 6% Easy Moderately easy Difficult 20% 60% Nightmare No answer It's easy for package developers but a nightmare for release coordinators. if it is not a special release (not one of them offered in developers page). 46% 47% Difficult Nightmare No answer The developer home page gives a quite good summary of the ongoing package approvals in my mind. Keep a dashboard like interface. Improve for multi package owners. October 3, 2020 Tag Collector Review, October 6 2013, CERN 15
Find information Find the release where is declared my package version is ? Find managers responsible of a package is ? 7% 13% 20% 7% Easy 33% Moderately easy Easy 13% 40% Moderately easy Difficult Nightmare No answer 33% In the end I usually end up checking this on AFS. . . Once you learn how to search for branch and leaf packages, it's not too difficult. But a better package search page would help a lot I think. Except there is no clean up process - managers listed may have left ATLAS This is difficult primarily because the "responsibles" listed in TC often aren't. The TC package managers for most packages are outdated anyway. The problem is often the discrepancy between the responsibles in TC 2, in svn and those who did the past 5 Changelog entries. Improve search ? VOMS for ATLAS membership for cleanup ? October 3, 2020 Tag Collector Review, October 6 2013, CERN 16
Release Coordinators October 3, 2020 Tag Collector Review, October 6 2013, CERN 17
Release life cycle Create a new release is ? Terminate a release is ? 9% 9% Easy Moderately easy Difficult Nightmare No answer 82% I just send an email to the release shifters list, so I haven't actually done this myself. Is done by the software build shifters To be honest I never did this myself. I was just put in charge of releases created by others. . . Same here, I always relied on release manager experts to do it for me. Release life cycle need to be clarify. Commands to hide for RC ? October 3, 2020 Tag Collector Review, October 6 2013, CERN 18
Release life cycle Do you know signification of release status ? 9% 9% No Some of them 27% I know them all 55% Release status No answer Approval modes Open Leaf locked (container based releases) Close Manager/Developer activated and Top package managers can manipulate and accept Only Top package managers can request/Coordinators Only Top package managers can manipulate and accept/Coordinators Only coordinators Only project coordinators Release status need to be simplified/clarified? October 3, 2020 Tag Collector Review, October 6 2013, CERN 19
Find release information Find my release information page is ? Find where is my release approval page is ? 9% 36% 27% 18% Easy Moderately easy Difficult Nightmare 36% No answer several clicks - why release for which I have admin rights are not listed separately? Usually needs a lot of scrolling. . . Bookmarks make it easier for me. Nightmare 27% No answer It is sometimes hard to navigate back to the "main" release page after doing modifications on a branch or leaf package. Add a customizable dashboard ? October 3, 2020 Tag Collector Review, October 6 2013, CERN 20
Approval management Manage approvals in my release is ? Manage bundles is ? 18% 27% Easy Moderately easy 45% Difficult Nightmare 36% No answer 55% Can be difficult if the same action has to be taken on multiple packages. Did manage to get confused by bundle approvals a few times. Very complicated workflow, i. e. if one has to break up bundles and reassemble them. Handling bundles is inherently tricky, but once I need to break up a bundle to modify it, things can get quite complicated. Approval and bundle interfaces need simplifications. October 3, 2020 Tag Collector Review, October 6 2013, CERN 21
Backup October 3, 2020 Tag Collector Review, October 6 2013, CERN 22
Release Coordinators (all users since 2001) Release Coordinators (experts, opened releases) 1 release less than 10 releases 33% 36% 38% between 10 and 100 releases more then 100 releases 64% Between 2 and 10 releases More than 10 releases No answer 29% October 3, 2020 Tag Collector Review, October 6 2013, CERN 23
Administrator Project Coordinators (all users) 3% 1 project 48% 49% between 2 and 10 projects more then 10 projects October 3, 2020 Tag Collector Review, October 6 2013, CERN 24
- Slides: 24