Streams monitoring enhancements Zbigniew Baranowski CERN IT Department
Streams monitoring enhancements Zbigniew Baranowski CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it
OUTLINE • Quick streams monitor review • Streams monitor update • OMS 10. 2. 0. 5 new features for streams Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 2
Streams monitor review • Overview page Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 3
Streams monitor review • Topology page Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 4
Streams monitor review • Database & instance dashboard Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 5
Streams monitor review • Database details Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 6
Streams monitor review • Streams dashboard Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 7
Streams monitor review • Streams details Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 8
Streams monitor review • Plots of all numeric metrics Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 9
Streams monitor review • Streams activity reports Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 10
Plots CSV export Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 11
Streams monitor updates • COOL sessions monitoring Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 12
Streams monitor updates • COOL sessions monitoring – – – COOL schema name Instance Number of opened and active sessions Number of processes used by sessions Memory stats • • Used Allocated Freeable Max allocated memory Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 13
Streams monitor updates • COOL sessions plots – Active vs. Total Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 14
Streams monitor updates • COOL sessions plots – Active vs. Total – Number of processes Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 15
Streams monitor updates • COOL sessions plots – Active vs. Total – Number of processes – Memory usage Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 16
Streams monitor updates • COOL sessions plots – – Active vs. Total Number of processes Memory usage Max allocated memory Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 17
Streams monitor updates • Time since last applied LCR – Monitoring of streams components state and latencies is not enough – Oracle bug: • Apply process consumes LCRs but not applies • LHCb setup was affected this week – Monitoring of last apply time was required Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it • Email notifications – When: sysdate – last apply >12 hours • New metric on monitoring pages • New plot Zbigniew Baranowski 18
Streams monitor updates Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 19
New features in OMS 10. 2. 0. 5 • Streams overview page – Streams summary (number of streams processes etc. ) – Performance plots(latencies and throughputs) Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 21
New features in OMS 10. 2. 0. 5 Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 22
New features in OMS 10. 2. 0. 5 Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 23
New features in OMS 10. 2. 0. 5 • Streams overview page – Streams summary (number of streams processes etc. ) – Performance plots(latencies and throughputs) • Path level views (only with 11 g DBs) – Latency – Throughput – Bottleneck indicators Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 24
New features in OMS 10. 2. 0. 5 Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 25
New features in OMS 10. 2. 0. 5 • Streams overview page – Streams summary (number of streams processes etc. ) – Performance plots(latencies and throughputs) • Path level views (only with 11 g DBs) – Latency – Throughput – Bottleneck indicators • Streams topology Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it – With radius of one database (unfortunately) Zbigniew Baranowski 26
New features in OMS 10. 2. 0. 5 Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 27
New features in OMS 10. 2. 0. 5 • Long active transactions Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 28
New features in OMS 10. 2. 0. 5 Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 29
New features in OMS 10. 2. 0. 5 • Long active transactions list (running more than 1 minute) – For capture and apply coordinator processes • Spilled streams transaction list for apply process Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 30
New features in OMS 10. 2. 0. 5 Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it Zbigniew Baranowski 31
OMS 10. 2. 0. 5 problems • Unable to display propagation process metrics – java. sql. SQLException: Fail to convert to internal representation Internet Services CERN IT Department CH-1211 Genève 23 Switzerland www. cern. ch/it • All capture process metrics marked as unavailable • Above problems have been fixed with patch 8445475 • Applied patch breaks apply server page • Not all plots are displayed – No data is currently available Zbigniew Baranowski 32
- Slides: 31