SAM Doc : Update-19
This page last changed on Jan 14, 2013 by tarragon.

Summary

Start Date 25 June 2012
Release Date 23 November 2012
Status Released
Release Manager Wojciech Lapka
Validation Steps SAM-2660 and SAM-2764

Description

This release is devoted to documentation and to the MyEGI component, improving many aspects of its visualization. Please note that it is highly recommended to first update to SAM Update-17 before upgrading to this release.

Technical details:

  • 287 tickets resolved
  • Status and Availability computation:
    • Improved availability re-computation algorithm and status computation bootstrapping
    • Log information about status of execution of MySQL events
    • Improvement of logging mechanism
  • Topology aggregation:
    • New ATP API package integrated in MyWLCG
    • VOFeed validation logs added to ATP probe
    • Bug fixes
  • Profile Management:
    • Added tagging capability and improving user interface
    • Changes to public Web API
      • Service types have been deprecated and are currently mapped to service flavors
  • MyEGI changes:
    • Major style and layout changes
    • Adding new view availability and reliability reporting
    • Trends dashboard: availability/reliability trends, Experiment usage
    • Public API documentation revised
    • Added MyEGI user and admin guides
    • Changed to Django-1.3 to improve security and functionality of several components (POEM, MyEGI, ATP)
  • Updated MySQL to non-vulnerable version (5.1.63) and improved MySQL database dump
  • Developer documentation for all components
  • Nagios configuration
    • Removed resource BDII from SAM/Nagios
    • Consume VO Nagios results in a Site Nagios instance
    • Removed probe 'org.nagios.NCGPidFile'
    • Added probe 'org.nagiosexchange.NCGLogFiles'
  • Probes integration and changes:
    • Added probe ch.cern.sam.MrsCheckSpool
    • Repackaging of perl-gridmon probe development framework
    • Integration of QCG/MAPPER probes
    • Integration of UNICORE Job and unicore6.StorageFactory
    • Fixing compatibility with EMI WN probe.
    • Fixing EMI version detection in the WN probe.
  • Fixing configuration issue with perl-Net-STOMP-Client-1.2.1
  • SAM configuration changes (glite-yaim-nagios):
    • Removed MDDB configuration

Package List

SAM-Nagios

atp-1.25.10-1.el5
glite-yaim-nagios-1.8.46-1.el5
grid-monitoring-config-gen-0.90.15-1.el5
grid-monitoring-probes-hr.srce-0.35.3-1.el5
grid-monitoring-probes-org.sam-0.5.7-1.el5
mrs-1.7.30-1.el5
mywlcg-1.4.17-6.el5
mywlcg-atp-web-1.26.1-3.el5
mywlcg-atp-api-1.25.20-1.el5
nagios-plugins-qcg-broker-2.6.1-1
nagios-plugins-qcg-ntf-2.6.0-3
nagios-plugins-qcg-comp-2.6.0-1
ncg-metric-config-1.1.6-1.el5
poem-0.9.64-1.el5
poem-sync-0.9.64-1.el5
sam-nagios-1.19.3-1.el5
sam-release-1.19.0-1.el5
sam-sync-1.0.9-1.el5
unicore-nagios-plugins-2.2.1-1.sl5

#Dependencies:
html5lib-0.10-1 #required by mywlcg.html5lib package is required for generating pdfs using python
pisa-3.0.33.2-1 #required by mywlcg.pisa package is required for generating pdfs using python
python-django-1.3.1-1.el5
django-taggit-0.9.3-2.el5
perl-Messaging-Message-1.3-1.el5
python-reportlab-2.5-3.el5

SAM-Gridmon

ace-0.2.3-2.el5
atp-1.25.10-1.el5
dax-1.0.11-1.el5
glite-yaim-nagios-1.8.46-1.el5
mrs-1.7.30-1.el5
mywlcg-1.4.17-6.el5
mywlcg-atp-web-1.26.1-3.el5
mywlcg-atp-api-1.25.20-1.el5
ncg-metric-config-1.1.6-1.el5
poem-0.9.64-1.el5
poem-sync-0.9.64-1.el5
sam-release-1.19.0-1.el5
sam-gridmon-1.19.1-2.el5

#Dependencies:
html5lib-0.10-1 #required by mywlcg.html5lib package is required for generating pdfs using python
pisa-3.0.33.2-1 #required by mywlcg.pisa package is required for generating pdfs using python
python-django-1.3.1-1.el5
django-taggit-0.9.3-2.el5
perl-Messaging-Message-1.3-1.el5
python-reportlab-2.5-3.el5

Configuration Changes

Common

  • New Yaim configuration variables:
    Component Name Description Default Mandatory Example
    all DB_PASS_R Database password (reader) No Yes my_reader_password
    all DB_PASS_W Database password (writer) No Yes my_writer_password
    all DB_USER_R Database user (reader) Yes Yes mrs
    all DB_USER_W Database user (writer) Yes Yes mrs
    MRS MRS_BOOTSTRAPPER_TIMEOUT Timeout (in seconds) for MRS bootstrapper Yes Yes 1800
    MRS MRS_LOG_LEVEL Level of debug for MRS log (possible values: DEBUG WARNING or ERROR) Yes Yes ERROR
    MyWLCG MYWLCG_ACCESS_CONTROL_ALLOW_ORIGIN Set Access-Control-Allow-Origin header Yes Yes *
  • Removed YAIM variables 
    DB_USER
DB_PASS
MDDB_SYNC_TIMEOUT

SAM-Gridmon

  • New Yaim configuration variables
    Component Name Description Default Mandatory Example
    MyWLCG MYWLCG_REPORTS To enable/disable Reports Yes Yes False
    MyWLCG MYWLCG_REPORT_AUTHENTIC_DNS List of authentic DNs that can generate official reports (Seperated by commas with no space) No No  
    MyWLCG MYWLCG_REPORT_EGI_PROFILES List of allowed profiles for EGI Reports Yes Yes ROC_CRITICAL
    MyWLCG MYWLCG_REPORT_WLCG_TIER1_HISTORY_PROFILES List of allowed profiles for WLCG Tier1 History Reports Yes Yes WLCG_CREAM_LCGCE_CRITICAL
    MywLCG MYWLCG_REPORT_WLCG_TIER1_SUMMARY_PROFILES List of allowed profiles for WLCG Tier1 Summary Reports Yes Yes WLCG_CREAM_LCGCE_CRITICAL
    MyWLCG MYWLCG_REPORT_WLCG_TIER1_VO_PROFILES List of allowed profiles for WLCG Tier1 VO Reports Yes Yes WLCG_CREAM_LCGCE_CRITICAL ALICE_CRITICAL ATLAS_CRITICAL CMS_CRITICAL LHCb_CRITICAL
    MyWLCG MYWLCG_REPORT_WLCG_TIER2_PROFILES List of allowed profiles for WLCG Tier2 Reports Yes Yes WLCG_CREAM_LCGCE_CRITICAL
    MyWLCG MYWLCG_TOPOLOGY To turn on/off Topology module in MyWLCG Yes Yes False
    MyWLCG MYWLCG_EGI_EMAILS List of emails to which EGI Reports will be send Yes Yes egi-league-results@hellasgrid.gr
    MyWLCG MYWLCG_WLCG_TIER1_EMAILS List of emails to which WLCG Tier1 Reports will be send Yes Yes worldwide-lcg-management-board@cern.ch
    MyWLCG MYWLCG_WLCG_TIER2_EMAILS List of emails to which WLCG Tier2 Reports will be send Yes Yes project-wlcg-cb@cern.ch

SAM-Nagios

  • New Yaim configuration variables:
    Component Name Description Default Mandatory Example
    grid-monitoring-probes-ch.cern.sam MRS_S_crit_recalc critical_recalc parameter of MrsCheckSpool probe Yes Yes 50000
    grid-monitoring-probes-ch.cern.sam MRS_S_crit_spool critical_spool parameter of MrsCheckSpool probe Yes Yes 10000
    grid-monitoring-probes-ch.cern.sam MRS_S_path_recalc web_service_path_recalc parameter of MrsCheckSpool probe Yes Yes myegi/sam-pi/num_records_metricdataforrecalculation
    grid-monitoring-probes-ch.cern.sam MRS_S_path_spool web_service_path_spool parameter of MrsCheckSpool probe Yes Yes myegi/sam-pi/num_records_metricdata_spool
    grid-monitoring-probes-ch.cern.sam MRS_S_warn_recalc warn_recalc parameter of MrsCheckSpool probe Yes Yes 25000
    grid-monitoring-probes-ch.cern.sam MRS_S_warn_spool warn_spool parameter of MrsCheckSpool probe Yes Yes 5000
    NCG NCG_INCLUDE_PROXY_CHECKS Switch on/off internal tests for generating proxy credential Yes No 1
    NCG NCG_SEND_TO_DASHBOARD Enables interaction with Dashboard on VO instances Yes Yes false
    NCG HOST_KEYSTORE_PASS Password used for encrypting host certificate keystore Yes Yes password
    MyWLCG MYWLCG_EGI_EMAILS List of emails to which EGI Reports will be send Yes Yes egi-league-results@hellasgrid.gr
    MyWLCG MYWLCG_WLCG_TIER1_EMAILS List of emails to which WLCG Tier1 Reports will be send Yes Yes worldwide-lcg-management-board@cern.ch
    MyWLCG MYWLCG_WLCG_TIER2_EMAILS List of emails to which WLCG Tier2 Reports will be send Yes Yes project-wlcg-cb@cern.ch

Known Issues

Please upgrade to SAM Update-17 first as the upgrade from SAM Update-15 directly to SAM Update-19 is not supported.
For machines running latest version of glite-UI (3.2.10-1 or higher):
Please restart Nagios after yaim execution. Otherwise you may see problems similar to SAM-1693. 
service nagios restart
Upgrading a node with yum requires a package exclusion, e.g.:
  1. on sam-nagios
    yum update --exclude sam-gridmon
  2. on sam-gridmon
    yum update --exclude sam-nagios
New package grid-monitoring-probes-org.sam-0.5.7-1.el5 solves the problem of running org.sam.WN* check on SL6 platform. More details can be found here: https://tomtools.cern.ch/jira/browse/SAM-2999. This package contains nagios binary that cannot be executed on 32-bit architecture and fails on some older WN platforms behind CE service type. Deployment of this package will cause CE service type tests to become UNKNOWN.
Because of a MRS bug SAM-3098, metrics are not added to metricdata tables on MySQL.
In order to fix it, please download the MRS-patch.sql attached (MRS-patch.sql) and deploy it (connected to your mysql db) with:
source MRS-patch.sql;
Perl module Directory::Queue is upgraded in Update-17. New dirqueue structure is not backward compatible with the previous version. After running Yaim please run:
rm -rf /var/spool/nagios2metricstore/*/*
rm -rf /var/spool/msg-nagios-bridge/*/*
If one doesn't perform steps described above SAM will not publish result to the central SAM-GridMon and test org.egee.SendToMsg will fail with the following error:
**ePN /usr/libexec/grid-monitoring/plugins/nagios/send_to_msg: "Directory::Queue::Simple: cannot link(/var/spool/msg-nagios-bridge/outgoing-messages/00000000/50b3452588432f, /var/spool/msg-nagios-bridge/outgoing-messages/00000000/50b3452588432f.lck): Operation not permitted".

Tickets List

Without Nightly Validation

jiraissues: Unable to determine if sort should be enabled.

During Nightly Validation

jiraissues: Unable to determine if sort should be enabled.

Attachments:
MRS-patch.sql (application/octet-stream)
Document generated by Confluence on Feb 27, 2014 10:19