5.0.0 to 5.1.0

Overview

This page details out the jobs required to be run as part of the upgrade from Sunbird Ed and CoKreat release 5.0.0 to release 5.1.0. Use the following table to understand the jobs that need to be executed in order to successfully complete the upgrade. Any jenkins job configuration or pre-requisites mentioned under manual configuration section needs to be done first before running any of the mentioned jobs. The order of the jobs should also be run as shown below. They can be run in parallel to speed up the execution.

Create cname for storage account

Create custom domain for storage account and update variables Before starting deployments.

Below are the steps to add custom domain to storage account

For Azure Click Here , AWS Click Here , for GCP Click Here

Build and Deploy for Sunbird

Sunbird_ED:

Service

Build job

Build tag

Deploy Job

Deploy tag

Comments

Sunbird RC

Service

Build Job

Build Tag

Deploy Job

Deploy Tag

Comments

Sunbird-Knowlg

Service/Compon

Build job

Build tag

Deploy Job

Deploy tag

Comments

Sunbird Lern

Service/Component

Build job

Build tag

Deploy Job

Deploy tag

Comments

Sunbird Inquiry

Service to be Build

Build Tag

Service to Deploy

Deploy Tag

Comments

Sunbird Obsrv

Service to be Build

Build Tag

Service to Deploy

Deploy Tag

Comments

Variables to update for Ed and vdn

variable name

description

Value

Add below Api's in Internal Kong configuration

Check if below api's exists or not before adding, if exists replace the existing one if not add it.

- name: createAndUpdateCertificateTemplate
  uris: "{{ private_ml_core_prefix }}/api/v1/certificateTemplates/createOrUpdate"
  upstream_url: "{{ ml_core_private_url }}/v1/certificateTemplates/createOrUpdate"
  strip_uri: true
  plugins:
  - name: jwt
  - name: cors
  - "{{ statsd_pulgin }}"
  - name: acl
    config.whitelist:
      - 'projectAccess'
  - name: rate-limiting
    config.policy: local
    config.hour: "{{ medium_rate_limit_per_hour }}"
    config.limit_by: credential
  - name: request-size-limiting
    config.allowed_payload_size: "{{ medium_request_size_limit }}"

- name: uploadCertificateTemplate
  uris: "{{ private_ml_core_prefix }}/api/v1/certificateTemplates/uploadTemplate"
  upstream_url: "{{ ml_core_private_url }}/v1/certificateTemplates/uploadTemplate"
  strip_uri: true
  plugins:
  - name: jwt
  - name: cors
  - "{{ statsd_pulgin }}"
  - name: acl
    config.whitelist:
      - 'projectAccess'
  - name: rate-limiting
    config.policy: local
    config.hour: "{{ medium_rate_limit_per_hour }}"
    config.limit_by: credential
  - name: request-size-limiting
    config.allowed_payload_size: "{{ medium_request_size_limit }}"

- name: projectCertificateReissue
  uris: "{{ private_ml_project_prefix }}/api/v1/userProjects/certificateReIssue"
  upstream_url: "{{ ml_project_private_url }}/v1/userProjects/certificateReIssue"
  strip_uri: true
  plugins:
  - name: jwt
  - name: cors
  - "{{ statsd_pulgin }}"
  - name: acl
    config.whitelist:
      - 'projectAccess'
  - name: rate-limiting
    config.policy: local
    config.hour: "{{ medium_rate_limit_per_hour }}"
    config.limit_by: credential
  - name: request-size-limiting
    config.allowed_payload_size: "{{ medium_request_size_limit }}"

- name: createCertificateTemplateSvg
  uris: "{{ private_ml_core_prefix }}/api/v1/certificateTemplates/createSvg"
  upstream_url: "{{ ml_core_private_url }}/v1/certificateTemplates/createSvg"
  strip_uri: true
  plugins:
  - name: jwt
  - name: cors
  - "{{ statsd_pulgin }}"
  - name: acl
    config.whitelist:
      - 'projectAccess'
  - name: rate-limiting
    config.policy: local
    config.hour: "{{ medium_rate_limit_per_hour }}"
    config.limit_by: credential
  - name: request-size-limiting
    config.allowed_payload_size: "{{ medium_request_size_limit }}"

- name: createAndUpdateCertificateBaseTemplate
  uris: "{{ private_ml_core_prefix }}/api/v1/certificateBaseTemplates/createOrUpdate"
  upstream_url: "{{ ml_core_private_url }}/v1/certificateBaseTemplates/createOrUpdate"
  strip_uri: true
  plugins:
  - name: jwt
  - name: cors
  - "{{ statsd_pulgin }}"
  - name: acl
    config.whitelist:
      - 'projectAccess'
  - name: rate-limiting
    config.policy: local
    config.hour: "{{ medium_rate_limit_per_hour }}"
    config.limit_by: credential
  - name: request-size-limiting
    config.allowed_payload_size: "{{ medium_request_size_limit }}"

- name: certificateBaseTemplateDetailsList
  uris: "{{ private_ml_core_prefix }}/api/v1/certificateBaseTemplates/list"
  upstream_url: "{{ ml_core_private_url }}/v1/certificateBaseTemplates/list"
  strip_uri: true
  plugins:
  - name: jwt
  - name: cors
  - "{{ statsd_pulgin }}"
  - name: acl
    config.whitelist:
      - 'projectAccess'
  - name: rate-limiting
    config.policy: local
    config.hour: "{{ medium_rate_limit_per_hour }}"
    config.limit_by: credential
  - name: request-size-limiting
    config.allowed_payload_size: "{{ medium_request_size_limit }}"

Execute Below API for ML Analytic Data Product Survey Status Report Configuration Update

curl -X POST \
  {{host}}/api/dataset/v1/add \
  -H 'authorization: Bearer {{jwt_token}}' \
  -H 'content-type: application/json' \
  -d '{
    "id": "ekstep.analytics.dataset.add",
    "ver": "1.0",
    "ts": "2016-12-07T12:40:40+05:30",
    "params": {
        "msgid": "4f04da60-1e24-4d31-aa7b-1daf91c46341"
    },
    "request": {"dataset":"druid-dataset","datasetSubId":"ml-survey-status-detail-exhaust","datasetConfig":{"type":"ml-survey-status-detail-exhaust","params":{"programId":"program-1","solutionId":"solution-1"}},"datasetType":"druid","visibility":"private","version":"v1","authorizedRoles":["PROGRAM_MANAGER","PROGRAM_DESIGNER"],"validationJson":{"type":"object","properties":{"tag":{"id":"http://api.ekstep.org/dataexhaust/request/tag","type":"string"},"dataset":{"id":"http://api.ekstep.org/dataexhaust/request/dataset","type":"string"},"datasetSubId":{"id":"http://api.ekstep.org/dataexhaust/request/datasetSubId","type":"string"},"requestedBy":{"id":"http://api.ekstep.org/dataexhaust/request/requestedBy","type":"string"},"encryptionKey":{"id":"http://api.ekstep.org/dataexhaust/request/encryptionKey","type":"string"},"datasetConfig":{"id":"http://api.ekstep.org/dataexhaust/request/datasetConfig","type":"object"}},"required":["tag","dataset","datasetConfig"]},"druidQuery":{"id":"ml-survey-status-detail-exhaust","labels":{"user_id" : "UUID","user_type" : "User type","user_sub_tye" : "User sub type","state_name" : "Declared State","district_name" : "District","block_name" : "Block","school_code" : "School ID","school_name" : "School Name","board_name" : "Declared Board","organisation_name" : "Org Name","program_name" : "Program Name","program_externalId" : "Program ID","survey_name" : "Survey Name","survey_externalId" : "Survey ID","survey_submission_id" : "survey_submission_id","submission_status" : "Status of submission","submission_date" : "Submission date"},"dateRange":{"interval":{"startDate":"1901-01-01","endDate":"2101-01-01"},"granularity":"all","intervalSlider":0},"metrics":[{"metric":"total_content_plays_on_portal","label":"total_content_plays_on_portal","druidQuery":{"intervals":"1901-01-01T00:00+00:00/2101-01-01T00:00:00+00:00","dataSource":"ml-survey-status","columns":["user_id", "user_type", "user_sub_type", "state_name", "district_name", "block_name", "school_code", "school_name", "board_name", "organisation_name", "program_name", "program_externalId", "survey_name", "survey_externalId", "survey_submission_id", "submission_status","submission_date"],"queryType":"scan"}}],"output":[{"zip":false,"label":"","dims":["date"],"fileParameters":["id","dims"],"metrics":["user_id", "user_type", "user_sub_type", "state_name", "district_name", "block_name", "school_code", "school_name", "board_name", "organisation_name", "program_name", "program_externalId", "survey_name", "survey_externalId", "survey_submission_id", "submission_status","submission_date"],"type":"csv"}],"sort":["UUID","Program ID","Survey ID","survey_submission_id"],"queryType":"scan"},"supportedFormats":["csv","zip"],"exhaustType":"OnDemand"}}'

Execute Below API for ML Analytic Data Product Project Status Report Configuration Update

curl -X POST \
  {{host}}/api/dataset/v1/add \
  -H 'authorization: Bearer {{jwt_token}}' \
  -H 'content-type: application/json' \
  -d '{
    "id": "ekstep.analytics.dataset.add",
    "ver": "1.0",
    "ts": "2016-12-07T12:40:40+05:30",
    "params": {
        "msgid": "4f04da60-1e24-4d31-aa7b-1daf91c46341"
    },
    "request": {
        "dataset": "druid-dataset",
        "datasetSubId": "ml-project-status-exhaust",
        "datasetConfig": {
            "type": "ml-project-status-exhaust",
            "params": {
                "programId": "program-1",
                "solutionId": "solution-1"
            }
        },
        "datasetType": "druid",
        "visibility": "private",
        "version": "v1",
        "authorizedRoles": [
            "PROGRAM_MANAGER",
            "PROGRAM_DESIGNER"
        ],
        "validationJson": {"type":"object","properties":{"tag":{"id":"http://api.ekstep.org/dataexhaust/request/tag","type":"string"},"dataset":{"id":"http://api.ekstep.org/dataexhaust/request/dataset","type":"string"},"datasetSubId":{"id":"http://api.ekstep.org/dataexhaust/request/datasetSubId","type":"string"},"requestedBy":{"id":"http://api.ekstep.org/dataexhaust/request/requestedBy","type":"string"},"encryptionKey":{"id":"http://api.ekstep.org/dataexhaust/request/encryptionKey","type":"string"},"datasetConfig":{"id":"http://api.ekstep.org/dataexhaust/request/datasetConfig","type":"object"}},"required":["tag","dataset","datasetConfig"]},
        "druidQuery":{"id":"ml-project-status-exhaust","labels":{"block_name":"Block","board_name":"Declared Board","project_title_editable":"Project Title","project_completed_date":"Project completion date of the user","user_type":"User Type","designation":"User sub type","school_code":"School ID","project_created_date":"Project start date of the user","project_last_sync":"Project last Synced date","project_duration":"Project Duration","status_of_project":"Project Status","project_id":"Project ID","project_description":"Project Objective","program_externalId":"Program ID","organisation_name":"Org Name","createdBy":"UUID","school_name":"School Name","district_name":"District","program_name":"Program Name","certificate_status_customised":"Certificate Status","state_name":"Declared State"},"dateRange":{"interval":{"startDate":"1901-01-01","endDate":"2101-01-01"},"granularity":"all","intervalSlider":0},"metrics":[{"metric":"total_content_plays_on_portal","label":"total_content_plays_on_portal","druidQuery":{"intervals":"1901-01-01T00:00+00:00/2101-01-01T00:00:00+00:00","dataSource":"sl-project","columns":["createdBy","user_type","designation","state_name","district_name","block_name","school_name","school_code","board_name","organisation_name","program_name","program_externalId","project_id","project_title_editable","project_description","project_created_date","project_completed_date","project_duration","project_last_sync","status_of_project","certificate_status_customised"],"queryType":"scan"}}],"output":[{"zip":false,"label":"","dims":["date"],"fileParameters":["id","dims"],"metrics":["createdBy","user_type","designation","state_name","district_name","block_name","school_name","school_code","board_name","organisation_name","program_name","program_externalId","project_id","project_title_editable","project_description","project_created_date","project_completed_date","project_duration","project_last_sync","status_of_project","certificate_status_customised"],"type":"csv"}],"sort":["UUID","Program ID","Project ID"],"queryType":"scan"},
        "supportedFormats": ["csv","zip"],
        "exhaustType": "OnDemand"
    }
}'

Add below variables in Core,KnowledgePlatform and Datapipeline inventory for ED and vdn

Refer to variables and comment section from here link to update below variables.

storage_endpoint variables for public, private, management and artifact is not required in current release, give empty string as value (cloud_public_storage_endpoint, cloud_private_storage_endpoint, etc)

// ###### common.yml ###############

cloud_public_storage_accountname: ""
cloud_public_storage_endpoint: ""
cloud_public_storage_region: ""
cloud_public_storage_project: ""

cloud_private_storage_accountname: ""
cloud_private_storage_endpoint: ""
cloud_private_storage_region: ""
cloud_private_storage_project: ""

cloud_management_storage_accountname: ""
cloud_management_storage_endpoint: ""
cloud_management_storage_region: ""
cloud_management_storage_project: ""

cloud_artifact_storage_accountname: ""
cloud_artifact_storage_endpoint: ""
cloud_artifact_storage_region: ""
cloud_artifact_storage_project: ""

Refer to variables and comment section from here link to update below secrets variables

// ########## secrets.yml ###############
cloud_public_storage_secret: ""
cloud_private_storage_secret: ""
cloud_management_storage_secret: ""
cloud_artifact_storage_secret: ""

Refer to variables and comment section from here link to update below variables

// ############# common.yml ##############
cloud_storage_certqr_bucketname: "certqr"
cloud_storage_chatbot_bucketname: "chatbot"
cloud_storage_dial_bucketname: "dial"
cloud_storage_flink_bucketname: "flink-state-backend"
cloud_storage_playercdn_bucketname: "player"
cloud_storage_public_bucketname: "public"
cloud_storage_publicreports_bucketname: "public-reports"
cloud_storage_privatereports_bucketname: "reports"
cloud_storage_samiksha_bucketname: "samiksha"
cloud_storage_schema_bucketname: "schema"
cloud_storage_sourcing_bucketname: "sourcing"
cloud_storage_offlineinstaller_bucketname: "{{env}}-offlineinstaller"
cloud_storage_content_bucketname: "sunbird-content-{{env}}"
cloud_storage_telemetry_bucketname: "telemetry-data-store"
cloud_storage_termsandcondtions_bucketname: "termsandcondtions"
cloud_storage_user_bucketname: "user"
cloud_storage_desktopappcrashlogs_bucketname: "desktopappcrashlogs"
cloud_storage_label_bucketname: "label"
cloud_storage_certservice_bucketname: "{{env}}-e-credentials"
cloud_storage_uci_bucketname: "uci-staging"
cloud_storage_cassandrabackup_bucketname: "cassandra-backup"
cloud_storage_dpcassandrabackup_bucketname: "dp-cassandra-backup"
cloud_storage_dppostgresbackup_bucketname: "dp-postgresql-backup"
cloud_storage_dpredisbackup_bucketname: "dp-redis-backup"
cloud_storage_esbackup_bucketname: "elasticsearch-snapshots"
cloud_storage_influxdbbackup_bucketname: "influxdb-backup"
cloud_storage_jenkinsbackup_bucketname: "jenkins-backup"
cloud_storage_mongobackup_bucketname: "mongodb-backup"
cloud_storage_neo4jbackup_bucketname: "neo4j-backup"
cloud_storage_redisbackup_bucketname: "redis-backup"
cloud_storage_artifacts_bucketname: "{{ artifacts_container }}"

Variables to remove from inventory
delete below variables from core/common.yml if its defined.
upstream_url
plugin_upstream_url
discussion_upstream_url

Migrations

Please execute the migration instructions mentioned by the dependent building blocks

Building Block

Migration Instructions link

Add new Form

To create a new form to show or remove the onboarding popups\

// Some codecurl --location --request POST '<https://{{env}}>/api/data/v1/form/create' \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {{API KEY}}' \
--data-raw '{
    "request": {
        "type": "newUserOnboarding",
        "action": "onboarding",
        "subType": "global",
        "rootOrgId": <please specify your root org id>,
        "component": "portal",
        "data":{
        "fields": {
                    "shownewUserOnboarding": "true"
                }
      }
    }
}'

CoKreat Deployments and migration

Please follow the instructions mentioned in the below link by CoKreat building block to upgrade CoKreat from 5.0.0 to 5.1.0

Variables lists and build tag is same for Ed and vdn. Use release-5.1.0-vdn_RC1 for knowlg service deployments in VDN

Last updated 1 year ago