Comment on page
5.2.0 to 6.0.0
This page details out the jobs required to be run as part of the upgrade of Sunbird Ed from release 5.2.0 to release 6.0.0. Use the following table to understand the Jenkins jobs that need to be executed in order to successfully complete the upgrade. Any jenkins job configuration or pre-requisites mentioned under manual configuration section needs to be done first before running any of the mentioned jobs. The order of the jobs should also be run as shown below.
1. Login to Jenkins VM
su jenkinsReplace the config.xml for the following mentioned Jenkins path
Jenkins Jobs | Path of Jenkins Server | Update Jenkins job URL |
|---|---|---|
Build/Core/Content | /var/lib/jenkins/jobs/Build/jobs/Core/jobs/Content | |
Build/Core/Learner | /var/lib/jenkins/jobs/Build/jobs/Core/jobs/Learner | |
Build/Core/Lms | /var/lib/jenkins/jobs/Build/jobs/Core/jobs/Lms | |
Build/Core/Analytics | /var/lib/jenkins/jobs/Build/jobs/Core/jobs/Analytics | |
Build/DataPipeline/CoreDataProducts | /var/lib/jenkins/jobs/Build/jobs/DataPipeline/jobs/CoreDataProducts | |
Build/DataPipeline/AnalyticsCore | /var/lib/jenkins/jobs/Build/jobs/DataPipeline/jobs/AnalyticsCore | |
Build/KnowledgePlatform/FlinkJobs | Build/KnowledgePlatform/FlinkJobs | |
Build/Lern/LernDataProducts | /var/lib/jenkins/jobs/Build/jobs/Lern/jobs/LernDataProducts | |
Build/Lern/LernFlinkJobs | /var/lib/jenkins/jobs/Build/jobs/Lern/jobs/LernFlinkJobs |
Update the following variables in sunbird-devops private repo according to your environment
cd ansible/inventory/dev/core/common.yaml### Variables added as part of 6.0.0 release - https://github.com/Sunbird-Ed/ml-analytics-service/tree/release-6.0.0/release-notes/6.0.0.md
#### ML Analytics Service #####
ml_analytics_cname_url: # Reference value - https://obj.dev.sunbirded.org/samiksha/
### Variables added as part of 6.0.0 release - https://github.com/Sunbird-Ed/ml-analytics-service/tree/release-6.0.0/release-notes/6.0.0.md
ml_analytics_client_id : project-sunbird-dev-client # Keycloak client group
ml_analytics_username : reportAdmin1 # Username of a user which has roles of "PROGRAM_MANAGER", "PROGRAM_DESIGNER", "REPORT_ADMIN", "REPORT_VIEWER"
ml_analytics_createdBy : fb85a044-d9eb-479b-a55a-faf1bfaea14d # Unique system generated user UUID which is the same user as above
ml_container: samiksha
# provide the s3 compatible endpoint
# for AWS
# cloud_private_storage_account_endpoint: "https://s3.{{ cloud_public_storage_region }}.amazonaws.com"
# for OCI
#cloud_private_storage_account_endpoint: "https://<oci_namespace>.compat.objectstorage.{{cloud_public_storage_region}}.oraclecloud.com"
cloud_private_storage_account_endpoint: "{{ cloud_public_storage_endpoint }}" # Leave Blank for Azure
bb_name: "" #Leave Blank
cd ansible/inventory/dev/core/secrets.yaml#### ML Analytics Service #####
### Variables added as part of 6.0.0 release - https://github.com/Sunbird-Ed/ml-analytics-service/tree/release-6.0.0/release-notes/6.0.0.md
# Examples values added below
ml_analytics_password : Test@123 # Password of a user which has roles of "PROGRAM_MANAGER", "PROGRAM_DESIGNER", "REPORT_ADMIN", "REPORT_VIEWER"
ml_analytics_authorization_access_token : 'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJkOTY3NzRjYzXXXXXXXXXXXXXQ4Y2RiOWQ2Mzg0OSJ9.n4hXxKxl_698yeZPSWhXeGvMOb4esfgPadIZe8jZ0Z4' # Bearer auth token which has access to APIs as defined in the https://github.com/Sunbird-Ed/ml-analytics-service/tree/release-6.0.0/release-notes/6.0.0.md
ml_analytics_client_secret : fd241dce-4ZZ9-47e1-97cf-1c7de7a44216 # Keycloak client secret for creating tokens, mapped to client id supplied to variable ml_analytics_client_id
cd ansible/inventory/dev/KnowledgePlatform/common.yamldruid_storage_type: "" # Your cloud service provider name. Supported values are aws, azure, gcloud
# Optional variables - Can be left blank
cloud_storage_report_verfication_bucketname: ""
dp_storage_endpoint_config: ""
cd ansible/inventory/dev/core/common.yamlml_cloud_config: |
CLOUD_STORAGE=AZURE
AZURE_ACCOUNT_NAME={{sunbird_public_storage_account_name}}
AZURE_ACCOUNT_KEY={{sunbird_public_storage_account_key}}
AZURE_STORAGE_CONTAINER=samiksha
Service | Build job | Build tag | Comments | Deploy Job | Deploy tag |
|---|---|---|---|---|---|
Portal | Build/Core/Player
| release-6.0.0_RC40 | | Deploy/Kubernetes/Player | release-6.0.0 |
ML Services | Build/managed-learn/ml-projects-service | release-6.0.0_RC7 | | Deploy/managed-learn/ml-projects-service | release-6.0.0 |
| Build/managed-learn/ml-core-service | release-6.0.0_RC22 | | Deploy/managed-learn/ml-core-service | release-6.0.0 |
| Build/managed-learn/ml-reports-service | release-6.0.0_RC7 | | Deploy/managed-learn/ml-reports-service | release-6.0.0 |
| Build/managed-learn/ml-survey-service | release-6.0.0_RC4 | | Deploy/managed-learn/ml-survey-service | release-6.0.0 |
ML-Analytics | | | ml_analytics_version: release-6.0.0_RC14 | Deploy/managed-learn/ml-analytics-service | release-5.2.0 |
Service/Component | Build job | Build tag | Deploy Job | Deploy tag | Comments |
|---|---|---|---|---|---|
Data pipeline | Build/Lern/FlinkJobs | github_release_tag:release-5.3.1_RC1,CLOUD_STORE_GROUP_ID:org.sunbird,CLOUD_STORE_ARTIFACT_ID:cloud-store-sdk_2.12,CLOUD_STORE_VERSION:1.4.6 | Deploy/Lern/FlinkJobs | release-5.3.0_RC5 | |
Lern data products | Build/Lern/LernDataProducts | github_release_tag:release-5.3.1_RC1,CLOUD_STORE_GROUP_ID:org.sunbird,CLOUD_STORE_ARTIFACT_ID:cloud-store-sdk_2.12,CLOUD_STORE_VERSION:1.4.6 | Deploy/Lern/LernDataProducts | release-5.1.0_RC1 | |
Analytics | Build/Core/Analytics | github_release_tag:release-5.1.2_RC1,CLOUD_STORE_GROUP_ID:org.sunbird,CLOUD_STORE_ARTIFACT_ID:cloud-store-sdk_2.12,CLOUD_STORE_VERSION:1.4.0 | Deploy/Kubernetes/Analytics | release-5.2.0-lern | |
User&Org Service | Build/Core/Learner | github_release_tag:release-5.3.1_RC1,CLOUD_STORE_GROUP_ID:org.sunbird,CLOUD_STORE_ARTIFACT_ID:cloud-store-sdk_2.12,CLOUD_STORE_VERSION:1.4.0CLOUD_STORE_GROUP_ID:org.sunbird,CLOUD_STORE_ARTIFACT_ID:cloud-store-sdk,CLOUD_STORE_VERSION:1.4.6 | Deploy/Kubernetes/Learner | release-5.3.0-lern | |
Batch Service | Build/Core/Lms | github_release_tag:release-5.3.2_RC1,,CLOUD_STORE_GROUP_ID:org.sunbird,CLOUD_STORE_ARTIFACT_ID:cloud-store-sdk_2.12,CLOUD_STORE_VERSION:1.4.0CLOUD_STORE_GROUP_ID:org.sunbird,CLOUD_STORE_ARTIFACT_ID:cloud-store-sdk,CLOUD_STORE_VERSION:1.4.6 | Deploy/Kubernetes/Lms | release-5.3.0-lern | |
Configurations to be done by System admin:
- 1.Setup default 'Data Security Policy' settings using tenant preference API.
curl --location --request PATCH '{{host}}/api/org/v2/preferences/update' \
--header 'x-authenticated-user-token: {{user_authentication_token}}' \
--header 'Authorization: Bearer {{kong_api_token}}' \
--header 'Content-Type: application/json' \
--data-raw '{
"request": {
"orgId": "default",
"key": "dataSecurityPolicy",
"data": {
"level": "PLAIN_DATASET",
"dataEncrypted": "No",
"comments": "Data is not encrypted",
"job": {
"userinfo-exhaust": {
"level": "PASSWORD_PROTECTED_DATASET",
"dataEncrypted": "No",
"comments": "Password protected file."
},
"program-user-exhaust": {
"level": "PASSWORD_PROTECTED_DATASET",
"dataEncrypted": "No",
"comments": "Password protected file."
}
},
"securityLevels": {
"PLAIN_DATASET": "Data is present in plain text/zip. Generally applicable to open datasets.",
"PASSWORD_PROTECTED_DATASET": "Password protected zip file. Generally applicable to non PII data sets but can contain sensitive information which may not be considered open.",
"TEXT_KEY_ENCRYPTED_DATASET": "Data encrypted with a user provided encryption key. Generally applicable to non PII data but can contain sensitive information which may not be considered open.",
"PUBLIC_KEY_ENCRYPTED_DATASET": "Data encrypted via an org provided public/private key. Generally applicable to all PII data exhaust."
}
}
}
}'
Service/Component | Build job | Build tag | Deploy Job | Deploy tag | Comments |
|---|---|---|---|---|---|
Data pipeline | Build/DataPipeline/AnalyticsCore | github_release_tag:release-5.1.2_RC1,CLOUD_STORE_GROUP_ID:org.sunbird,CLOUD_STORE_ARTIFACT_ID:cloud-store-sdk_2.12,CLOUD_STORE_VERSION:1.4.0 | Deploy/DataPipeline/AnalyticsCore | release-5.0.0_RC1 | |
Core Data products | Build/DataPipeline/CoreDataProducts | github_release_tag:release-5.1.2_RC1,CLOUD_STORE_GROUP_ID:org.sunbird,CLOUD_STORE_ARTIFACT_ID:cloud-store-sdk_2.12,CLOUD_STORE_VERSION:1.4.0 | Deploy/DataPipeline/CoreDataProducts | release-5.0.0_RC1 | |
Service/Component | Build job | Build tag | Deploy Job | Deploy tag | Comments |
|---|---|---|---|---|---|
Knowledge-platform-jobs | Build/KnowledgePlatform/FlinkJobs | github_release_tag:release-5.5.0_RC2,CLOUD_STORE_GROUP_ID:org.sunbird,CLOUD_STORE_ARTIFACT_ID:cloud-store-sdk_2.12,CLOUD_STORE_VERSION:1.4.6 | Deploy/KnowledgePlatform/FlinkJobs | release-5.4.1_RC1 | Jobs to be deployed: 1. asset-enrichment 2.content-publish 3. post-publish-processor 4. qrcode-image-generator 5. video-stream-generator |
content-service | Build/Core/Content | github_release_tag:release-5.5.0_RC1,CLOUD_STORE_GROUP_ID:org.sunbird,CLOUD_STORE_ARTIFACT_ID:cloud-store-sdk,CLOUD_STORE_VERSION:1.4.6 | Deploy/Kubernetes/Content | release-6.0.0 | |
Plugins | Build/Plugins/ContentPlugins | release-5.2.1_RC3 | Deploy/Plugins/ContentPlugins | release-6.0.0 | |
Plugins | Build/Plugins/ContentEditor | release-5.2.1_RC3 | Deploy/Plugins/ContentEditor | release6.0.0 | |
Service/Component | Build job | Build tag | Deploy Job | Deploy tag | Comments |
|---|---|---|---|---|---|
Assessment | Build/Core/Assessment | inquiry_release_tag:release-5.6.0_RC1,core_release_tag:release-5.2.0_RC2 | Deploy/Kubernetes/Assessment | release-6.0.0 | |
InQuiryFlink Job | Build/KnowledgePlatform/InquiryFlinkJob | release-5.7.0_RC2 | Deploy/KnowledgePlatform/InquiryFlinkJob | release-5.7.0_RC1 | |
Last modified 11d ago