🧒
Sunbird ED
new release -7.0.0 (Jun 2024)Askdot
latest
latest
  • Learn
    • Overview
    • Capabilities
      • Learning Apps
      • Asset Sourcing
      • Organised Collections
      • Discover Content - Digital & Phygital
      • User Engagement
      • Rich and diverse content
      • Versatile Question Bank
      • Observability
      • Launch Course
      • Verifiable Credentials
      • Multi-Channel Chatbot
      • Targeted Programs
      • Manage Learn
        • Overview
        • What is an entity?
        • What is a Program?
        • What is a Project?
        • What is Observation?
        • What is a Survey?
        • What is Observation as a task inside a Project?
      • Product and Developer's Guide
        • Learning apps
          • Workflows
            • Onboarding of Users
            • Discovery of Content
            • Play content
            • Track progress and Earn credentials
            • Interacting / Collaborating
        • Asset Sourcing
        • Organised Collections
        • Discover Content - Digital & Phygital
        • User Engagement
        • Rich and Diverse Content
        • Versatile Question Bank
        • Observability
        • Launch Courses
        • Verifiable Credentials
        • Multi-Channel Chatbot
        • Targeted Programs
        • Manage Learn
          • Overview
          • Component Diagram
          • ML Core Service
          • ML Project Service
          • ML Survey Service
          • ML Report Service
          • ML Analytics Service
    • Tech Overview
      • Design Principles
      • Technical Architecture Diagram
      • Tech Stack
    • Adopters
      • DIKSHA
    • Roadmap
      • Plan for 2025-2026
      • Releases and Dates
  • USE
    • Getting Started - Setup
      • Pre-requisites
      • Install
      • Functional Configurations
    • Developer Guide - Overview
      • Architecture - Component Diagram
      • System Requirements
        • Learning Apps
      • Install Locally
        • SunbirdED Mobile
        • SunbirdED Portal
      • Easy Installer
        • Adding Support for a New Cloud Provider
      • Configuration
        • SunbirdEd Portal
        • Sunbird Mobile
      • Portal
        • Component Diagram
        • I18N (Resource Bundles)
        • Branding Name and Logo Configuration Guide
      • Desktop
        • Component Diagram
      • Mobile
        • Component Diagram
        • sunbird-mobile-sdk
        • Sunbird-mobile-app plugins
        • Configurations to setup mobile app
        • I18N (Resource Bundles)
      • Form service
        • Component Diagram
        • Data model
        • API's
      • Manage Learn
        • ML Core Service
          • Overview
          • User Flow Diagram
          • Component Diagram
          • Data Model
          • Folder Structure
          • API's
          • Deployment Overview
          • Local Service Setup Guide
        • ML Project Service
          • Overview
          • User Flow Diagram
          • Component Diagram
          • Data Model
          • Folder Structure
          • API's
          • Deployment Overview
          • Local Service Setup Guide
        • ML Survey Service
          • Overview
          • User Flow Diagram
          • Component Diagram
          • Data Model
          • Folder Structure
          • API's
          • Deployment Overview
          • Local Service Setup Guide
        • ML Report Service
          • Overview
          • User Flow Diagram
          • Component Diagram
          • Data Model
          • Folder Structure
          • API's
          • Deployment Overview
          • Local Service Setup Guide
        • ML Analytics Service
          • Overview
          • Component Diagram
          • Data Model
          • Setup Guide
            • ENV Variables ( Config.ini)
          • Ingestions
          • Folder Structure
          • Report creation and Updation Scripts
          • Deployment Overview
      • UI (User interface)
        • Angular Material
          • Overview
          • Installation
          • Material Icons
          • Components Usage
          • Theming
          • Theme Setup
          • Customazion
            • Palette
            • Theme
            • Accessibility
            • Colors
            • Typography
            • Components
              • Buttons
              • CC Components
        • Component Style Guide Version 1
          • Accordion
          • Buttons
          • Cards
          • Forms
          • Grid & Layout
          • Labels
          • Modals
          • Pagination
          • Rating
          • Search Box
          • Select Box
          • Tables
          • Tabs
          • Toast messages
          • Tooltip
          • Typography
        • SB-Styles: A Comprehensive Design Resource
        • SB-Themes Repository: Unifying Design Across Platforms
          • Classical Theme in Sunbird-Ed portal
          • Joyful Theme in Sunbird-Ed portal:
      • Reference Apps
        • Independent Libraries
          • Common Consumption Components
          • SunbirdEd Forms
          • Sunbird Client Services
          • Sunbird Styles
          • Sunbird Themes
          • Sunbird Tag Manager
      • API's
      • CSP changes
      • Cloud-Store SDK Maven Deployment Guide
      • Other
        • Building Images
        • Minimal forms
        • Telemetry
          • Trackable Collection
        • Platform
        • Learning Apps
    • Learn More
      • Dependencies
      • Specifications
        • SOFIE
        • SOFIE Implementation
  • SB Ed Releases
    • Sunbird v7.6.0 (Latest)
    • Sunbird v7.5.1
    • Release - SB 7.5
    • SB Release - 7.0.0
      • Release notes
      • Updating Sunbird Releases
        • 6.0.1 to 7.0.0
      • Release Calendar 7.0.0
      • Demo of released items
  • Engage
    • Discuss
    • Contribute to Sunbird ED
    • Extend / Contribute to Sunbird
    • Issue tracker/ Create Issue
  • Misc
    • Templates
      • Upgrade Sunbird release document
      • Release Notes
    • Misc Pages
      • Portal - Manage Learn - Reports
      • App - Manage Learn - component diagram
        • Projects
        • Observation & Survey
        • Program
      • Portal - Manage Learn - component diagram
      • Mobile form configurations
      • Content Indexing Flow
      • What are multiple databases used for
      • Course completion, reports and certificate issue
      • How to access Flink UI
      • What are all the flink jobs for
      • How does data flow into Druid
      • Minimal build properties
      • Delete User Functionality
    • Archived
      • High Level Capabilities
      • Workflows
      • Where Sunbird ED helps you
      • ED - Mobile App (going to archive)
        • 4.10.3 to 5.0.0
        • 5.0.0 to 5.1.0
      • Setting up Sunbird ED
      • Developer Documentation
      • API Reference Documentation
      • Detailed Documentation
      • Source Code
      • Actors & Actions
      • Detailed Capabilities
      • Data
      • Terminology
      • 5.1.0-hotfix (OCI )
      • Getting started (v7)
        • Deployment Overview
        • Pre-requisites
        • Install
          • Provision Cloud Infrastructure
          • Setup Jenkins
          • Update Ansible Variables
          • Build, Provision and Deploy
          • Functional Configurations
Powered by GitBook
On this page
  1. USE
  2. Developer Guide - Overview
  3. Manage Learn
  4. ML Analytics Service

Report creation and Updation Scripts

Migration scripts are used to create, update or retire Charts and Reports in the respective environments.

The automation scripts will be executed once in all releases to create new reports or charts, update the existing reports or charts or retire existing reports or charts based on the requirements for a release.

During each release, a new folder with the release version is created and the JSON config for the CRUD operation is created in the respective directories.

The index.py file will control the execution of the JSON config. The sequence of execution of the operations can be specified in the index file.

The index file will internally access the library created to call the backend APIs to perform CRUD operations.

The status is logged in the Mongo logger collection to avoid the duplication run of the charts/reports in case of any duplicate execution of the script or any re-run due to error.

\

Folder Structure :

Steps to Execute

Step 1: Login to the ml-analytics-service server

Step 2: Switch to the user

sudo su data-pipeline

Step 3: Activate virtual environment

. /opt/sparkjobs/spark_venv/bin/activate

Step 4: Navigate the release folder inside migrations

cd /opt/sparkjobs/ml-analytics-service/migrations/releases/6.0.0/

Step 5: Run the script which will create backend and frontend report configs

python index.py

Step 6: Running the Jenkins Job to Generate CSV and JSON in the Cloud

To execute the Jenkins Job and generate the CSV and JSON files in the cloud, please follow the steps below:

1. Log in to the VPN environment.

2. Access the Jenkins UI by navigating to the appropriate URL.

3. Once logged in, locate the "Deploy" section and select the desired environment (e.g., Staging, Production, etc.).

4. In the selected environment, find the "DataPipeline" folder and click on it.

5. Inside the "DataPipeline" folder, locate the "RunReportjob" job and click on it.

6. On the job page, look for the option to "Build with Parameters" and click on it.

7. You will be prompted to provide the required parameters for the job.

1. Specify the "report_id" parameter with the appropriate value.

(Retrieve all the "report_id" value from the 12th point and pass it here one by one -For example, you can refer to the "ml_report_name" value) or (in this path -

https://github.com/Sunbird-Ed/ml-analytics-service/tree/release-6.0.0/migrations/releases/6.0.0/config/backend/create )

Once you have obtained the required "report_id" value, provide it as the parameter during the job execution.

2. The "private_branch" parameter should default to the latest branch, so no action is needed unless specified otherwise.

3. If there is a "branch_or_tag" parameter, it should be set to the same value as the "private_branch" parameter.

8. Once you have entered the necessary parameters, click on the "Build" button to start the job.

9. The Jenkins job will now run, executing the required tasks to generate the CSV and JSON files in the cloud.

10. Monitor the job's progress through the Jenkins UI to ensure it completes successfully.

11. Once the job finishes, the CSV and JSON files should be generated in the designated cloud storage location.

12. Run the RunReportjob in Jenkins with the new report_id's :

Please note that the specific steps may vary depending on your Jenkins configuration and environment. Ensure you have the necessary permissions and access rights to perform these actions.

RunReportJob :

We utilize the Jenkins job named "RunReportJob" to acquire data or updated information tailored for HawkEye reports, which are displayed in the user interface.

This job functions as a backend process responsible for initiating a data product that retrieves data from Druid, a database system that stores data in both CSV and JSON formats. Once the data retrieval is complete, these files are stored within a Cloud environment. As a subsequent step, the gathered data becomes accessible through a designated portal, allowing users to conveniently view and engage with the generated reports.

Diksha_Reports_Analytics_Job_HawkEye_API'sDiksha_Reports_Analytics_Job_HawkEye_API's
Logo
Diksha_Reports_Portal_HawkEye_API'sDiksha_Reports_Portal_HawkEye_API's
Logo