Chalk'it Python Management

The Python management features of Chalk'it allows to build and share Chalk'it dashboards with Python code execution in client-side using Pyodide or in server-side using Docker-based Chalk'it servers.

Client-side execution with Pyodide

The main tab Project librairies allows to select and install Pyodide-compatible Python librairies. Search box above may be used to find necessary librairies, and the Load button to install them into the browser.

After setup, they are associated to the projet, and may be called from all Python Script (Pyodide) dataNodes.

Server-side execution with Docker

First select the Python management feature.

Then click on the New button.

Docker image creation with requirements.txt

The Docker image creation wizard is then displayed.

Image name parameter is required. Select the Base image to use. If packages or requirements need to be installed in the image, the user can load from the local disk:

• a requirements.txt file (as specified in https://pip.pypa.io/en/stable/user_guide/#requirements-files)
• a package.zip file. Useful to deploy your own packages. Follow packaging instructions here https://packaging.python.org/tutorials/packaging-projects/. Test your package locally before uploading it.

Click Create to leave the wizard. The browser now displays the newly created Docker image. Please note that it does not yet have an image.

Click the gear icon located on the top right corner of the python image as indicated on the following picture. It starts building the image on the server.

The build process is launched on the server site and the image status changes to Building.

Click the eye icon to view the server's Docker image build log.

If build is completed succesfully, the bottom of the log screen indicates that the image was successfully built and tagged.

The image status then changes from Building to Ready. The size of the image is displayed.

Security restrictions

For security reasons, Python code runs on Docker containers that do not have disk or network rigths.

Usage in Python dataNodes

Python script dataNode allows the evaluation of a Python script in the selected Docker image. See its documentation in Python dataNodes documentation.

Docker image creation with package.zip

Two examples are provided.

MNIST predictor

This toy example uses Python and is based on the well known MNIST database of handwritten digits. In this dashboard, we embedd a previously trained Convolutional Neural Network (CNN) to predict the digit written on a selected image. CNN weights are defined as Keras h5 file and packaged in the zip file into mnist.h5 file.

• Create a Docker image for this package py-mnist-demo.zip, as illustrated above (name it mnist for example). Its directory structure is as follows:

  ├───py-mnist-demo
  │   │   MANIFEST.in
  │   │   README.md
  │   │   setup.py
  │   │   
  │   └───mnist
  │       │   predictor.py
  │       │   __init__.py
  │       │   
  │       └───resources
  │               0-img.npz
  │               mnist.h5
  

• Open this Chalk'it project mnist_prediction.xprjson to use this package. Edit datanode predictor to specify the mnist docker image as "Docker image". Save the result. After that modification, you will be able to run the exemple.

  

CSV & Pandas server-side

This toy example illustrates how to embedd a CSV file into a package.zip file to host-it server-side into a Docker image, and query it using the pandas library.

• Create a Docker image for this package py-csv-pandas-demo.zip, as illustrated above (name it serverSideCSV for example). Its directory structure is as follows:

  ├───py-csv-pandas-demo
  │   │   MANIFEST.in
  │   │   README.md
  │   │   setup.py
  │   │   
  │   └───reader
  │       │   query.py
  │       │   __init__.py
  │       │   
  │       └───resources
  │               CH4.csv
  

• Open this Chalk'it project server-side-csv-demo.xprjson to use this package. Edit datanode getData to specify the serverSideCSV docker image as "Docker image". Save the result. After that modification, you will be able to run the exemple.