0 votes
1 view

Hi,

I am currently using the Knowage Docker Server on Windows 10 and I would like to create a dataset using the Knowage Python Server but I am struggling to make it work. 

Here is my docker-compose file:

version: "3.1"
services:
  portainer:
    image: portainer/portainer
    command: -H unix:///var/run/docker.sock
    ports:
      - 9000:9000
      - 8000:8000
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - portainer_data:/data

  knowage:
    container_name: knowage
    image: knowagelabs/knowage-server-docker:8.1.0-SNAPSHOT
    hostname: knowage
    depends_on:
      - knowagedb
      - knowagecache
      - hazelcast
    ports:
      - "8080:8080"
    networks:
      - main
    environment:
      - DB_HOST=$DB_HOST
      - DB_PORT=$DB_PORT
      - DB_DB=$DB_DB
      - DB_USER=$DB_USER
      - DB_PASS=$DB_PASS

      - CACHE_DB_HOST=$CACHE_DB_HOST
      - CACHE_DB_PORT=$CACHE_DB_PORT
      - CACHE_DB_DB=$CACHE_DB_DB
      - CACHE_DB_USER=$CACHE_DB_USER
      - CACHE_DB_PASS=$CACHE_DB_PASS

      - HMAC_KEY=$HMAC_KEY
      - PASSWORD_ENCRYPTION_SECRET=$PASSWORD_ENCRYPTION_SECRET
      - PUBLIC_ADDRESS=localhost

      - HAZELCAST_HOSTS=hazelcast
      - HAZELCAST_PORT=5701
    volumes:
      - ./resources:/home/knowage/apache-tomcat/resources

  hazelcast:
    image: hazelcast/hazelcast:3.6.5
    networks:
      - main
    environment:
      - JAVA_OPTS=-Dhazelcast.local.publicAddress=hazelcast -Dhazelcast.config=/opt/hazelcast/hazelcast.xml
    volumes:
      - ./hazelcast-server.xml:/opt/hazelcast/hazelcast.xml

  knowagepython:
    image: knowagelabs/knowage-python-docker:8.1.0-SNAPSHOT
    hostname: knowagepython
    environment:
      - HMAC_KEY=$HMAC_KEY
      - KNOWAGE_PUBLIC_ADDRESS=http://localhost:8080
      - PUBLIC_ADDRESS=localhost
    networks:
      - main
    volumes:
      - "db:/var/lib/mysql"
    ports:
      - "5000:5000"

  knowager:
    image: knowagelabs/knowage-r-docker:8.1.0-SNAPSHOT
    environment:
      - HMAC_KEY=$HMAC_KEY
    networks:
      - main

  knowagedb:
    image: mariadb:10.3
    environment:
      - MYSQL_USER=$DB_USER
      - MYSQL_PASSWORD=$DB_PASS
      - MYSQL_DATABASE=$DB_DB
      - MYSQL_RANDOM_ROOT_PASSWORD=yes
    networks:
      - main
    volumes:
      - ./mariadb-data:/var/lib/mysql

  knowagecache:
    image: mariadb:10.3
    environment:
      - MYSQL_USER=$CACHE_DB_USER
      - MYSQL_PASSWORD=$CACHE_DB_PASS
      - MYSQL_DATABASE=$CACHE_DB_DB
      - MYSQL_RANDOM_ROOT_PASSWORD=yes
    networks:
      - main
    volumes:
      - ./mariadb-cache:/var/lib/mysql

networks:
  main:


volumes:
  portainer_data:
  db:

Following the documentation, I created 2 variables of category PYTHON_CONFIGURATION:

python.virtualenv.0.backend.url (http://knowagepython:5000)

python.virtualenv.0.frontend.url (http://knowagepython:5000)

Then I went to the Roles management section and checked the EDIT PYTHON SCRIPTS option.

Finally, I create my dataset and when checking the environment by selecting the python.virtualenv.0.backend.url I can see that the REST method is working in the logs (code 200) and I can see all the Python libraries available

But when trying the run my Python script I get a 401 error message (unauthorized).

2021-07-27 13:37:33 +0000] [36] [DEBUG] POST /dataset

172.27.0.7 - - [27/Jul/2021:13:37:33 +0000] "POST /dataset HTTP/1.1" 401 12 "-" "Jakarta Commons-HttpClient/3.0.1"

Could you please help me out or tell me what I am doing wrong?

Thanks in advance.

Regards,

Environment knowage-python-docker:8.1.0-SNAPSHOT, Docker, Windows 10
asked Aug 2, 2021 in API and SDK by yannick (120 points) | 1 view
Hi,

I see you are using the same HMAC key for knowage, knowagepython and knowager containers: that's perfect. You are using the $HMAC_KEY placeholder as value then you should check if you have a .env file in the same directory of your docker-compose.yaml; please check the value of that placeholder in the previous file: you don't need very exotic characters, you just need some random, base64 string for example.

I'll try to replicate your case but remember that versione 8.1.0-SNAPSHOT is in a very early stage of development.
I have checked the environment variables are used correctly by getting inside the Python webservice container and by looking at the config.xml file (cat config.xml) and everything looks as set in the env file used by Docker.

1 Answer

0 votes
Go to Roles Management, select your role, then under Authorizations, select Federated Dataset. Full documentation and step by step can be found here https://github.com/xogutu/knowage/blob/main/DataScience/DockerSetup.md
answered Aug 9, 2021 by xogutu (220 points)

I checked all the boxes regarding the roles as you can see below:

I can see the Python libraries from the Python environment.

But I still get a 401 unauthorized error when trying to create a Python dataset.

Here is the log from the Python webservice.

[2021-10-18 05:46:30 +0000] [15] [DEBUG] POST /dataset

192.168.0.18 - - [18/Oct/2021:05:46:30 +0000] "POST /dataset HTTP/1.1" 401 12 "-" "Jakarta Commons-HttpClient/3.0.1"

I just cannot get it to work unfortunately. 

Hi,

Did you fix this? If you did, how did you do it please?

Thanks.
1,553 questions
1,017 answers
2,037 comments
2,567 users