Welcome to Knowage Q&A, where you can ask questions and receive answers from other members of the community.

Categories

Most popular tags

cockpit dataset error chart datasource installer datasets installation widget parameters business-model olap knowage knowage_7_1 integration login knowage_7_1-ce report-designer cross-navigation analyticaldriver

How to launch Knowage-python service

+1 vote
1 view

I try to start using the python widget in Knowage application, but I have problem to set flask service on servers like WSGI or Apache. I follow the knowage documentation but there is no details how to launch such service. I want to mention I use the Windows 10 Pr.
Honestly I’m not much experience with python yet, so I need help I would like to get more details about such configuration, for example some sample files code how to deploy web service on those servers. The knowage documentation contains general information about it and sometimes confusing , for example it says to use the entry point ( The entry point for the application is Knowage-Python/pythonwebservice/knowage-python.py and the default port is 5000)

The python module for Knowage is available on GitHub and the structure of my project application below

|__ myapps
     |__ pythonwebservice
           |__ api
           |__ logs
           |__ __init__.py
           |__ knowage-python.py
     |__ wsgi_scripts
          |__ my_wsgi.py

The file knowage-python.py contains

import sys
from app.services import create_app
from app.utilities import constants
import logging

LOG_FORMAT = "[%(filename)s:%(lineno)s - %(funcName)20s() ] %(message)s"
logging.basicConfig(format=LOG_FORMAT)

application = create_app()

if __name__ == '__main__':
    if len(sys.argv) > 1:
        port = int(sys.argv[0])
    else:
        port = 5000
    open(constants.LOG_FILE, 'w+').close() #clean log file
    original_stderr = sys.stderr
    #sys.stderr = open(constants.LOG_FILE, 'a')
    application.run(host='127.0.0.1', debug=False, port=port)
    #application.run(host='0.0.0.0', debug=False, port=port, ssl_context=('cert.pem', 'key.pem'))

The my_wsgi.py

import sys
sys.path.insert(0, 'C:\\myapps')
from pythonwebservice import app as application

and the virtualhost for Apache myapp.conf

<VirtualHost *:5000>
    ServerName localhost
    WSGIScriptAlias / C:/myapps/wsgi_scripts/my_wsgi.py                           
    <Directory C:/myapps/pythonwebservice>
            Options +Indexes +Includes +FollowSymLinks +MultiViews
            AllowOverride All
            Require local
            Require all granted
    </Directory>
</VirtualHost>

It doesn't work.

I will apprecite any suggestions or examples

Environment Knowage 7,2 Windows 10 prof, Chrome
in Integration by (290 points)

Hi,

as specified in the documentation you quoted, knowage-python.py is the entry point of the application.
This means that you don't have to create a WSGI script yourself, but you can directly use knowage-python.py file.

Best regards,
Marco

by (9.3k points)

Thank you very much for your respond. As you have recomended I have opened the cmd window and lunch the knowage-python service directly. Finally I get (below) so it seems to be  OK,

 Serving Flask app "pythonwebservice.app.services" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
[_internal.py:113 -                 _log() ]  * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

Also I have made jpy setting, creating the jpyconfig.properties file.and placed it  in knowagedataminingengine as documentation says. I set the config xml file as well.

When I try to create new DataSet in Knowage  clicking the "CHECK ENVIROMENT" nothing is displayed, but I see the errors in terminal

[_internal.py:113 -                 _log() ]  * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
[_internal.py:113 -                 _log() ] 127.0.0.1 - - [16/Oct/2020 19:26:52] "[33mGET / HTTP/1.1[0m" 404 -
[_internal.py:113 -                 _log() ] 127.0.0.1 - - [16/Oct/2020 19:27:39] code 400, message Bad request version ('÷\x8cáê\x82§\x11Üu\x00\x00VÀ$À(\x00=À&À*\x00k\x00jÀ')
[_internal.py:113 -                 _log() ] 127.0.0.1 - - [16/Oct/2020 19:27:39] "[35m[1m Á  ½_‰ØŽ«˜C›²öÍL¼d%-E· ÷ŒáꂧÜu  VÀ$À( =À&À* k jÀ[0m" HTTPStatus.BAD_REQUEST -
[_internal.py:113 -                 _log() ] 127.0.0.1 - - [16/Oct/2020 19:27:39] code 400, message Bad request version ('z\x00\x00VÀ$À(\x00=À&À*\x00k\x00jÀ')
[_internal.py:113 -                 _log() ] 127.0.0.1 - - [16/Oct/2020 19:27:39] "[35m[1m Á  ½_‰ØýjD9qÿrœsDœ¹Cy¬6\ú\ùwCz  VÀ$À( =À&À* k jÀ[0m" HTTPStatus.BAD_REQUEST -
[_internal.py:113 -                 _log() ] 127.0.0.1 - - [16/Oct/2020 19:27:39] code 400, message Bad HTTP/0.9 request type ('\x16\x03\x03\x00Á\x01\x00\x00½\x03\x03_\x89Ø')
[_internal.py:113 -                 _log() ] 127.0.0.1 - - [16/Oct/2020 19:27:39] "[35m[1m Á  ½_‰Ø’a’øô0FåÆvXJ=½˜sRäÞdÖÅ=Ší½  VÀ$À( =À&À* k jÀ[0m" HTTPStatus.B

So my question is ; where could  be the error ?, why the service doesn't recognize the requests from Knowage?

by (290 points)

Hello,

what is the address you specified in the Configuration Management?
The service is running with http protocol, is it possible that you are trying to contact it via https instead?

Also, as previously said the entry point is knowage-python.py file, but we strongly recommend not to run it as a simple script (via the python interpreter).
What I mean is that you should use the knowage-python.py file as an entry point in your server settings (WSGIScriptAlias = knowage-python.py).

Bye,
Marco

by (9.3k points)

Thanks for your suggestion, I have done it, but without sucess. Ok let me present my settings:

my Configuration Management


so all of them are localhost:5000

It is the jpyconfig.properties

# Created by 'jpyutil.py' tool on 2020-10-10 23:48:25.757449
# This file is read by the jpy Java API (org.jpy.PyLib class) in order to find shared libraries
jpy.jpyLib = C:\\Users\\xxxxxxxx\\Desktop\\Knowage\\venv\\Lib\\site-packages\\jpy.cp38-win_amd64.pyd
jpy.jdlLib = C:\\Users\\xxxxxxx\\Desktop\\Knowage\\venv\\Lib\\site-packages\\jdl.cp38-win_amd64.pyd
jpy.pythonLib = C:\\Users\\xxxxxxx\\Desktop\\Knowage\\venv\\Scripts\\python38.dll
jpy.pythonPrefix = C:\\Users\\xxxxxxx\\Desktop\Knowage\\venv\\Scripts
jpy.pythonExecutable = C:\\Users\\xxxxxxx\\Desktop\Knowage\\venv\\Scripts\\python.exe

Config.xml

<data>
             <environment name="knowage-python">
                     <hmackey>38ONPZsG</hmackey>
                     <knowageaddress>localhost:8080</knowageaddress>
                     <pythonaddress>localhost:5000</pythonaddress>
                     <bokehportsrange>57000-58000</bokehportsrange>
             </environment> 
</data>

Part of my httpd.conf

   ...
    ...

    LoadFile "C:/Program Files/Python38/python38.dll" 
    WSGIPythonHome  "C:/Program Files/Python38"
    LoadModule wsgi_module  "C:/Users/xxxxxxxxx/Desktop/Knowage/venv/Lib/site- packages/mod_wsgi/server/mod_wsgi.cp38-win_amd64.pyd

         # Include Flask file 
         Include conf/my_virt.conf

my virtualhost file - my_virt.conf as   EntryPoint

<VirtualHost *:5000>
   ServerName localhost   
   WSGIScriptAlias / C:/Users/xxxxxxx/Desktop/Knowage/venv/Lib/site-packages/pythonwebservice/knowage-python.py
     <Directory C:/Users/xxxxxxxx/Desktop/Knowage/venv/Lib/site-packages/pythonwebservice>
            Options +Indexes +Includes +FollowSymLinks +MultiViews
            AllowOverride All
            Require local
            Require all granted
     </Directory>
</VirtualHost>

I have lauch the Apache and try to set DataSet in Knowage, I see it tries to connect to knowage-python but it doesn't get the python modules.

I have check the error log files and:

The Apache error.log

[Mon Oct 19 14:46:53.703853 2020] [mpm_winnt:notice] [pid 18508:tid 848] AH00455: Apache/2.4.43 (Win64) PHP/7.4.8 mod_wsgi/4.7.1 Python/3.8 configured -- resuming normal operations
[Mon Oct 19 14:46:53.703853 2020] [mpm_winnt:notice] [pid 18508:tid 848] AH00456: Apache Lounge VS16 Server built: Apr 21 2020 16:23:13
[Mon Oct 19 14:46:53.703853 2020] [core:notice] [pid 18508:tid 848] AH00094: Command line: 'C:\\Apache24\\bin\\httpd.exe -d C:/Apache24'
[Mon Oct 19 14:46:53.714821 2020] [mpm_winnt:notice] [pid 18508:tid 848] AH00418: Parent: Created child process 14912
[Mon Oct 19 14:46:55.373820 2020] [mpm_winnt:notice] [pid 14912:tid 924] AH00354: Child: Starting 64 worker threads.

The Apache access.log

127.0.0.1 - - [19/Oct/2020:14:48:03 +0200] "\x16\x03\x03" 400 226
127.0.0.1 - - [19/Oct/2020:14:48:03 +0200] "\x16\x03\x03" 400 226
127.0.0.1 - - [19/Oct/2020:14:48:03 +0200] "\x16\x03\x03" 400 226
127.0.0.1 - - [19/Oct/2020:14:48:03 +0200] "\x16\x03\x03" 400 226

After it I have opened localhost:5000 in chrome

I see the new lines in Apache error.log

[Mon Oct 19 14:49:11.714764 2020] [wsgi:error] [pid 14912:tid 1396] [client 127.0.0.1:59514] mod_wsgi (pid=14912): Failed to exec Python script file 'C:/Users/xxxxxxx/Desktop/Knowage/venv/Lib/site-packages/pythonwebservice/knowage-python.py'.
[Mon Oct 19 14:49:11.714764 2020] [wsgi:error] [pid 14912:tid 1396] [client 127.0.0.1:59514] mod_wsgi (pid=14912): Exception occurred processing WSGI script 'C:/Users/xxxxxxx/Desktop/Knowage/venv/Lib/site-packages/pythonwebservice/knowage-python.py'.
[Mon Oct 19 14:49:11.731606 2020] [wsgi:error] [pid 14912:tid 1396] [client 127.0.0.1:59514] Traceback (most recent call last):\r
[Mon Oct 19 14:49:11.731606 2020] [wsgi:error] [pid 14912:tid 1396] [client 127.0.0.1:59514]   File "C:/Users/xxxxxxx/Desktop/Knowage/venv/Lib/site-packages/pythonwebservice/knowage-python.py", line 4, in <module>\r
[Mon Oct 19 14:49:11.731606 2020] [wsgi:error] [pid 14912:tid 1396] [client 127.0.0.1:59514]     from app.services import create_app\r
[Mon Oct 19 14:49:11.731606 2020] [wsgi:error] [pid 14912:tid 1396] [client 127.0.0.1:59514] ModuleNotFoundError: No module named 'app'\r
[Mon Oct 19 14:49:11.850870 2020] [wsgi:error] [pid 14912:tid 1388] [client 127.0.0.1:59515] mod_wsgi (pid=14912): Failed to exec Python script file 'C:/Users/xxxxxxx/Desktop/Knowage/venv/Lib/site-packages/pythonwebservice/knowage-python.py'., referer: http://localhost:5000/
[Mon Oct 19 14:49:11.850870 2020] [wsgi:error] [pid 14912:tid 1388] [client 127.0.0.1:59515] mod_wsgi (pid=14912): Exception occurred processing WSGI script 'C:/Users/xxxxxxx/Desktop/Knowage/venv/Lib/site-packages/pythonwebservice/knowage-python.py'., referer: http://localhost:5000/
[Mon Oct 19 14:49:11.850870 2020] [wsgi:error] [pid 14912:tid 1388] [client 127.0.0.1:59515] Traceback (most recent call last):\r, referer: http://localhost:5000/

and te Apache access.log has extra lines

127.0.0.1 - - [19/Oct/2020:14:49:11 +0200] "GET / HTTP/1.1" 500 530
127.0.0.1 - - [19/Oct/2020:14:49:11 +0200] "GET /favicon.ico HTTP/1.1" 500 530
127.0.0.1 - - [19/Oct/2020:14:50:11 +0200]

=====================================================
..

So it seems to be problem with http protocol, but where could be the problem ?

Regards

miroslaw.

by (290 points)

Hi Miroslaw,

there are some updates to the knowage-python module expected to come up with the 7.3 release that might help you solve the problem.

However, by the errors contained in the Apache error.log it looks like the configuration in the VirtualHost file is not correct. I think that what Apache should do is not directly serve the python module, but it should only work as a proxy towards an application server such as waitress.

by (9.3k points)

Hi,

Thanks for your respond, my question is when you expect new update of knowage-python ? Also I have watched your webinar and see the working python module, so on what platform was made the Knowage and python module, what kind of web service have been used, is it Linux, docker server?
Mean time I have made some changes as you have suggested in your last answer, and made some test of it.

1.Part of code knowage-python.py (port 5001)

if __name__ == '__main__':
    if len(sys.argv) > 1:
        port = int(sys.argv[0])
    else:
        port = 5001
    open(constants.LOG_FILE, 'w+').close() #clean log file
    original_stderr = sys.stderr
    #sys.stderr = open(constants.LOG_FILE, 'a')
    application.run(host='127.0.0.1', debug=False, port=port)
    #application.run(host='0.0.0.0', debug=False, port=port, ssl_context=('cert.pem', 'key.pem'))

2.Apache proxy settings

VirtualHost *:5000>

    ProxyPreserveHost On

    ProxyPass / http://127.0.0.1:5001/
    ProxyPassReverse / http://127.0.0.1:5001/

    serverName localhost

</VirtualHost>

3.Chrome browser http://localhost:5000

4.Terminal message from running knowage-python application on dev server

WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
[_internal.py:113 -                 _log() ]  * Running on http://127.0.0.1:5001/ (Press CTRL+C to quit)
[_internal.py:113 -                 _log() ] 127.0.0.1 - - [21/Oct/2020 12:58:44] "GET / HTTP/1.1" 404 -
[_internal.py:113 -                 _log() ] 127.0.0.1 - - [21/Oct/2020 12:59:46] "GET / HTTP/1.1" 404 -

After it I've tried open python enviroment from Knowage creating new Dataset , it seems to try connect but nothing come out

5.It is the Apache access error log

::1 - - [21/Oct/2020:12:56:12 +0200] "\x16\x03\x01\x02" 400 226
::1 - - [21/Oct/2020:12:56:12 +0200] "\x16\x03\x01\x02" 400 226
::1 - - [21/Oct/2020:12:58:44 +0200] "GET / HTTP/1.1" 404 232
::1 - - [21/Oct/2020:12:59:44 +0200] "-" 408 -
::1 - - [21/Oct/2020:12:59:46 +0200] "GET / HTTP/1.1" 404 232

It looks like Knowage still  tries to conenct with Apache by https protocol . Could be the Knowage requests  sent encrypted somehow?

Regards

Miroslaw

by (290 points)

Please log in or register to answer this question.

Powered by Question2Answer
...