Bug 1889726

The problem is:

/builddir/build/BUILD/unbound-1.10.1/unbound-1.10.1/pythonmod/pythonmod.c:371: undefined reference to `PyParser_SimpleParseFile'

I don't see this explicitly mentioned in https://docs.python.org/3.10/whatsnew/3.10.html but I see it mentioned in in https://docs.python.org/3.9/whatsnew/3.9.html#deprecated

Marking as high severity, because unbound is needed by gnutls.

Hi Tomáš,

I lack python expertise to see obvious fix. It seems offending line [1] is executed only to obtain Exception to print into log.

There is already PyRun_SimpleFile, which runs the script. It would print execption to stderr.
Second time, it runs the same code to include it in log. It should prefix the second message with:

./unbound-checkconf unbound.conf 
  File "./pythonmod/examples/log.py", line 63
    print "-"*100
          ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("-"*100)?
[1605031806] unbound-checkconf[2797510:0] error: pythonmod: can't parse Python script ./pythonmod/examples/log.py
[1605031806] unbound-checkconf[2797510:0] error: pythonmod: python error:   File "./pythonmod/examples/log.py", line 63
    print "-"*100
          ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("-"*100)?

This part is repeated just to be catched also in logs under unbound. Could we instead modify PyRun_SimpleFile call to produce just one exception,
then print it to stderr once and once into the log? We could then remove deprecated function just for that purpose.

[1605031806] unbound-checkconf[2797510:0] error: pythonmod: can't parse Python script ./pythonmod/examples/log.py
[1605031806] unbound-checkconf[2797510:0] error: pythonmod: python error:   File "./pythonmod/examples/log.py", line 63

But it seems PyRun_SimpleFile does not throw Exception. Can you recommend variant or flags, which would make it raise an Exception, which log_py_err() would then to log file?
After commenting out PyParser_SimpleParseFile it reports None, so it did not already raise an exception.

[1605032274] unbound-checkconf[2798279:0] error: pythonmod: python error: NoneType: None

I could not find any information about exceptions in documentation [2]. Is there better place to look into?

Reproducing the error is quite simple. Enter unbound build directory and unbound-1.* in it.

cd unbound-1.*/unbound-1.*

cat <<EOF > unbound.conf
server:
	module-config: "python iterator"

python:
	python-script: ./pythonmod/examples/log.py
EOF

./unbound-checkconf unbound.conf

Because examples are written for python2, they emit error.

1. https://github.com/NLnetLabs/unbound/blob/master/pythonmod/pythonmod.c#L371
2. https://docs.python.org/3.9/c-api/veryhigh.html#c.PyParser_SimpleParseFile

Victor is better suited to answer your question.

> I don't see this explicitly mentioned in https://docs.python.org/3.10/whatsnew/3.10.html but I see it mentioned in in https://docs.python.org/3.9/whatsnew/3.9.html#deprecated

I asked to explicitly document removed functions:
https://bugs.python.org/issue40939#msg380826

unbound used the removed function in pythonmod/pythonmod.c. Extract of unbound-1.12.0.tar.gz:

   if (PyRun_SimpleFile(script_py, pe->fname) < 0) {
      log_err("pythonmod: can't parse Python script %s", pe->fname);
      /* print the error to logs too, run it again */
      fseek(script_py, 0, SEEK_SET);
      /* we don't run the file, like this, because then side-effects
       *    s = PyRun_File(script_py, pe->fname, Py_file_input, 
       *        PyModule_GetDict(PyImport_AddModule("__main__")), pe->dict);
       * could happen (again). Instead we parse the file again to get
       * the error string in the logs, for when the daemon has stderr
       * removed.  SimpleFile run already printed to stderr, for then
       * this is called from unbound-checkconf or unbound -dd the user
       * has a nice formatted error.
      */
      /* ignore the NULL return of _node, it is NULL due to the parse failure
       * that we are expecting */
      (void)PyParser_SimpleParseFile(script_py, pe->fname, Py_file_input);
      log_py_err();
      PyGILState_Release(gil);
      fclose(script_py);
      return 0;
   }


> But it seems PyRun_SimpleFile does not throw Exception. Can you recommend variant or flags, which would make it raise an Exception, which log_py_err() would then to log file?

I'm not aware of any replacement for PyParser_SimpleParseFile() :-( I asked on the usptream issue:
https://bugs.python.org/issue40939#msg380828

Victor, how do we move this forward?

Miro:
> Victor, how do we move this forward?

I reported the issue to upstream: https://github.com/NLnetLabs/unbound/issues/378

We need to have this fixed in order to be able to do the actual upgrade in Fedora. 

There are now 2 upstream issues, but no movement. Victor, could you please help with the fix?

This bug appears to have been reported against 'rawhide' during the Fedora 34 development cycle.
Changing version to 34.

Oh, it seems like Wouter Wijngaards fixed the issue 6 days ago, without updating my issue https://github.com/NLnetLabs/unbound/issues/378

I created a PR to backport his change: https://github.com/NLnetLabs/unbound/issues/378

> I created a PR to backport his change: https://github.com/NLnetLabs/unbound/issues/378

Oops, the correct URL is: https://src.fedoraproject.org/rpms/unbound/pull-request/6

Thank You!