Suricata » Suricata-Update

Sascha Steinbiss wrote:

Hi,

I noticed that both suricata (via python/setup.py) and suricata-update (via its setup.py) both install code as part of the the suricata Python package, which leads to /usr/lib/python3/dist-packages/suricata/__init__.py (or equivalent) being claimed and installed by both. This apparently works if only Python's install facilities or pip are used -- however, in Debian packaging, both Debian packages (suricata and suricata-update) contain their own copies of this file and try to overwrite each other. This would require quite a bit of extra hassle to address.

I was wondering if it might be possible to rename the package for suricata-update (e.g. from suricata.update to suricata-update or the like) so both can be packaged independently? If you think differently, I would need to work around this issue somehow.

Hi Sascha!

The package name is already suricata-update (https://github.com/OISF/suricata-update/blob/master/setup.py#L32) but I get the problem you're referring to. suricata is the top directory for both the cases and hence it has been included in packages for both suricata and suricata-update. However, the package name "suricata.update" which you see at https://github.com/OISF/suricata-update/blob/master/setup.py#L39 means that we're including suricata/update (the directory) as a valid sub-package in our suricata-update package. Changing that to suricata-update would mean changing the current directory structure of the project. Not sure if it would be a good idea.

If you think I have misunderstood the problem, please let me know.

Thanks!
Sascha

Hi Shivani, thanks for your quick reply and clarification!

I understand that the intention is to ship the suricata.update sub-package as part of suricata-update, to complement the rest of the sub-packages in the suricata Python parent package (such as suricata.ctl and suricata.sc). This does not change the fact that both create the same suricata/__init__.py file during installation. In Debian this is a problem because in general a file with a given path can only ever be part of exactly one Debian package, and trying to install the suricata-update Debian package on top of the suricata Debian package will fail because this file will be present in both packages and apt won't allow one Debian package to overwrite a file that another Debian package created.

However, I have found that the following simple change:

--- a/setup.py
+++ b/setup.py
@@ -35,7 +35,6 @@
     "author": "Jason Ish",
     "author_email": "ish@unx.ca",
     "packages": [
-        "suricata",
         "suricata.update",
         "suricata.update.commands",
         "suricata.update.configs",

solves the problem for me. setup.py install will no longer create the __init__.py file, and suricata-update still works. Co-installation with the main suricata package's Python tree is possible. suricata-update still works with no problems. Would that be a viable solution from your point of view?

Thanks
Sascha

[..]

solves the problem for me. setup.py install will no longer create the __init__.py file, and suricata-update still works. Co-installation with the main suricata package's Python tree is possible. suricata-update still works with no problems. Would that be a viable solution from your point of view?

Seems like this breaks suricata-update for Python 2. I get the following error on a fresh debian:stable container after running "python setup.py install":

root@2338b01bc9b6:~/suricata-update# suricata-update 
Traceback (most recent call last):
  File "/usr/local/bin/suricata-update", line 32, in <module>
    from suricata.update import main
ImportError: No module named suricata.update

If this is the case, I think I may have found a solution that involves renaming the directory in setup.py. We'll have to survive Python 2 at least till the end of this year. :)

Edit: Just checked the deb package requires python3. So seems like its good to go.

solves the problem for me. setup.py install will no longer create the __init__.py file, and suricata-update still works. Co-installation with the main suricata package's Python tree is possible. suricata-update still works with no problems. Would that be a viable solution from your point of view?

Seems like this breaks suricata-update for Python 2. I get the following error on a fresh debian:stable container after running "python setup.py install":
[...]
for the solution that you have mentioned. Could you please confirm this on your end?

Indeed I can. OK so probably not a generic solution. I must admit I only checked for Python3 as in Debian we can define which Python version is used for a given package.

If this is the case, I think I may have found a solution that involves renaming the directory in setup.py. We'll have to survive Python 2 at least till the end of this year. :)

Oh, nice! I'd be curious to see what the solution was exactly. The less patches we need in Debian the better :)

Edit: Just checked the deb package requires python3. So seems like its good to go.

Yep, for all my recent packages I use Python3 -- it looks like it's quite a bit of work to get rid of all Python2 references in Debian before bullseye...

Anyway, thanks for looking at it!
Have a nice Sunday,
S.

I should not that this was done on purpose, as what I gathered from the Python tooling is that its a supported feature of the language.

Can your solution in https://redmine.openinfosecfoundation.org/issues/3156#note-3 be reserved to just the Debian packaging? I'm not sure it makes sense for us to pull that in.

Jason Ish wrote:

I should not that this was done on purpose, as what I gathered from the Python tooling is that its a supported feature of the language.

I see. It might be that Debian's default file handling is a bit strict there.

Can your solution in https://redmine.openinfosecfoundation.org/issues/3156#note-3 be reserved to just the Debian packaging? I'm not sure it makes sense for us to pull that in.

Sure. That's what I am doing right now -- I'm maintaining this change as part of a Debian-only patchset on top of your code. I agree that it's too specific to go into upstream.
Thanks for the clarification.

Yes, I think so. I will remove the suricata package from the setup.py files using Debian-specific patches.
However, this no longer makes it possible to just import suricata instead of import suricata.ctl, for example. Is that something you are planning to make use of in the near future?

I am curious -- can this be closed now?

Sascha Steinbiss wrote:

I am curious -- can this be closed now?

Do you have a reasonable work-around in your Debian builds?

Jason Ish wrote:

Sascha Steinbiss wrote:

I am curious -- can this be closed now?

Do you have a reasonable work-around in your Debian builds?

Yes, just as outlined above.