Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1232342

Summary: hawkey only works with XML repo metadata and not with sqlite databases
Product: [Fedora] Fedora Reporter: Yavor Atanasov <yavor.k.atanasov>
Component: hawkeyAssignee: Packaging Maintenance Team <packaging-team-maint>
Status: CLOSED DEFERRED QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 23CC: jsilhan, mluscon, packaging-team-maint, rholy, vmukhame
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-07-29 10:50:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Yavor Atanasov 2015-06-16 14:14:32 UTC
Hey guys,

Currently, to initiate a hawkey Repo object, we need to provide the different types of metadata (primary, filelists etc.) in the form of xml files - http://hawkey.readthedocs.org/en/latest/reference-py-repo.html

Are there plans to have hawkey work with the sqlite databases as well or it is a conscious decision to ignore those in favour of xml.

Thank you
Yavor

Comment 1 Radek Holy 2015-06-16 15:14:16 UTC
Hi, I'd say that if we/you find a valid use case, we can consider adding the support.

Comment 2 Yavor Atanasov 2015-06-17 10:36:49 UTC
Hi Radek,

To give you a bit of context, my team is part of BBC's online platform teams. More specifically we create the services and tools that form the deployment pipeline for all of BBC's "cloud" services. Part of that ecosystem are a couple of services - one that bakes virtual machine images based on a set of repositories, packages and configuration, and another which is essentially a yum repository as a service - it allows our teams to create and maintain isolated repos. The repo service generates the yum metadata for a given repository on the fly when requested. Generating the metadata in the form of sqlite files is very fast for us as the data model behind the service mirrors closely the schema required for sqlite files (all we really do is copy required tables into sqlite, compress it and serve it). We do not currently generate XML files as yum works happily with sqlite.

We are migrating the bakery services to use hawkey and librepo. We want to use some of that goodness to speed up the baking process and do smarter things with pulling repo metadata, packages, installation etc.

It has all been great before we hit the problem I alluded to above - hawkey cannot currently work with our repository service as it requires XML files. You can see why it would be beneficial for us if hawkey could read the metadata from sqlite instead.

Out of curiosity what is the reason for choosing XML over sqlite (if there was a particular reason)?

Anyway, that's our use case.
Yavor

Comment 3 Radek Holy 2015-06-17 10:55:28 UTC
Thanks for the use case. Makes a lot of sense.

BTW, one problem that annoys me all the time is that the repodata format is not documented. So far, I looked at "createrepo" when I wanted to know something about the format. Using this as the documentation of the format, one can say that the XML files are mandatory since it's not possible to create sqlite-only repository using "createrepo". And if they are mandatory, we don't need the sqlite files.

Anyway, I guess that the main reason is that hawkey is a wrapper around libsolv and libsolv doesn't seem to support sqlite files.

Comment 4 Yavor Atanasov 2015-06-18 13:31:21 UTC
Would Red Hat be open to additions to libsolv so that works with sqlite files as well? I just want to rule out a possibility in which RH wants to standardise the poorly documented repo metadata formats and do that in favour of XML trying to deprecate sqlite.

Yavor

Comment 5 Radek Holy 2015-06-19 08:55:52 UTC
I believe that this is more a question for the libsolv upstream: https://github.com/openSUSE/libsolv

Comment 6 Yavor Atanasov 2015-06-19 09:27:27 UTC
Ah yes, I missed the fact that their github repo accepts issues to be raised (as opposed to the hawkey repo). Will raise an issue there.

Thank you Radek.

Comment 7 Jan Kurik 2015-07-15 13:59:40 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 23 development cycle.
Changing version to '23'.

(As we did not run this process for some time, it could affect also pre-Fedora 23 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 23 End Of Life. Thank you.)

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora23

Comment 8 Honza Silhan 2015-07-29 10:50:39 UTC
see issue in libsolv project: https://github.com/openSUSE/libsolv/issues/92

solv files would be more preferred. If you need exactly sqlite format, please, post your use case.

Comment 9 Honza Silhan 2015-07-29 10:57:27 UTC
* I mean the use case of other users wanting this feature.