Building Koha Debian packages in Docker

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Building Koha Debian packages in Docker

David Cook

Hi folks,

 

I was wondering if anyone had a Docker image for building Koha Debian packages. I scouted around a bit and found this image from Jonathan https://gitlab.com/joubu/koha-dpkg-docker but nothing else.

 

So I’ve put together my own project which you can find here: https://gitlab.com/minusdavid/koha-deb-builder-docker

 

The “build.sh” script is still rough around the edges, but it all works. Basically you build the Docker image, dump a tarball into “input” (generated using git archive with a properly formatted prefix and filename), run a container, run the build.sh script, and then when the process is done you should have some Debian packages in your “output” directory which is shared with your host computer.

 

Happy to accept pull requests or talk about it in general. The Docker image is pretty heavy, but that’s because the Debian packaging tools are heavy.

 

The build process is also a bit long because it has to install all the “build dependencies”, although I think we could tweak the debian/control file so that we don’t have to install everything just to build the packages.

 

Also I couldn’t get the tests to complete successfully on the first go so I’ve turned them off for now. Fixing the image or the tests so that they can run would be cool.

 

Anyway, just thought I’d share that with everyone.

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 


_______________________________________________
Koha-devel mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

Re: Building Koha Debian packages in Docker

Tomas Cohen Arazi
David, try this


I basically maintain the base.tgz files outside of it. It works nicely. The source code is on my gitlab. The plan was to automate the packages building and eventually set a pipeline for also testing the install process of the results.

El 22 nov. 2018 04:59, "David Cook" <[hidden email]> escribió:

Hi folks,

 

I was wondering if anyone had a Docker image for building Koha Debian packages. I scouted around a bit and found this image from Jonathan https://gitlab.com/joubu/koha-dpkg-docker but nothing else.

 

So I’ve put together my own project which you can find here: https://gitlab.com/minusdavid/koha-deb-builder-docker

 

The “build.sh” script is still rough around the edges, but it all works. Basically you build the Docker image, dump a tarball into “input” (generated using git archive with a properly formatted prefix and filename), run a container, run the build.sh script, and then when the process is done you should have some Debian packages in your “output” directory which is shared with your host computer.

 

Happy to accept pull requests or talk about it in general. The Docker image is pretty heavy, but that’s because the Debian packaging tools are heavy.

 

The build process is also a bit long because it has to install all the “build dependencies”, although I think we could tweak the debian/control file so that we don’t have to install everything just to build the packages.

 

Also I couldn’t get the tests to complete successfully on the first go so I’ve turned them off for now. Fixing the image or the tests so that they can run would be cool.

 

Anyway, just thought I’d share that with everyone.

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 



_______________________________________________
Koha-devel mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

Re: Building Koha Debian packages in Docker

David Cook

Thanks for sharing that, Tomas. I thought that you’d already done some work on this, but it wasn’t jumping out at me the other day. Where’s it in your gitlab? Is it the koha-dpkg-docker repository?

 

I’m curious though why one would use pbuilder in Docker. Is it so that you can use the base.tgz files to store the koha-perldeps? That could be convenient.

 

I’m curious why we have every Koha Perl dependency in the build dependencies section. Is it for the sake of the tests? Surely Koha’s “build” dependencies would be pretty low. Just whatever is needed by rewrite-Config.PL really? But I suppose the tests would have a lot of dependencies, which could be painful to enumerate separately.  

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

From: Tomas Cohen Arazi [mailto:[hidden email]]
Sent: Thursday, 22 November 2018 8:38 PM
To: David Cook <[hidden email]>
Cc: koha-devel <[hidden email]>
Subject: Re: Building Koha Debian packages in Docker

 

David, try this

 

 

I basically maintain the base.tgz files outside of it. It works nicely. The source code is on my gitlab. The plan was to automate the packages building and eventually set a pipeline for also testing the install process of the results.

 

El 22 nov. 2018 04:59, "David Cook" <[hidden email]> escribió:

Hi folks,

 

I was wondering if anyone had a Docker image for building Koha Debian packages. I scouted around a bit and found this image from Jonathan https://gitlab.com/joubu/koha-dpkg-docker but nothing else.

 

So I’ve put together my own project which you can find here: https://gitlab.com/minusdavid/koha-deb-builder-docker

 

The “build.sh” script is still rough around the edges, but it all works. Basically you build the Docker image, dump a tarball into “input” (generated using git archive with a properly formatted prefix and filename), run a container, run the build.sh script, and then when the process is done you should have some Debian packages in your “output” directory which is shared with your host computer.

 

Happy to accept pull requests or talk about it in general. The Docker image is pretty heavy, but that’s because the Debian packaging tools are heavy.

 

The build process is also a bit long because it has to install all the “build dependencies”, although I think we could tweak the debian/control file so that we don’t have to install everything just to build the packages.

 

Also I couldn’t get the tests to complete successfully on the first go so I’ve turned them off for now. Fixing the image or the tests so that they can run would be cool.

 

Anyway, just thought I’d share that with everyone.

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

 


_______________________________________________
Koha-devel mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

Re: Building Koha Debian packages in Docker

Chris Cormack-7
More than just the tests. It also makes sure all the dependencies are packaged too. That way you can't build a package you can't install (because one of its depencies isn't packaged)

Chris

On 26 November 2018 8:19:36 PM NZDT, David Cook <[hidden email]> wrote:

Thanks for sharing that, Tomas. I thought that you’d already done some work on this, but it wasn’t jumping out at me the other day. Where’s it in your gitlab? Is it the koha-dpkg-docker repository?

 

I’m curious though why one would use pbuilder in Docker. Is it so that you can use the base.tgz files to store the koha-perldeps? That could be convenient.

 

I’m curious why we have every Koha Perl dependency in the build dependencies section. Is it for the sake of the tests? Surely Koha’s “build” dependencies would be pretty low. Just whatever is needed by rewrite-Config.PL really? But I suppose the tests would have a lot of dependencies, which could be painful to enumerate separately.  

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

From: Tomas Cohen Arazi [mailto:[hidden email]]
Sent: Thursday, 22 November 2018 8:38 PM
To: David Cook <[hidden email]>
Cc: koha-devel <[hidden email]>
Subject: Re: Building Koha Debian packages in Docker

 

David, try this

 

 

I basically maintain the base.tgz files outside of it. It works nicely. The source code is on my gitlab. The plan was to automate the packages building and eventually set a pipeline for also testing the install process of the results.

 

El 22 nov. 2018 04:59, "David Cook" <[hidden email]> escribió:

Hi folks,

 

I was wondering if anyone had a Docker image for building Koha Debian packages. I scouted around a bit and found this image from Jonathan https://gitlab.com/joubu/koha-dpkg-docker but nothing else.

 

So I’ve put together my own project which you can find here: https://gitlab.com/minusdavid/koha-deb-builder-docker

 

The “build.sh” script is still rough around the edges, but it all works. Basically you build the Docker image, dump a tarball into “input” (generated using git archive with a properly formatted prefix and filename), run a container, run the build.sh script, and then when the process is done you should have some Debian packages in your “output” directory which is shared with your host computer.

 

Happy to accept pull requests or talk about it in general. The Docker image is pretty heavy, but that’s because the Debian packaging tools are heavy.

 

The build process is also a bit long because it has to install all the “build dependencies”, although I think we could tweak the debian/control file so that we don’t have to install everything just to build the packages.

 

Also I couldn’t get the tests to complete successfully on the first go so I’ve turned them off for now. Fixing the image or the tests so that they can run would be cool.

 

Anyway, just thought I’d share that with everyone.

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

 


--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
_______________________________________________
Koha-devel mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

Re: Building Koha Debian packages in Docker

David Cook

While I can see the utility of that usage, are there any examples of other projects doing that? I don’t think that I’ve seen that elsewhere, and it doesn’t seem like the Debian way of doing things.

 

It seems to me that the Build-Depends should just be for build dependencies, and Depends should be for package dependencies. The lack of packaged dependencies could be caught in a different part of the pipeline. And when it is caught, then that dependency could be packaged, and the same packaged koha-common could be used. That seems to be a more standard way of handling the packaging process, no?

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Chris Cormack
Sent: Monday, 26 November 2018 6:51 PM
To: [hidden email]
Cc: 'koha-devel' <[hidden email]>
Subject: Re: [Koha-devel] Building Koha Debian packages in Docker

 

More than just the tests. It also makes sure all the dependencies are packaged too. That way you can't build a package you can't install (because one of its depencies isn't packaged)

Chris

On 26 November 2018 8:19:36 PM NZDT, David Cook <[hidden email]> wrote:

Thanks for sharing that, Tomas. I thought that you’d already done some work on this, but it wasn’t jumping out at me the other day. Where’s it in your gitlab? Is it the koha-dpkg-docker repository?

 

I’m curious though why one would use pbuilder in Docker. Is it so that you can use the base.tgz files to store the koha-perldeps? That could be convenient.

 

I’m curious why we have every Koha Perl dependency in the build dependencies section. Is it for the sake of the tests? Surely Koha’s “build” dependencies would be pretty low. Just whatever is needed by rewrite-Config.PL really? But I suppose the tests would have a lot of dependencies, which could be painful to enumerate separately.  

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

From: Tomas Cohen Arazi [[hidden email]]
Sent: Thursday, 22 November 2018 8:38 PM
To: David Cook <[hidden email]>
Cc: koha-devel <[hidden email]>
Subject: Re: Building Koha Debian packages in Docker

 

David, try this

 

 

I basically maintain the base.tgz files outside of it. It works nicely. The source code is on my gitlab. The plan was to automate the packages building and eventually set a pipeline for also testing the install process of the results.

 

El 22 nov. 2018 04:59, "David Cook" <[hidden email]> escribió:

Hi folks,

 

I was wondering if anyone had a Docker image for building Koha Debian packages. I scouted around a bit and found this image from Jonathan https://gitlab.com/joubu/koha-dpkg-docker but nothing else.

 

So I’ve put together my own project which you can find here: https://gitlab.com/minusdavid/koha-deb-builder-docker

 

The “build.sh” script is still rough around the edges, but it all works. Basically you build the Docker image, dump a tarball into “input” (generated using git archive with a properly formatted prefix and filename), run a container, run the build.sh script, and then when the process is done you should have some Debian packages in your “output” directory which is shared with your host computer.

 

Happy to accept pull requests or talk about it in general. The Docker image is pretty heavy, but that’s because the Debian packaging tools are heavy.

 

The build process is also a bit long because it has to install all the “build dependencies”, although I think we could tweak the debian/control file so that we don’t have to install everything just to build the packages.

 

Also I couldn’t get the tests to complete successfully on the first go so I’ve turned them off for now. Fixing the image or the tests so that they can run would be cool.

 

Anyway, just thought I’d share that with everyone.

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

 


--
Sent from my Android device with K-9 Mail. Please excuse my brevity.


_______________________________________________
Koha-devel mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

Re: Building Koha Debian packages in Docker

Chris Cormack-7
If you like, I note it's been working fine for over 8 years now.
And was packaged by 2 debian developers and a debian maintainer. I'd strongly resist changing it

Chris

On 27 November 2018 1:32:33 PM NZDT, David Cook <[hidden email]> wrote:

While I can see the utility of that usage, are there any examples of other projects doing that? I don’t think that I’ve seen that elsewhere, and it doesn’t seem like the Debian way of doing things.

 

It seems to me that the Build-Depends should just be for build dependencies, and Depends should be for package dependencies. The lack of packaged dependencies could be caught in a different part of the pipeline. And when it is caught, then that dependency could be packaged, and the same packaged koha-common could be used. That seems to be a more standard way of handling the packaging process, no?

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Chris Cormack
Sent: Monday, 26 November 2018 6:51 PM
To: [hidden email]
Cc: 'koha-devel' <[hidden email]>
Subject: Re: [Koha-devel] Building Koha Debian packages in Docker

 

More than just the tests. It also makes sure all the dependencies are packaged too. That way you can't build a package you can't install (because one of its depencies isn't packaged)

Chris

On 26 November 2018 8:19:36 PM NZDT, David Cook <[hidden email]> wrote:

Thanks for sharing that, Tomas. I thought that you’d already done some work on this, but it wasn’t jumping out at me the other day. Where’s it in your gitlab? Is it the koha-dpkg-docker repository?

 

I’m curious though why one would use pbuilder in Docker. Is it so that you can use the base.tgz files to store the koha-perldeps? That could be convenient.

 

I’m curious why we have every Koha Perl dependency in the build dependencies section. Is it for the sake of the tests? Surely Koha’s “build” dependencies would be pretty low. Just whatever is needed by rewrite-Config.PL really? But I suppose the tests would have a lot of dependencies, which could be painful to enumerate separately.  

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

From: Tomas Cohen Arazi [[hidden email]]
Sent: Thursday, 22 November 2018 8:38 PM
To: David Cook <[hidden email]>
Cc: koha-devel <[hidden email]>
Subject: Re: Building Koha Debian packages in Docker

 

David, try this

 

 

I basically maintain the base.tgz files outside of it. It works nicely. The source code is on my gitlab. The plan was to automate the packages building and eventually set a pipeline for also testing the install process of the results.

 

El 22 nov. 2018 04:59, "David Cook" <[hidden email]> escribió:

Hi folks,

 

I was wondering if anyone had a Docker image for building Koha Debian packages. I scouted around a bit and found this image from Jonathan https://gitlab.com/joubu/koha-dpkg-docker but nothing else.

 

So I’ve put together my own project which you can find here: https://gitlab.com/minusdavid/koha-deb-builder-docker

 

The “build.sh” script is still rough around the edges, but it all works. Basically you build the Docker image, dump a tarball into “input” (generated using git archive with a properly formatted prefix and filename), run a container, run the build.sh script, and then when the process is done you should have some Debian packages in your “output” directory which is shared with your host computer.

 

Happy to accept pull requests or talk about it in general. The Docker image is pretty heavy, but that’s because the Debian packaging tools are heavy.

 

The build process is also a bit long because it has to install all the “build dependencies”, although I think we could tweak the debian/control file so that we don’t have to install everything just to build the packages.

 

Also I couldn’t get the tests to complete successfully on the first go so I’ve turned them off for now. Fixing the image or the tests so that they can run would be cool.

 

Anyway, just thought I’d share that with everyone.

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

 


--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
_______________________________________________
Koha-devel mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

Re: Building Koha Debian packages in Docker

David Cook

There’s certainly something to be said for “if it’s not broken, don’t fix it”. It certainly does work well enough.

 

I seem to recall reading something years ago from Lars about some uncertainty on packaging full web applications, versus individual modules or DOTADIW programs, for Debian but I think that was about other details.  

 

I’m not really advocating for change. Really just asking questions about why things are done the way they’re done.

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

From: Chris Cormack [mailto:[hidden email]]
Sent: Tuesday, 27 November 2018 11:35 AM
To: David Cook <[hidden email]>; [hidden email]
Subject: RE: [Koha-devel] Building Koha Debian packages in Docker

 

If you like, I note it's been working fine for over 8 years now.
And was packaged by 2 debian developers and a debian maintainer. I'd strongly resist changing it

Chris

On 27 November 2018 1:32:33 PM NZDT, David Cook <[hidden email]> wrote:

While I can see the utility of that usage, are there any examples of other projects doing that? I don’t think that I’ve seen that elsewhere, and it doesn’t seem like the Debian way of doing things.

 

It seems to me that the Build-Depends should just be for build dependencies, and Depends should be for package dependencies. The lack of packaged dependencies could be caught in a different part of the pipeline. And when it is caught, then that dependency could be packaged, and the same packaged koha-common could be used. That seems to be a more standard way of handling the packaging process, no?

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

From: [hidden email] [[hidden email]] On Behalf Of Chris Cormack
Sent: Monday, 26 November 2018 6:51 PM
To: [hidden email]
Cc: 'koha-devel' <[hidden email]>
Subject: Re: [Koha-devel] Building Koha Debian packages in Docker

 

More than just the tests. It also makes sure all the dependencies are packaged too. That way you can't build a package you can't install (because one of its depencies isn't packaged)

Chris

On 26 November 2018 8:19:36 PM NZDT, David Cook <[hidden email]> wrote:

Thanks for sharing that, Tomas. I thought that you’d already done some work on this, but it wasn’t jumping out at me the other day. Where’s it in your gitlab? Is it the koha-dpkg-docker repository?

 

I’m curious though why one would use pbuilder in Docker. Is it so that you can use the base.tgz files to store the koha-perldeps? That could be convenient.

 

I’m curious why we have every Koha Perl dependency in the build dependencies section. Is it for the sake of the tests? Surely Koha’s “build” dependencies would be pretty low. Just whatever is needed by rewrite-Config.PL really? But I suppose the tests would have a lot of dependencies, which could be painful to enumerate separately.  

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

From: Tomas Cohen Arazi [[hidden email]]
Sent: Thursday, 22 November 2018 8:38 PM
To: David Cook <[hidden email]>
Cc: koha-devel <[hidden email]>
Subject: Re: Building Koha Debian packages in Docker

 

David, try this

 

 

I basically maintain the base.tgz files outside of it. It works nicely. The source code is on my gitlab. The plan was to automate the packages building and eventually set a pipeline for also testing the install process of the results.

 

El 22 nov. 2018 04:59, "David Cook" <[hidden email]> escribió:

Hi folks,

 

I was wondering if anyone had a Docker image for building Koha Debian packages. I scouted around a bit and found this image from Jonathan https://gitlab.com/joubu/koha-dpkg-docker but nothing else.

 

So I’ve put together my own project which you can find here: https://gitlab.com/minusdavid/koha-deb-builder-docker

 

The “build.sh” script is still rough around the edges, but it all works. Basically you build the Docker image, dump a tarball into “input” (generated using git archive with a properly formatted prefix and filename), run a container, run the build.sh script, and then when the process is done you should have some Debian packages in your “output” directory which is shared with your host computer.

 

Happy to accept pull requests or talk about it in general. The Docker image is pretty heavy, but that’s because the Debian packaging tools are heavy.

 

The build process is also a bit long because it has to install all the “build dependencies”, although I think we could tweak the debian/control file so that we don’t have to install everything just to build the packages.

 

Also I couldn’t get the tests to complete successfully on the first go so I’ve turned them off for now. Fixing the image or the tests so that they can run would be cool.

 

Anyway, just thought I’d share that with everyone.

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

 


--
Sent from my Android device with K-9 Mail. Please excuse my brevity.


_______________________________________________
Koha-devel mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

Re: Building Koha Debian packages in Docker

Tomas Cohen Arazi
If I had time to spend on the subject, I would look for ways to build Debian packages without the need to chroot. With that solved, it will be very easy to create those pipelines for testing the packages.

El lun., 26 nov. 2018 a las 21:55, David Cook (<[hidden email]>) escribió:

There’s certainly something to be said for “if it’s not broken, don’t fix it”. It certainly does work well enough.

 

I seem to recall reading something years ago from Lars about some uncertainty on packaging full web applications, versus individual modules or DOTADIW programs, for Debian but I think that was about other details.  

 

I’m not really advocating for change. Really just asking questions about why things are done the way they’re done.

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

From: Chris Cormack [mailto:[hidden email]]
Sent: Tuesday, 27 November 2018 11:35 AM
To: David Cook <[hidden email]>; [hidden email]
Subject: RE: [Koha-devel] Building Koha Debian packages in Docker

 

If you like, I note it's been working fine for over 8 years now.
And was packaged by 2 debian developers and a debian maintainer. I'd strongly resist changing it

Chris

On 27 November 2018 1:32:33 PM NZDT, David Cook <[hidden email]> wrote:

While I can see the utility of that usage, are there any examples of other projects doing that? I don’t think that I’ve seen that elsewhere, and it doesn’t seem like the Debian way of doing things.

 

It seems to me that the Build-Depends should just be for build dependencies, and Depends should be for package dependencies. The lack of packaged dependencies could be caught in a different part of the pipeline. And when it is caught, then that dependency could be packaged, and the same packaged koha-common could be used. That seems to be a more standard way of handling the packaging process, no?

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

From: [hidden email] [[hidden email]] On Behalf Of Chris Cormack
Sent: Monday, 26 November 2018 6:51 PM
To: [hidden email]
Cc: 'koha-devel' <[hidden email]>
Subject: Re: [Koha-devel] Building Koha Debian packages in Docker

 

More than just the tests. It also makes sure all the dependencies are packaged too. That way you can't build a package you can't install (because one of its depencies isn't packaged)

Chris

On 26 November 2018 8:19:36 PM NZDT, David Cook <[hidden email]> wrote:

Thanks for sharing that, Tomas. I thought that you’d already done some work on this, but it wasn’t jumping out at me the other day. Where’s it in your gitlab? Is it the koha-dpkg-docker repository?

 

I’m curious though why one would use pbuilder in Docker. Is it so that you can use the base.tgz files to store the koha-perldeps? That could be convenient.

 

I’m curious why we have every Koha Perl dependency in the build dependencies section. Is it for the sake of the tests? Surely Koha’s “build” dependencies would be pretty low. Just whatever is needed by rewrite-Config.PL really? But I suppose the tests would have a lot of dependencies, which could be painful to enumerate separately.  

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

From: Tomas Cohen Arazi [[hidden email]]
Sent: Thursday, 22 November 2018 8:38 PM
To: David Cook <[hidden email]>
Cc: koha-devel <[hidden email]>
Subject: Re: Building Koha Debian packages in Docker

 

David, try this

 

 

I basically maintain the base.tgz files outside of it. It works nicely. The source code is on my gitlab. The plan was to automate the packages building and eventually set a pipeline for also testing the install process of the results.

 

El 22 nov. 2018 04:59, "David Cook" <[hidden email]> escribió:

Hi folks,

 

I was wondering if anyone had a Docker image for building Koha Debian packages. I scouted around a bit and found this image from Jonathan https://gitlab.com/joubu/koha-dpkg-docker but nothing else.

 

So I’ve put together my own project which you can find here: https://gitlab.com/minusdavid/koha-deb-builder-docker

 

The “build.sh” script is still rough around the edges, but it all works. Basically you build the Docker image, dump a tarball into “input” (generated using git archive with a properly formatted prefix and filename), run a container, run the build.sh script, and then when the process is done you should have some Debian packages in your “output” directory which is shared with your host computer.

 

Happy to accept pull requests or talk about it in general. The Docker image is pretty heavy, but that’s because the Debian packaging tools are heavy.

 

The build process is also a bit long because it has to install all the “build dependencies”, although I think we could tweak the debian/control file so that we don’t have to install everything just to build the packages.

 

Also I couldn’t get the tests to complete successfully on the first go so I’ve turned them off for now. Fixing the image or the tests so that they can run would be cool.

 

Anyway, just thought I’d share that with everyone.

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

 


--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

_______________________________________________
Koha-devel mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


--
Tomás Cohen Arazi
Theke Solutions (http://theke.io)
✆ +54 9351 3513384
GPG: B2F3C15F

_______________________________________________
Koha-devel mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

Re: Building Koha Debian packages in Docker

David Cook

If you shared some of your ideas, Tomas, perhaps I could do some work on that one?

With https://gitlab.com/minusdavid/koha-deb-builder-docker, I’m not using a chroot. I just use the debian:jessie parent image, install build tools, take the upstream tarball (generated via git archive), use dch to update the changelog, currently use apt-get -y build-dep koha to install the build dependencies, then run “debuild”. The slowest bit of the process is installing those build dependencies.

 

(Full disclosure: I’m using DEB_BUILD_OPTIONS=nocheck as I recall one of the tests causing a fatal error, and I put off analysing that until I had the build process nailed down in the docker. That’ll need revisiting.)

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

From: Tomas Cohen Arazi [mailto:[hidden email]]
Sent: Tuesday, 27 November 2018 11:57 AM
To: David Cook <[hidden email]>
Cc: Chris Cormack <[hidden email]>; koha-devel <[hidden email]>
Subject: Re: [Koha-devel] Building Koha Debian packages in Docker

 

If I had time to spend on the subject, I would look for ways to build Debian packages without the need to chroot. With that solved, it will be very easy to create those pipelines for testing the packages.

 

El lun., 26 nov. 2018 a las 21:55, David Cook (<[hidden email]>) escribió:

There’s certainly something to be said for “if it’s not broken, don’t fix it”. It certainly does work well enough.

 

I seem to recall reading something years ago from Lars about some uncertainty on packaging full web applications, versus individual modules or DOTADIW programs, for Debian but I think that was about other details.  

 

I’m not really advocating for change. Really just asking questions about why things are done the way they’re done.

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

From: Chris Cormack [mailto:[hidden email]]
Sent: Tuesday, 27 November 2018 11:35 AM
To: David Cook <[hidden email]>; [hidden email]
Subject: RE: [Koha-devel] Building Koha Debian packages in Docker

 

If you like, I note it's been working fine for over 8 years now.
And was packaged by 2 debian developers and a debian maintainer. I'd strongly resist changing it

Chris

On 27 November 2018 1:32:33 PM NZDT, David Cook <[hidden email]> wrote:

While I can see the utility of that usage, are there any examples of other projects doing that? I don’t think that I’ve seen that elsewhere, and it doesn’t seem like the Debian way of doing things.

 

It seems to me that the Build-Depends should just be for build dependencies, and Depends should be for package dependencies. The lack of packaged dependencies could be caught in a different part of the pipeline. And when it is caught, then that dependency could be packaged, and the same packaged koha-common could be used. That seems to be a more standard way of handling the packaging process, no?

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

From: [hidden email] [[hidden email]] On Behalf Of Chris Cormack
Sent: Monday, 26 November 2018 6:51 PM
To: [hidden email]
Cc: 'koha-devel' <[hidden email]>
Subject: Re: [Koha-devel] Building Koha Debian packages in Docker

 

More than just the tests. It also makes sure all the dependencies are packaged too. That way you can't build a package you can't install (because one of its depencies isn't packaged)

Chris

On 26 November 2018 8:19:36 PM NZDT, David Cook <[hidden email]> wrote:

Thanks for sharing that, Tomas. I thought that you’d already done some work on this, but it wasn’t jumping out at me the other day. Where’s it in your gitlab? Is it the koha-dpkg-docker repository?

 

I’m curious though why one would use pbuilder in Docker. Is it so that you can use the base.tgz files to store the koha-perldeps? That could be convenient.

 

I’m curious why we have every Koha Perl dependency in the build dependencies section. Is it for the sake of the tests? Surely Koha’s “build” dependencies would be pretty low. Just whatever is needed by rewrite-Config.PL really? But I suppose the tests would have a lot of dependencies, which could be painful to enumerate separately.  

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

From: Tomas Cohen Arazi [[hidden email]]
Sent: Thursday, 22 November 2018 8:38 PM
To: David Cook <[hidden email]>
Cc: koha-devel <[hidden email]>
Subject: Re: Building Koha Debian packages in Docker

 

David, try this

 

 

I basically maintain the base.tgz files outside of it. It works nicely. The source code is on my gitlab. The plan was to automate the packages building and eventually set a pipeline for also testing the install process of the results.

 

El 22 nov. 2018 04:59, "David Cook" <[hidden email]> escribió:

Hi folks,

 

I was wondering if anyone had a Docker image for building Koha Debian packages. I scouted around a bit and found this image from Jonathan https://gitlab.com/joubu/koha-dpkg-docker but nothing else.

 

So I’ve put together my own project which you can find here: https://gitlab.com/minusdavid/koha-deb-builder-docker

 

The “build.sh” script is still rough around the edges, but it all works. Basically you build the Docker image, dump a tarball into “input” (generated using git archive with a properly formatted prefix and filename), run a container, run the build.sh script, and then when the process is done you should have some Debian packages in your “output” directory which is shared with your host computer.

 

Happy to accept pull requests or talk about it in general. The Docker image is pretty heavy, but that’s because the Debian packaging tools are heavy.

 

The build process is also a bit long because it has to install all the “build dependencies”, although I think we could tweak the debian/control file so that we don’t have to install everything just to build the packages.

 

Also I couldn’t get the tests to complete successfully on the first go so I’ve turned them off for now. Fixing the image or the tests so that they can run would be cool.

 

Anyway, just thought I’d share that with everyone.

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

 


--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

_______________________________________________
Koha-devel mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


 

--

Tomás Cohen Arazi

Theke Solutions (http://theke.io)
+54 9351 3513384
GPG: B2F3C15F


_______________________________________________
Koha-devel mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

Re: Building Koha Debian packages in Docker

Tomas Cohen Arazi
It's all here:


I thought it was documented on the Docker hub.

El lun., 26 nov. 2018 a las 22:10, David Cook (<[hidden email]>) escribió:

If you shared some of your ideas, Tomas, perhaps I could do some work on that one?

With https://gitlab.com/minusdavid/koha-deb-builder-docker, I’m not using a chroot. I just use the debian:jessie parent image, install build tools, take the upstream tarball (generated via git archive), use dch to update the changelog, currently use apt-get -y build-dep koha to install the build dependencies, then run “debuild”. The slowest bit of the process is installing those build dependencies.

 

(Full disclosure: I’m using DEB_BUILD_OPTIONS=nocheck as I recall one of the tests causing a fatal error, and I put off analysing that until I had the build process nailed down in the docker. That’ll need revisiting.)

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

From: Tomas Cohen Arazi [mailto:[hidden email]]
Sent: Tuesday, 27 November 2018 11:57 AM
To: David Cook <[hidden email]>
Cc: Chris Cormack <[hidden email]>; koha-devel <[hidden email]>
Subject: Re: [Koha-devel] Building Koha Debian packages in Docker

 

If I had time to spend on the subject, I would look for ways to build Debian packages without the need to chroot. With that solved, it will be very easy to create those pipelines for testing the packages.

 

El lun., 26 nov. 2018 a las 21:55, David Cook (<[hidden email]>) escribió:

There’s certainly something to be said for “if it’s not broken, don’t fix it”. It certainly does work well enough.

 

I seem to recall reading something years ago from Lars about some uncertainty on packaging full web applications, versus individual modules or DOTADIW programs, for Debian but I think that was about other details.  

 

I’m not really advocating for change. Really just asking questions about why things are done the way they’re done.

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

From: Chris Cormack [mailto:[hidden email]]
Sent: Tuesday, 27 November 2018 11:35 AM
To: David Cook <[hidden email]>; [hidden email]
Subject: RE: [Koha-devel] Building Koha Debian packages in Docker

 

If you like, I note it's been working fine for over 8 years now.
And was packaged by 2 debian developers and a debian maintainer. I'd strongly resist changing it

Chris

On 27 November 2018 1:32:33 PM NZDT, David Cook <[hidden email]> wrote:

While I can see the utility of that usage, are there any examples of other projects doing that? I don’t think that I’ve seen that elsewhere, and it doesn’t seem like the Debian way of doing things.

 

It seems to me that the Build-Depends should just be for build dependencies, and Depends should be for package dependencies. The lack of packaged dependencies could be caught in a different part of the pipeline. And when it is caught, then that dependency could be packaged, and the same packaged koha-common could be used. That seems to be a more standard way of handling the packaging process, no?

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

From: [hidden email] [[hidden email]] On Behalf Of Chris Cormack
Sent: Monday, 26 November 2018 6:51 PM
To: [hidden email]
Cc: 'koha-devel' <[hidden email]>
Subject: Re: [Koha-devel] Building Koha Debian packages in Docker

 

More than just the tests. It also makes sure all the dependencies are packaged too. That way you can't build a package you can't install (because one of its depencies isn't packaged)

Chris

On 26 November 2018 8:19:36 PM NZDT, David Cook <[hidden email]> wrote:

Thanks for sharing that, Tomas. I thought that you’d already done some work on this, but it wasn’t jumping out at me the other day. Where’s it in your gitlab? Is it the koha-dpkg-docker repository?

 

I’m curious though why one would use pbuilder in Docker. Is it so that you can use the base.tgz files to store the koha-perldeps? That could be convenient.

 

I’m curious why we have every Koha Perl dependency in the build dependencies section. Is it for the sake of the tests? Surely Koha’s “build” dependencies would be pretty low. Just whatever is needed by rewrite-Config.PL really? But I suppose the tests would have a lot of dependencies, which could be painful to enumerate separately.  

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

From: Tomas Cohen Arazi [[hidden email]]
Sent: Thursday, 22 November 2018 8:38 PM
To: David Cook <[hidden email]>
Cc: koha-devel <[hidden email]>
Subject: Re: Building Koha Debian packages in Docker

 

David, try this

 

 

I basically maintain the base.tgz files outside of it. It works nicely. The source code is on my gitlab. The plan was to automate the packages building and eventually set a pipeline for also testing the install process of the results.

 

El 22 nov. 2018 04:59, "David Cook" <[hidden email]> escribió:

Hi folks,

 

I was wondering if anyone had a Docker image for building Koha Debian packages. I scouted around a bit and found this image from Jonathan https://gitlab.com/joubu/koha-dpkg-docker but nothing else.

 

So I’ve put together my own project which you can find here: https://gitlab.com/minusdavid/koha-deb-builder-docker

 

The “build.sh” script is still rough around the edges, but it all works. Basically you build the Docker image, dump a tarball into “input” (generated using git archive with a properly formatted prefix and filename), run a container, run the build.sh script, and then when the process is done you should have some Debian packages in your “output” directory which is shared with your host computer.

 

Happy to accept pull requests or talk about it in general. The Docker image is pretty heavy, but that’s because the Debian packaging tools are heavy.

 

The build process is also a bit long because it has to install all the “build dependencies”, although I think we could tweak the debian/control file so that we don’t have to install everything just to build the packages.

 

Also I couldn’t get the tests to complete successfully on the first go so I’ve turned them off for now. Fixing the image or the tests so that they can run would be cool.

 

Anyway, just thought I’d share that with everyone.

 

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

 

Office: 02 9212 0899

Direct: 02 8005 0595

 

 


--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

_______________________________________________
Koha-devel mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


 

--

Tomás Cohen Arazi

Theke Solutions (http://theke.io)
+54 9351 3513384
GPG: B2F3C15F



--
Tomás Cohen Arazi
Theke Solutions (http://theke.io)
✆ +54 9351 3513384
GPG: B2F3C15F

_______________________________________________
Koha-devel mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/