From allan at archlinux.org  Mon Dec  1 23:19:32 2008
From: allan at archlinux.org (Allan McRae)
Date: Tue,  2 Dec 2008 14:19:32 +1000
Subject: [pacman-dev] [PATCH] Add optdepends to PKGBUILD.proto
Message-ID: <1228191572-707-1-git-send-email-allan@archlinux.org>

Signed-off-by: Allan McRae <allan at archlinux.org>
---
 PKGBUILD.proto |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/PKGBUILD.proto b/PKGBUILD.proto
index 3265530..eddcf75 100644
--- a/PKGBUILD.proto
+++ b/PKGBUILD.proto
@@ -14,6 +14,7 @@ license=('GPL')
 groups=()
 depends=()
 makedepends=()
+optdepends=()
 provides=()
 conflicts=()
 replaces=()
-- 
1.6.0.4



From gerbra at archlinux.de  Tue Dec  2 07:12:02 2008
From: gerbra at archlinux.de (Gerhard Brauer)
Date: Tue, 2 Dec 2008 13:12:02 +0100
Subject: [pacman-dev] Package/database signing
Message-ID: <20081202131202.63e927b7@tux1.brauer.lan>

(I hope i don't triple-post this mail, sorry)

Hi,
i took this mail from arch-dev-public and arch-general discussion about
"Can we trust our mirrors"


Am Mon, 1 Dec 2008 11:06:02 -0600
schrieb "Aaron Griffin" <aaronmgriffin at gmail.com>:
> 
> There's too much talk on this idea. Before we go ahead and do this,
> could someone submit this patch to the pacman-dev list, so the pacman
> developers can give it a once-over. Just make sure to let them know
> that this is a temporary solution.  

I attach my patch to repo-add and a pacman download script to explain
how things *could* be done. A proof of concept, working and tested on
my local repo.

> Additionally - where will gpg get the key from on gerolde? Shouldn't
> this be configurable, or even set via an optarg to the -s param?  

In my environment the signing key is getting from my sec-keyring, cause
i start repo-add under my UID. It's the default key in gpg.conf if you
have more than one key. And of cource: on gpg command line the key could
be selected by several criteria, also from a special keyring. Keyring
or keys must kept secure of cource, so mostly the are in $HOME/.gpg

This is only a concept for signing the package db for the repos. These
database files are created only on gerolde (?) an should be signed only
there and from the process which creates the database. This is repo-add.

#Q: Which key, where does it come from, how to prevent this key(s)?
I would assume only one secret key for database signing. AFAIK from
Pierre repo-add is under normal procedures not called directly by the
developers, only from upload/build scripts.
So one solution maybe: create a dedicated user. repo-add then is
running *only* under this UID. The secret key could be in the $HOME of
this user, so it's secured from other access. Only god mode users on
gerolde could "see" it, and maintain it (revoke, renew, etc.).
In the scripts which called repo-add (or if single users must call
repo-add directly) we could call it with su or sudo. Without special
key selecting then the key is took from $repo-add-userid's keyring and
cause it's the only keys it is automatically the default key.

#Q: How to implement this in pacman on users side?
In my test i use the pacman.conf option XferCommand and a special
download script for. I call it like this:
XferCommand = /tmp/pacsigdl %o %u
Each download is checked if this is a *.db.tar,gz. If yes, the file is
verified with the public key part from above. If verification is ok
then pacman continues with this db file as usual, if verification
failed pacman ignores the downloaded (corrupted) file and signals this
for this special repo db (but continues with the rest of a -Syu). No
new database for a repo -> no new packages to download/install.

That way with XferCommand is not a real solution:
a) It makes pacman intern downloading (libalpm) obsolete.
b) Downloads for *all** packages(incl. database files) must be handled
with wget or curl, which produces verbose or zero output which breaks
our normal pacman output.
c) The verifying part should not be delegated to the user. Ideally
pacman would handle this process itself. (I'm nor a c programmer).

#Q: Must we real use gpg command line? Must we use gpg?
My experience with other Trust-Solutions are low. Also it seems that
there are no good libraries to gpg for accessing functions from c
programs. http://www.gnupg.org/related_software/libraries.en.html
Thomas(brain0) mentions for ex. gnutls or openssl
If we could integrate gpg (command-line or by lib) better in our
framework then i think: gpg is a good, well known solution.

#Q: Signing only the database is not secure!
Yes, it would currently only secure our package hash/checksum functions
against manipulations, not the packages itself.
But IMHO it's a good starting point to get forward. Ideally we must sign
the db files also when we sometimes have a super dupper package signing
framework ;-) So let's start (and gain experience) fist with the db
files.
Currently our check-sums(md5) are only download check-sums. They could
be manipulate (and therefor easily the corresponding package) on each
station after the db leaves gerolde. From a secure view the checksums
are useless.
If we sign our db file we secure the checksums against manipulating.
Not the packages itself. But package manipulating is a lot more
difficult to do as in our current situation.
And we could switch to a better package check summing solution than
md5sum to gain more secure against package manipulation. Ideally we
come some day to a framework were also the packages are signed by
packagers/developers. Some days. Next step.

That are my thoughts on this thing. Surely not complete.
(Sorry for my not good English ;-) I attach also a tar archive with a) a
repo-add patch and b) a download script for using in pacman.conf)

Regards
	Gerhard
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pacman-sig.tar.gz
Type: application/x-gzip
Size: 1609 bytes
Desc: not available
URL: <http://archlinux.org/pipermail/pacman-dev/attachments/20081202/aa8f66df/attachment.bin>

From dpmcgee at gmail.com  Tue Dec  2 21:35:16 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Tue, 2 Dec 2008 20:35:16 -0600
Subject: [pacman-dev] [PATCH] Add optdepends to PKGBUILD.proto
In-Reply-To: <1228191572-707-1-git-send-email-allan@archlinux.org>
References: <1228191572-707-1-git-send-email-allan@archlinux.org>
Message-ID: <449c10960812021835q34d854fasd283fa61d7295167@mail.gmail.com>

On Mon, Dec 1, 2008 at 10:19 PM, Allan McRae <allan at archlinux.org> wrote:
> Signed-off-by: Allan McRae <allan at archlinux.org>

Added to maint branch, thanks.

-Dan


From dan at archlinux.org  Tue Dec  2 23:15:54 2008
From: dan at archlinux.org (Dan McGee)
Date: Tue,  2 Dec 2008 23:15:54 -0500 (EST)
Subject: [pacman-dev] [GIT] The official pacman repository branch, maint,
	updated. v3.2.1-22-ga1f7c83
Message-ID: <20081203041554.BEB8B2A237@gerolde.archlinux.org>

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "The official pacman repository".

The branch, maint has been updated
       via  a1f7c83dbf3bce492163362d2896e3a4176be616 (commit)
       via  6d8a6aef094c162f46a8b9be6a118a502fabca61 (commit)
       via  b99bebc008dcf944a88f99bb44ac9029557e4149 (commit)
      from  a50b067470a8046dabdff66f6266d2208b2f8372 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
-----------------------------------------------------------------------

Summary of changes:
 PKGBUILD.proto         |    1 +
 lib/libalpm/be_files.c |    5 ++++-
 lib/libalpm/delta.c    |   15 +++++++++++++++
 src/pacman/callback.c  |    1 +
 4 files changed, 21 insertions(+), 1 deletions(-)


hooks/post-receive
--
The official pacman repository


From dan at archlinux.org  Tue Dec  2 23:15:54 2008
From: dan at archlinux.org (Dan McGee)
Date: Tue,  2 Dec 2008 23:15:54 -0500 (EST)
Subject: [pacman-dev] [GIT] The official pacman repository branch, master,
	updated. v3.2.1-52-g61c6552
Message-ID: <20081203041554.D75282A238@gerolde.archlinux.org>

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "The official pacman repository".

The branch, master has been updated
       via  61c6552862345cb155903cd1566f1cef5c527a94 (commit)
       via  a1f7c83dbf3bce492163362d2896e3a4176be616 (commit)
       via  6d8a6aef094c162f46a8b9be6a118a502fabca61 (commit)
       via  b99bebc008dcf944a88f99bb44ac9029557e4149 (commit)
       via  a50b067470a8046dabdff66f6266d2208b2f8372 (commit)
       via  346139298bc547d1cbeb5d41f2067577b96ff1fa (commit)
       via  f7192b595881103f145a118d63d1b342ffd740b4 (commit)
      from  9394f229a0328228e810b7d4588b24643b42df6a (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
commit 61c6552862345cb155903cd1566f1cef5c527a94
Merge: 9394f229a0328228e810b7d4588b24643b42df6a a1f7c83dbf3bce492163362d2896e3a4176be616
Author: Dan McGee <dan at archlinux.org>
Date:   Tue Dec 2 22:15:02 2008 -0600

    Merge branch 'maint'
    
    Conflicts:
    	lib/libalpm/dload.c

-----------------------------------------------------------------------

Summary of changes:
 PKGBUILD.proto         |    1 +
 lib/libalpm/be_files.c |   11 +++++++----
 lib/libalpm/delta.c    |   15 +++++++++++++++
 lib/libalpm/dload.c    |   10 ++++------
 scripts/makepkg.sh.in  |    2 +-
 src/pacman/callback.c  |    1 +
 src/pacman/pacman.c    |   46 ++++++++++++++++++++++++++++++++++------------
 src/pacman/sync.c      |    3 ++-
 8 files changed, 65 insertions(+), 24 deletions(-)


hooks/post-receive
--
The official pacman repository


From allan at archlinux.org  Thu Dec  4 08:23:56 2008
From: allan at archlinux.org (Allan McRae)
Date: Thu,  4 Dec 2008 23:23:56 +1000
Subject: [pacman-dev] [PATCH] contirb/pactree: fix option parsing
Message-ID: <1228397036-6174-1-git-send-email-allan@archlinux.org>

The option parsing was catching any "-d" in an arguement so packages
with this in their name did not work.

Also removed commented code line that appears to be inserted during
testing

Signed-off-by: Allan McRae <allan at archlinux.org>
---
 contrib/pactree |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)

diff --git a/contrib/pactree b/contrib/pactree
index d9fa8b3..df53671 100755
--- a/contrib/pactree
+++ b/contrib/pactree
@@ -208,11 +208,10 @@ for (( n=0 ; n < $len_options ; n++ )); do
 		continue
 	fi
 
-	if [[ "${options[$n]}" =~ -d[[:digit:]]* || "${options[$n]}" == "--depth" ]]; then
+	if [[ "${options[$n]}" =~ -d[[:digit:]]+ || "${options[$n]}" == "--depth" ]]; then
 		if [[ "${options[$n]#-d}" =~ [[:digit:]]+ ]]; then
 			max_depth="${options[$n]#-d}"
 		elif [[ ${options[$((n+1))]} =~ [[:digit:]]+ ]]; then
-#		if [ ${options[$((n+1))]} -eq ${options[$((n+1))]} 2>/dev/null ]; then
 			max_depth="${options[$((n+1))]}"
 			unset options[$((n+1))]
 			((++n))
-- 
1.6.0.4


From dpmcgee at gmail.com  Thu Dec  4 09:11:17 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Thu, 4 Dec 2008 08:11:17 -0600
Subject: [pacman-dev] [PATCH] contirb/pactree: fix option parsing
In-Reply-To: <1228397036-6174-1-git-send-email-allan@archlinux.org>
References: <1228397036-6174-1-git-send-email-allan@archlinux.org>
Message-ID: <449c10960812040611vc7888a8odb447dbc3865ce7f@mail.gmail.com>

On Thu, Dec 4, 2008 at 7:23 AM, Allan McRae <allan at archlinux.org> wrote:
> The option parsing was catching any "-d" in an arguement so packages
> with this in their name did not work.
>
> Also removed commented code line that appears to be inserted during
> testing
>
> Signed-off-by: Allan McRae <allan at archlinux.org>
> ---

Thanks. Applied to my maint branch.

From gerbra at archlinux.de  Thu Dec  4 09:43:08 2008
From: gerbra at archlinux.de (Gerhard Brauer)
Date: Thu, 4 Dec 2008 15:43:08 +0100
Subject: [pacman-dev] Dan's pacman tree build&test
Message-ID: <20081204154308.11a350d8@tux1.brauer.lan>

Hi Dan,

git clone http://code.toofishes.net/gitprojects/pacman.git
(branch is master)

1. if have to install asciidox (for a2x), that's current not a
(build)depend of pacman

make stops with an error

make[2]: Entering directory `/home/gerhard/al-de/pac-git/pacman/doc'
a2x -d manpage -f manpage --xsltproc-opts='-param man.endnotes.list.enabled 0' --xsltproc-opts='-param man.en
dnotes.are.numbered 0' --asciidoc-opts="-f asciidoc.conf -a pacman_version="3.2.1" -a pacman_date="`date +%Y-
%m-%d`" -a sysconfdir=/etc" PKGBUILD.5.txt
./PKGBUILD.5.xml:148: element literal: validity error : Element emphasis is not declared in literal list of p
ossible children
e. The syntax is: <literal>source=(<emphasis>filename::url</emphasis>)</literal>
                                                                               ^
./PKGBUILD.5.xml:780: element programlisting: validity error : No declaration for attribute language of eleme
nt programlisting
<programlisting language="sh" linenumbering="unnumbered"># Maintainer: Joe User 
                                                        ^
./PKGBUILD.5.xml:780: parser error : error parsing attribute name
isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user
                                                                               ^
./PKGBUILD.5.xml:780: parser error : attributes construct error
isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user
                                                                               ^
./PKGBUILD.5.xml:780: parser error : Couldn't find end of Start Tag joe.user line 780
isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user
                                                                               ^
a2x: failed: xmllint --nonet --noout --valid "./PKGBUILD.5.xml"
make[2]: *** [PKGBUILD.5] Fehler 1
make[2]: Leaving directory `/home/gerhard/al-de/pac-git/pacman/doc'
make[1]: *** [all-recursive] Fehler 1
make[1]: Leaving directory `/home/gerhard/al-de/pac-git/pacman'
make: *** [all] Fehler 2

Seems any error in XML-Input doc file.

Is it ok to post to this ML? Or better a private mail address?

Gerhard

From allan at archlinux.org  Thu Dec  4 09:53:48 2008
From: allan at archlinux.org (Allan McRae)
Date: Fri, 05 Dec 2008 00:53:48 +1000
Subject: [pacman-dev] Dan's pacman tree build&test
In-Reply-To: <20081204154308.11a350d8@tux1.brauer.lan>
References: <20081204154308.11a350d8@tux1.brauer.lan>
Message-ID: <BAY102-DAV132B047E446B38D9D9A1A686020@phx.gbl>

Gerhard Brauer wrote:
> Hi Dan,
>
> git clone http://code.toofishes.net/gitprojects/pacman.git
> (branch is master)
>
> 1. if have to install asciidox (for a2x), that's current not a
> (build)depend of pacman
>
> make stops with an error
>
> make[2]: Entering directory `/home/gerhard/al-de/pac-git/pacman/doc'
> a2x -d manpage -f manpage --xsltproc-opts='-param man.endnotes.list.enabled 0' --xsltproc-opts='-param man.en
> dnotes.are.numbered 0' --asciidoc-opts="-f asciidoc.conf -a pacman_version="3.2.1" -a pacman_date="`date +%Y-
> %m-%d`" -a sysconfdir=/etc" PKGBUILD.5.txt
> ./PKGBUILD.5.xml:148: element literal: validity error : Element emphasis is not declared in literal list of p
> ossible children
> e. The syntax is: <literal>source=(<emphasis>filename::url</emphasis>)</literal>
>                                                                                ^
> ./PKGBUILD.5.xml:780: element programlisting: validity error : No declaration for attribute language of eleme
> nt programlisting
> <programlisting language="sh" linenumbering="unnumbered"># Maintainer: Joe User 
>                                                         ^
> ./PKGBUILD.5.xml:780: parser error : error parsing attribute name
> isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user
>                                                                                ^
> ./PKGBUILD.5.xml:780: parser error : attributes construct error
> isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user
>                                                                                ^
> ./PKGBUILD.5.xml:780: parser error : Couldn't find end of Start Tag joe.user line 780
> isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user
>                                                                                ^
> a2x: failed: xmllint --nonet --noout --valid "./PKGBUILD.5.xml"
> make[2]: *** [PKGBUILD.5] Fehler 1
> make[2]: Leaving directory `/home/gerhard/al-de/pac-git/pacman/doc'
> make[1]: *** [all-recursive] Fehler 1
> make[1]: Leaving directory `/home/gerhard/al-de/pac-git/pacman'
> make: *** [all] Fehler 2
>
> Seems any error in XML-Input doc file.
>
> Is it ok to post to this ML? Or better a private mail address?
>
> Gerhard
>   

Not a specific answer to your question but I always just do:

./autogen.sh
./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var 
--disable-doc
make

which avoids building docs altogether. 

Allan



From gerbra at archlinux.de  Thu Dec  4 13:44:05 2008
From: gerbra at archlinux.de (Gerhard Brauer)
Date: Thu, 4 Dec 2008 19:44:05 +0100
Subject: [pacman-dev] Dan's pacman tree build&test
In-Reply-To: <20081204154308.11a350d8@tux1.brauer.lan>
References: <20081204154308.11a350d8@tux1.brauer.lan>
Message-ID: <20081204194405.22c27e6d@tux1.brauer.lan>

Ok, have tested the package signing feature from Dan's pacman git.
(Thanks Allan for the hint with --disable-doc)

I test with the abook package from extra.

1)
makepkg
==> Finished making: abook 0.5.6-2 i686 (Thu Dec  4 15:52:44 UTC 2008)
==> Signing package...
==> ERROR: Cannot find the gpg binary! Is gnupg installed?

That's right, it is a fresh VM ;-)

2)
makepkg
==> Finished making: abook 0.5.6-2 i686 (Thu Dec  4 15:55:34 UTC 2008)
==> Signing package...
gpg: directory `/root/.gnupg' created
gpg: new configuration file `/root/.gnupg/gpg.conf' created
gpg: WARNING: options in `/root/.gnupg/gpg.conf' are not yet active during this run
gpg: keyring `/root/.gnupg/secring.gpg' created
gpg: keyring `/root/.gnupg/pubring.gpg' created
gpg: no default secret key: secret key not available
gpg: signing failed: secret key not available
==> WARNING: Failed to sign package file.

That's right. I still have no gpg key.
After setting up all gpg things makepkg builds and signs the package.

3)
Add a repo: mypkg
repo-add ad the abook package and puts also the %PGPSIG% field in the desc file.

4)
pacman -S mypkg/abook
checking package integrity...
warning: gpg cmdline: gpg --verify --no-default-keyring --keyserver-options no-auto-key-retrieve --keyring /tmp/testing.gpg - /var/cache/pacman/pkg/abook-0.5.6-2-i686.pkg.tar.gz
error: failed to commit transaction (invalid or corrupted package)
abook-0.5.6-2-i686.pkg.tar.gz is invalid or corrupted
Errors occurred, no packages were upgraded.

Ok, i have not imported the public key to root's keyring.

5)
[root at archtest ~]# LANG=C pacman -S mypkg/abook
resolving dependencies...
looking for inter-conflicts...

Targets (1): abook-0.5.6-2  

Total Download Size:    0.00 MB
Total Installed Size:   0.20 MB

Proceed with installation? [Y/n] 
checking package integrity...
warning: gpg cmdline: gpg --verify --no-default-keyring --keyserver-options no-auto-key-retrieve --keyring /tmp/testing.gpg - /var/cache/pacman/pkg/abook-0.5.6-2-i686.pkg.tar.gz
(1/1) checking for file conflicts                   [#####################] 100%
(1/1) installing abook                              [#####################] 100%

Problem/Question:
Where could i define the public keyring location?
According to commit: "Add keyring location as option on libalpm handle" the is a libalpm option
--keyring. But i have no plan where to define it (in pacman.conf i got an error).
I copied my keyring temporary to /tmp/testing.gpg what seems the default search path and
filename. Doing this i could install above abook from my repo.

6)
[root at archtest ~]# LANG=C pacman -Sy mypkg/abook
:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
 mypkg is up to date
warning: abook-0.5.6-2 is up to date -- reinstalling
resolving dependencies...
looking for inter-conflicts...

Targets (1): abook-0.5.6-2  

Total Download Size:    0.05 MB
Total Installed Size:   0.20 MB

Proceed with installation? [Y/n] 
:: Retrieving packages from mypkg...
 abook-0.5.6-2-i686        49.6K   20.9M/s 00:00:00 [#####################] 100%
checking package integrity...
warning: gpg cmdline: gpg --verify --no-default-keyring --keyserver-options no-a
uto-key-retrieve --keyring /tmp/testing.gpg - /var/cache/pacman/pkg/abook-0.5.6-
2-i686.pkg.tar.gz
error: failed to commit transaction (invalid or corrupted package)
abook-0.5.6-2-i686.pkg.tar.gz is invalid or corrupted
Errors occurred, no packages were upgraded.

Here if have modified the abook-0.5.6-2-i686.pkg.tar.gz package, copied to my repo,
do a repo-add but use the old *.sig signature. This modified package gets not
installed.
Maybe the error/reason could be more explained.

Summary:
I think most of the signing part (makepkg, repo-add) and the verifying
part (pacman) works so far. Awesome!
gpg verifying is good integrated in pacman, the "warning: gpg cmdline"
line thing i assume is a test/debug thing.

Next step could be: verifying the database files during pacman -Sy ?


Regards
	Gerhard

From linuxmania at gmail.com  Thu Dec  4 18:32:04 2008
From: linuxmania at gmail.com (Giovanni Scafora)
Date: Thu, 4 Dec 2008 15:32:04 -0800
Subject: [pacman-dev] [translation] Update Italian translation
Message-ID: <c357d3ef0812041532i60e786f0m1df0b0116491ee51@mail.gmail.com>

Hi guys,

there is an update Italian translation of pacman.
Thanks.


-- 
Arch Linux Developer (voidnull)
AUR & Pacman Italian Translations
Microdia Developer
http://www.archlinux.it
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pacman_IT.tar.gz
Type: application/x-gzip
Size: 12947 bytes
Desc: not available
URL: <http://archlinux.org/pipermail/pacman-dev/attachments/20081204/946ef665/attachment-0001.bin>

From dan at archlinux.org  Thu Dec  4 21:48:50 2008
From: dan at archlinux.org (Dan McGee)
Date: Thu,  4 Dec 2008 20:48:50 -0600
Subject: [pacman-dev] [PATCH] makepkg: save and restore shell options before
	and after build()
Message-ID: <1228445330-21769-1-git-send-email-dan@archlinux.org>

Fix the issue uncovered by FS#12344. In this instance, the dotglob shopt was
being set in the build() function but never cleared, causing issues in the
remaining parts of the makepkg script.

Signed-off-by: Dan McGee <dan at archlinux.org>
---
 scripts/makepkg.sh.in |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in
index 4cc255c..25e4cc8 100644
--- a/scripts/makepkg.sh.in
+++ b/scripts/makepkg.sh.in
@@ -675,6 +675,8 @@ run_build() {
 
 	# ensure all necessary build variables are exported
 	export CFLAGS CXXFLAGS MAKEFLAGS CHOST
+	# save our shell options so build() can't override what we need
+	local $shellopts=$(shopts -p)
 
 	local ret=0
 	if [ "$LOGGING" = "1" ]; then
@@ -695,6 +697,8 @@ run_build() {
 	else
 		build 2>&1 || ret=$?
 	fi
+	# reset our shell options
+	eval "$shellopts"
 
 	if [ $ret -gt 0 ]; then
 		error "$(gettext "Build Failed.")"
-- 
1.6.0.4


From dan at archlinux.org  Thu Dec  4 21:51:40 2008
From: dan at archlinux.org (Dan McGee)
Date: Thu, 4 Dec 2008 20:51:40 -0600
Subject: [pacman-dev] [PATCH] makepkg: save and restore shell options
	before and after build()
In-Reply-To: <1228445330-21769-1-git-send-email-dan@archlinux.org>
References: <1228445330-21769-1-git-send-email-dan@archlinux.org>
Message-ID: <449c10960812041851t780a7e7co74012002d52eaa14@mail.gmail.com>

On Thu, Dec 4, 2008 at 8:48 PM, Dan McGee <dan at archlinux.org> wrote:
> Fix the issue uncovered by FS#12344. In this instance, the dotglob shopt was
> being set in the build() function but never cleared, causing issues in the
> remaining parts of the makepkg script.
>
> Signed-off-by: Dan McGee <dan at archlinux.org>
> ---
>  scripts/makepkg.sh.in |    4 ++++
>  1 files changed, 4 insertions(+), 0 deletions(-)
>
> diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in
> index 4cc255c..25e4cc8 100644
> --- a/scripts/makepkg.sh.in
> +++ b/scripts/makepkg.sh.in
> @@ -675,6 +675,8 @@ run_build() {
>
>        # ensure all necessary build variables are exported
>        export CFLAGS CXXFLAGS MAKEFLAGS CHOST
> +       # save our shell options so build() can't override what we need
> +       local $shellopts=$(shopts -p)
I could have used this as a litmus test to see if anyone actually
tried this, but assume I put "shopt" (drop the s) here instead. Fixed
locally. :)

>
>        local ret=0
>        if [ "$LOGGING" = "1" ]; then
> @@ -695,6 +697,8 @@ run_build() {
>        else
>                build 2>&1 || ret=$?
>        fi
> +       # reset our shell options
> +       eval "$shellopts"
>
>        if [ $ret -gt 0 ]; then
>                error "$(gettext "Build Failed.")"
> --
> 1.6.0.4
>
>

From allan at archlinux.org  Thu Dec  4 22:00:36 2008
From: allan at archlinux.org (Allan McRae)
Date: Fri, 05 Dec 2008 13:00:36 +1000
Subject: [pacman-dev] [PATCH] makepkg: save and restore shell options
 before and after build()
In-Reply-To: <449c10960812041851t780a7e7co74012002d52eaa14@mail.gmail.com>
References: <1228445330-21769-1-git-send-email-dan@archlinux.org>
	<449c10960812041851t780a7e7co74012002d52eaa14@mail.gmail.com>
Message-ID: <BAY102-DAV4A4659E4C4FC866129FCE86FE0@phx.gbl>

Dan McGee wrote:
> On Thu, Dec 4, 2008 at 8:48 PM, Dan McGee <dan at archlinux.org> wrote:
>   
>> Fix the issue uncovered by FS#12344. In this instance, the dotglob shopt was
>> being set in the build() function but never cleared, causing issues in the
>> remaining parts of the makepkg script.
>>
>> Signed-off-by: Dan McGee <dan at archlinux.org>
>> ---
>>  scripts/makepkg.sh.in |    4 ++++
>>  1 files changed, 4 insertions(+), 0 deletions(-)
>>
>> diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in
>> index 4cc255c..25e4cc8 100644
>> --- a/scripts/makepkg.sh.in
>> +++ b/scripts/makepkg.sh.in
>> @@ -675,6 +675,8 @@ run_build() {
>>
>>        # ensure all necessary build variables are exported
>>        export CFLAGS CXXFLAGS MAKEFLAGS CHOST
>> +       # save our shell options so build() can't override what we need
>> +       local $shellopts=$(shopts -p)
>>     
> I could have used this as a litmus test to see if anyone actually
> tried this, but assume I put "shopt" (drop the s) here instead. Fixed
> locally. :)
>
>   

Ha,  i was just trying to figure out why I had not "shopts" command !

>>        local ret=0
>>        if [ "$LOGGING" = "1" ]; then
>> @@ -695,6 +697,8 @@ run_build() {
>>        else
>>                build 2>&1 || ret=$?
>>        fi
>> +       # reset our shell options
>> +       eval "$shellopts"
>>
>>        if [ $ret -gt 0 ]; then
>>                error "$(gettext "Build Failed.")"
>> --
>> 1.6.0.4
>>
>>     

Strange problem....  Patch looks good to me.

Allan




From dpmcgee at gmail.com  Thu Dec  4 22:12:07 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Thu, 4 Dec 2008 21:12:07 -0600
Subject: [pacman-dev] Dan's pacman tree build&test
In-Reply-To: <20081204194405.22c27e6d@tux1.brauer.lan>
References: <20081204154308.11a350d8@tux1.brauer.lan>
	<20081204194405.22c27e6d@tux1.brauer.lan>
Message-ID: <449c10960812041912v6387f0c2l55345731896d22d5@mail.gmail.com>

On Thu, Dec 4, 2008 at 12:44 PM, Gerhard Brauer <gerbra at archlinux.de> wrote:
> Ok, have tested the package signing feature from Dan's pacman git.
> (Thanks Allan for the hint with --disable-doc)
>
> I test with the abook package from extra.

Woohoo! Thanks for testing, this is much appreciated.

> 1)
> makepkg
> ==> Finished making: abook 0.5.6-2 i686 (Thu Dec  4 15:52:44 UTC 2008)
> ==> Signing package...
> ==> ERROR: Cannot find the gpg binary! Is gnupg installed?
> 2)
> makepkg
> ==> Finished making: abook 0.5.6-2 i686 (Thu Dec  4 15:55:34 UTC 2008)
> ==> Signing package...
> gpg: directory `/root/.gnupg' created
> gpg: new configuration file `/root/.gnupg/gpg.conf' created
> gpg: WARNING: options in `/root/.gnupg/gpg.conf' are not yet active during this run
> gpg: keyring `/root/.gnupg/secring.gpg' created
> gpg: keyring `/root/.gnupg/pubring.gpg' created
> gpg: no default secret key: secret key not available
> gpg: signing failed: secret key not available
> ==> WARNING: Failed to sign package file.
>
> That's right. I still have no gpg key.
> After setting up all gpg things makepkg builds and signs the package.
So it sounds like we have a relatively sane makepkg patch, with most
of the failure conditions working OK? This is good, and it means we
are mostly done in this department.

> 3)
> Add a repo: mypkg
> repo-add ad the abook package and puts also the %PGPSIG% field in the desc file.
Sweet. I think we are good here too.

> 4)
> pacman -S mypkg/abook
> checking package integrity...
> warning: gpg cmdline: gpg --verify --no-default-keyring --keyserver-options no-auto-key-retrieve --keyring /tmp/testing.gpg - /var/cache/pacman/pkg/abook-0.5.6-2-i686.pkg.tar.gz
> error: failed to commit transaction (invalid or corrupted package)
> abook-0.5.6-2-i686.pkg.tar.gz is invalid or corrupted
> Errors occurred, no packages were upgraded.
>
> Ok, i have not imported the public key to root's keyring.
>
> 5)
> [root at archtest ~]# LANG=C pacman -S mypkg/abook
> resolving dependencies...
> looking for inter-conflicts...
>
> Targets (1): abook-0.5.6-2
>
> Total Download Size:    0.00 MB
> Total Installed Size:   0.20 MB
>
> Proceed with installation? [Y/n]
> checking package integrity...
> warning: gpg cmdline: gpg --verify --no-default-keyring --keyserver-options no-auto-key-retrieve --keyring /tmp/testing.gpg - /var/cache/pacman/pkg/abook-0.5.6-2-i686.pkg.tar.gz
> (1/1) checking for file conflicts                   [#####################] 100%
> (1/1) installing abook                              [#####################] 100%
>
> Problem/Question:
> Where could i define the public keyring location?
> According to commit: "Add keyring location as option on libalpm handle" the is a libalpm option
> --keyring. But i have no plan where to define it (in pacman.conf i got an error).
> I copied my keyring temporary to /tmp/testing.gpg what seems the default search path and
> filename. Doing this i could install above abook from my repo.
You're delving into uncoded territory here, and not completely
thought-out territory. This still needs some work.

> 6)
> [root at archtest ~]# LANG=C pacman -Sy mypkg/abook
> :: Synchronizing package databases...
>  core is up to date
>  extra is up to date
>  community is up to date
>  mypkg is up to date
> warning: abook-0.5.6-2 is up to date -- reinstalling
> resolving dependencies...
> looking for inter-conflicts...
>
> Targets (1): abook-0.5.6-2
>
> Total Download Size:    0.05 MB
> Total Installed Size:   0.20 MB
>
> Proceed with installation? [Y/n]
> :: Retrieving packages from mypkg...
>  abook-0.5.6-2-i686        49.6K   20.9M/s 00:00:00 [#####################] 100%
> checking package integrity...
> warning: gpg cmdline: gpg --verify --no-default-keyring --keyserver-options no-a
> uto-key-retrieve --keyring /tmp/testing.gpg - /var/cache/pacman/pkg/abook-0.5.6-
> 2-i686.pkg.tar.gz
> error: failed to commit transaction (invalid or corrupted package)
> abook-0.5.6-2-i686.pkg.tar.gz is invalid or corrupted
> Errors occurred, no packages were upgraded.
>
> Here if have modified the abook-0.5.6-2-i686.pkg.tar.gz package, copied to my repo,
> do a repo-add but use the old *.sig signature. This modified package gets not
> installed.
> Maybe the error/reason could be more explained.
Yeah, once again this is definitely work in progress.  There is still
a good bit to be done, as the current pacman/libalpm/gpg integration
is hairy.

> Summary:
> I think most of the signing part (makepkg, repo-add) and the verifying
> part (pacman) works so far. Awesome!
> gpg verifying is good integrated in pacman, the "warning: gpg cmdline"
> line thing i assume is a test/debug thing.
>
> Next step could be: verifying the database files during pacman -Sy ?
There is nothing to verify about the database yet. Eventually we can
sign these as well if necessary, but right now the only sigs are on
the packages themselves. This is an area that will need work as it is
possible to make completely valid databases with valid packages, but
an attacker could purposely hold back package releases to keep
vulnerabilities open.

Thanks for your help and feedback.

-Dan

From aaronmgriffin at gmail.com  Thu Dec  4 23:39:36 2008
From: aaronmgriffin at gmail.com (Aaron Griffin)
Date: Thu, 4 Dec 2008 22:39:36 -0600
Subject: [pacman-dev] [PATCH] makepkg: save and restore shell options
	before and after build()
In-Reply-To: <BAY102-DAV4A4659E4C4FC866129FCE86FE0@phx.gbl>
References: <1228445330-21769-1-git-send-email-dan@archlinux.org>
	<449c10960812041851t780a7e7co74012002d52eaa14@mail.gmail.com>
	<BAY102-DAV4A4659E4C4FC866129FCE86FE0@phx.gbl>
Message-ID: <d64a48980812042039j5acda667w59466960d00c580@mail.gmail.com>

On Thu, Dec 4, 2008 at 9:00 PM, Allan McRae <allan at archlinux.org> wrote:
> Dan McGee wrote:
>>
>> On Thu, Dec 4, 2008 at 8:48 PM, Dan McGee <dan at archlinux.org> wrote:
>>
>>>
>>> Fix the issue uncovered by FS#12344. In this instance, the dotglob shopt
>>> was
>>> being set in the build() function but never cleared, causing issues in
>>> the
>>> remaining parts of the makepkg script.
>>>
>>> Signed-off-by: Dan McGee <dan at archlinux.org>
>>> ---
>>>  scripts/makepkg.sh.in |    4 ++++
>>>  1 files changed, 4 insertions(+), 0 deletions(-)
>>>
>>> diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in
>>> index 4cc255c..25e4cc8 100644
>>> --- a/scripts/makepkg.sh.in
>>> +++ b/scripts/makepkg.sh.in
>>> @@ -675,6 +675,8 @@ run_build() {
>>>
>>>       # ensure all necessary build variables are exported
>>>       export CFLAGS CXXFLAGS MAKEFLAGS CHOST
>>> +       # save our shell options so build() can't override what we need
>>> +       local $shellopts=$(shopts -p)
>>>
>>
>> I could have used this as a litmus test to see if anyone actually
>> tried this, but assume I put "shopt" (drop the s) here instead. Fixed
>> locally. :)
>>
>>
>
> Ha,  i was just trying to figure out why I had not "shopts" command !
>
>>>       local ret=0
>>>       if [ "$LOGGING" = "1" ]; then
>>> @@ -695,6 +697,8 @@ run_build() {
>>>       else
>>>               build 2>&1 || ret=$?
>>>       fi
>>> +       # reset our shell options
>>> +       eval "$shellopts"
>>>
>>>       if [ $ret -gt 0 ]; then
>>>               error "$(gettext "Build Failed.")"
>>> --
>>> 1.6.0.4
>>>
>>>
>
> Strange problem....  Patch looks good to me.

With even stranger symptoms! Thayer build a package that had 2
.PKGINFO files in it. I didn't even know tar could do that. This
actually caused goofy issues on gerolde when validating that the
package's arch was the same as the repo it was going to - because the
check basically cat's the .PKGINFO and looks for the arch setting.
Instead of 'i686' it was getting 'i686\ni686'

From allan at archlinux.org  Thu Dec  4 23:48:12 2008
From: allan at archlinux.org (Allan McRae)
Date: Fri, 05 Dec 2008 14:48:12 +1000
Subject: [pacman-dev] [PATCH] makepkg: save and restore shell options
 before and after build()
In-Reply-To: <d64a48980812042039j5acda667w59466960d00c580@mail.gmail.com>
References: <1228445330-21769-1-git-send-email-dan@archlinux.org>	<449c10960812041851t780a7e7co74012002d52eaa14@mail.gmail.com>	<BAY102-DAV4A4659E4C4FC866129FCE86FE0@phx.gbl>
	<d64a48980812042039j5acda667w59466960d00c580@mail.gmail.com>
Message-ID: <BAY102-DAV438A3FCE264B7674CEB2D86FE0@phx.gbl>

Aaron Griffin wrote:
> On Thu, Dec 4, 2008 at 9:00 PM, Allan McRae <allan at archlinux.org> wrote:
>   
>> Dan McGee wrote:
>>     
>>> On Thu, Dec 4, 2008 at 8:48 PM, Dan McGee <dan at archlinux.org> wrote:
>>>
>>>       
>>>> Fix the issue uncovered by FS#12344. In this instance, the dotglob shopt
>>>> was
>>>> being set in the build() function but never cleared, causing issues in
>>>> the
>>>> remaining parts of the makepkg script.
>>>>
>>>> Signed-off-by: Dan McGee <dan at archlinux.org>
>>>> ---
>>>>  scripts/makepkg.sh.in |    4 ++++
>>>>  1 files changed, 4 insertions(+), 0 deletions(-)
>>>>
>>>> diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in
>>>> index 4cc255c..25e4cc8 100644
>>>> --- a/scripts/makepkg.sh.in
>>>> +++ b/scripts/makepkg.sh.in
>>>> @@ -675,6 +675,8 @@ run_build() {
>>>>
>>>>       # ensure all necessary build variables are exported
>>>>       export CFLAGS CXXFLAGS MAKEFLAGS CHOST
>>>> +       # save our shell options so build() can't override what we need
>>>> +       local $shellopts=$(shopts -p)
>>>>
>>>>         
>>> I could have used this as a litmus test to see if anyone actually
>>> tried this, but assume I put "shopt" (drop the s) here instead. Fixed
>>> locally. :)
>>>
>>>
>>>       
>> Ha,  i was just trying to figure out why I had not "shopts" command !
>>
>>     
>>>>       local ret=0
>>>>       if [ "$LOGGING" = "1" ]; then
>>>> @@ -695,6 +697,8 @@ run_build() {
>>>>       else
>>>>               build 2>&1 || ret=$?
>>>>       fi
>>>> +       # reset our shell options
>>>> +       eval "$shellopts"
>>>>
>>>>       if [ $ret -gt 0 ]; then
>>>>               error "$(gettext "Build Failed.")"
>>>> --
>>>> 1.6.0.4
>>>>
>>>>
>>>>         
>> Strange problem....  Patch looks good to me.
>>     
>
> With even stranger symptoms! Thayer build a package that had 2
> .PKGINFO files in it. I didn't even know tar could do that. This
> actually caused goofy issues on gerolde when validating that the
> package's arch was the same as the repo it was going to - because the
> check basically cat's the .PKGINFO and looks for the arch setting.
> Instead of 'i686' it was getting 'i686\ni686'

It took me a while to try and duplicate this because looking at the 
package file or extracting, you only see one .PKGINFO file.  However, we 
directly read from the package file rather than extracting to disk so 
that causes the issue.  To see it by eye:

bsdtar -czvf test.tar.gz .PKGINFO .PKGINFO .PKGINFO
bsdtar -xzvf test.tar.gz





From gerbra at archlinux.de  Fri Dec  5 04:42:56 2008
From: gerbra at archlinux.de (Gerhard Brauer)
Date: Fri, 5 Dec 2008 10:42:56 +0100
Subject: [pacman-dev] Dan's pacman tree build&test
In-Reply-To: <449c10960812041912v6387f0c2l55345731896d22d5@mail.gmail.com>
References: <20081204154308.11a350d8@tux1.brauer.lan>
	<20081204194405.22c27e6d@tux1.brauer.lan>
	<449c10960812041912v6387f0c2l55345731896d22d5@mail.gmail.com>
Message-ID: <20081205104256.714b36f6@tux1.brauer.lan>

On Thu, 4 Dec 2008 21:12:07 -0600
wrote "Dan McGee" <dpmcgee at gmail.com>:

> On Thu, Dec 4, 2008 at 12:44 PM, Gerhard Brauer <gerbra at archlinux.de>
> wrote:
> > Summary:
> > I think most of the signing part (makepkg, repo-add) and the
> > verifying part (pacman) works so far. Awesome!
> > gpg verifying is good integrated in pacman, the "warning: gpg
> > cmdline" line thing i assume is a test/debug thing.
> >
> > Next step could be: verifying the database files during pacman -Sy ?
> >
> There is nothing to verify about the database yet. Eventually we can
> sign these as well if necessary, but right now the only sigs are on
> the packages themselves.

I think signing the database files on gerolde is equal important than
signing the packages. Cause pacman will have not a default setting
like: check **all** packages if they were signed (local or foreign
repos). So the %PGPSIG% field in the database is the only indicator for
pacman: is this a signed package or not. So we must secure the database
files against manipulations like removing, modifying this field.

> This is an area that will need work as it is
> possible to make completely valid databases with valid packages, but
> an attacker could purposely hold back package releases to keep
> vulnerabilities open.

That's also a good point. Some propositions on this were to get the
database files only from ftp.archlinux.org. But these are also only
mirrors and this thought is also not doable cause the different sync
levels of our mirrors.
One short idea: Pierre and myself do still mirror checking on their
sync states. That checks could maybe enhanced to check if the databases
are on a quiet actual level or integrity... Hmmmm
 
> Thanks for your help and feedback.

No thanks needed. For myself i WANT this feature.

Some thoughts about more generally things which may need a little time
to discuss (i don't want answers, this are only things i ask myself):

a) On official repos (core,extra,...) pacman should not be allowed to
install unsigned packages from. But pacman should still honor own local
or foreign repos which may be unsigned.

b) To solve this (and the point: where is the keyring?) maybe we could
check a new entry in pacman.conf for the repos:
[core]
Keyring = /etc/pacman.d/archlinux.gpg
Include = /etc/pacman.d/mirrorlist
So pacman could decide: Have i to check this repo for signed packages
and where the needed public keyring could be found. So also local or
foreign repos could use the signing feature.

c) Should we add an option to makepkg to let the developer/packager
choose which secret key from his keyring should be used for signing?
Maybe he won't use his default key and have a extra archlinux key
generated.

d) Currently we work on the libalpm integration. But what when users
must or will use wget/curl via XferCommand? Sure, we could provide
skeleton example scripts how to integrate gpg in this. But we give this
work more i users hand. Or may our state: pacman and its secure
framework is *only* given if you use the libalpm way?

e) What's with our other devel tools (for ex. makechrootpkg)? Is
signing also integrated in this tools?

This weekend i will put the "signing pacman" on my machine to test it
with my complete own repo, not only on a single package.

> -Dan

Regards
	Gerhard

From aaronmgriffin at gmail.com  Fri Dec  5 12:08:01 2008
From: aaronmgriffin at gmail.com (Aaron Griffin)
Date: Fri, 5 Dec 2008 11:08:01 -0600
Subject: [pacman-dev] Dan's pacman tree build&test
In-Reply-To: <20081205104256.714b36f6@tux1.brauer.lan>
References: <20081204154308.11a350d8@tux1.brauer.lan>
	<20081204194405.22c27e6d@tux1.brauer.lan>
	<449c10960812041912v6387f0c2l55345731896d22d5@mail.gmail.com>
	<20081205104256.714b36f6@tux1.brauer.lan>
Message-ID: <d64a48980812050908q6d1e1f17vd2f6a5c719407412@mail.gmail.com>

On Fri, Dec 5, 2008 at 3:42 AM, Gerhard Brauer <gerbra at archlinux.de> wrote:
> e) What's with our other devel tools (for ex. makechrootpkg)? Is
> signing also integrated in this tools?

makechrootpkg just calls makepkg, so it should work, assuming there is
a key in the chroot. Not sure what would be needed there.

From jud at judfilm.net  Fri Dec  5 21:12:05 2008
From: jud at judfilm.net (Jud)
Date: Sat, 6 Dec 2008 12:12:05 +1000
Subject: [pacman-dev] PKGBUILD.proto
Message-ID: <20081206121205.264bfaf0@judfilm.net>

Hi,

Dan suggested I send this to the pacman-dev list.

After completing some research and asking alot of questions I present
some minor changes to PKGBUILD.proto supplied as a .diff to be merged
after your approval. I believe it helps the intended audience create a
better PKGBUILD in less time according to the latest Arch Packaging
Standards.

Cheers
Jud


Inline:
--- PKGBUILD.proto	2008-12-05 23:32:33.000005000 +1000
+++ PKGBUILD.proto.new	2008-12-05 23:37:45.374547000 +1000
@@ -3,13 +3,15 @@
 # NOTE: Please fill out the license field for your package! If it is
unknown, # then please put 'unknown'.
 
-# Contributor: Your Name <youremail at domain.com>
+
+# Contributor: Your Name <youremail at domain.com> # Use dots only to
reduce spam +
 pkgname=NAME
-pkgver=VERSION
+pkgver=VERSION # Note: if pkgver is '0.99-10' then use an underscore,
i.e. '0.99_10' pkgrel=1
 pkgdesc=""
-arch=()
-url=""
+url="http://ADDRESS/"
+arch=('i686' 'x86_64')
 license=('GPL')
 groups=()
 depends=()
@@ -20,17 +22,13 @@
 replaces=()
 backup=()
 options=()
-install=
-source=($pkgname-$pkgver.tar.gz)
-noextract=()
-md5sums=() #generate with 'makepkg -g'
+install=(${pkgname}.install)
+source=(http://ADDRESS/TO/FILE/${pkgname}-${pkgver}.tar.gz)
+md5sums=() # Generate with 'makepkg -g'
 
 build() {
-  cd "$srcdir/$pkgname-$pkgver"
-
-  ./configure --prefix=/usr
+  cd ${srcdir}${pkgname}-${pkgver}
+  ./configure --prefix=usr
   make || return 1
-  make DESTDIR="$pkgdir/" install
+  make DESTDIR=${pkgdir} install || return 1
 }
-
-# vim:set ts=2 sw=2 et:
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PKGBUILD.proto.diff
Type: text/x-patch
Size: 1176 bytes
Desc: not available
URL: <http://archlinux.org/pipermail/pacman-dev/attachments/20081206/d3f02097/attachment.bin>

From allan at archlinux.org  Fri Dec  5 21:47:16 2008
From: allan at archlinux.org (Allan McRae)
Date: Sat, 06 Dec 2008 12:47:16 +1000
Subject: [pacman-dev] PKGBUILD.proto
In-Reply-To: <20081206121205.264bfaf0@judfilm.net>
References: <20081206121205.264bfaf0@judfilm.net>
Message-ID: <BAY102-DAV33427B2A8FEF67D567A2A86FF0@phx.gbl>

Jud wrote:
> Hi,
>
> Dan suggested I send this to the pacman-dev list.
>
> After completing some research and asking alot of questions I present
> some minor changes to PKGBUILD.proto supplied as a .diff to be merged
> after your approval. I believe it helps the intended audience create a
> better PKGBUILD in less time according to the latest Arch Packaging
> Standards.
>
> Cheers
> Jud
>
>
> Inline:
> --- PKGBUILD.proto	2008-12-05 23:32:33.000005000 +1000
> +++ PKGBUILD.proto.new	2008-12-05 23:37:45.374547000 +1000
> @@ -3,13 +3,15 @@
>  # NOTE: Please fill out the license field for your package! If it is
> unknown, # then please put 'unknown'.
>  
> -# Contributor: Your Name <youremail at domain.com>
> +
> +# Contributor: Your Name <youremail at domain.com> # Use dots only to
> reduce spam +
>   

I'm sure people can figure that out for themselves....

>  pkgname=NAME
> -pkgver=VERSION
> +pkgver=VERSION # Note: if pkgver is '0.99-10' then use an underscore,
> i.e. '0.99_10' pkgrel=1
>  pkgdesc=""
> -arch=()
> -url=""
> +url="http://ADDRESS/"
> +arch=('i686' 'x86_64')
>  license=('GPL')
>  groups=()
>  depends=()
> @@ -20,17 +22,13 @@
>  replaces=()
>  backup=()
>  options=()
> -install=
> -source=($pkgname-$pkgver.tar.gz)
> -noextract=()
> -md5sums=() #generate with 'makepkg -g'
> +install=(${pkgname}.install)
>   

I really dislike the brackets there.  install holds a value not an array 
much like pkgname, pkgrel.

> +source=(http://ADDRESS/TO/FILE/${pkgname}-${pkgver}.tar.gz)
> +md5sums=() # Generate with 'makepkg -g'
>  
>  build() {
> -  cd "$srcdir/$pkgname-$pkgver"
> -
> -  ./configure --prefix=/usr
> +  cd ${srcdir}${pkgname}-${pkgver}
> +  ./configure --prefix=usr
>    make || return 1
> -  make DESTDIR="$pkgdir/" install
> +  make DESTDIR=${pkgdir} install || return 1
>  }
> -
> -# vim:set ts=2 sw=2 et:
>   
> ------------------------------------------------------------------------
>
> _______________________________________________
> pacman-dev mailing list
> pacman-dev at archlinux.org
> http://archlinux.org/mailman/listinfo/pacman-dev




From belanger at ASTRO.UMontreal.CA  Fri Dec  5 22:25:18 2008
From: belanger at ASTRO.UMontreal.CA (=?ISO-8859-1?Q?Eric_B=E9langer?=)
Date: Fri, 5 Dec 2008 22:25:18 -0500 (EST)
Subject: [pacman-dev] PKGBUILD.proto
In-Reply-To: <BAY102-DAV33427B2A8FEF67D567A2A86FF0@phx.gbl>
References: <20081206121205.264bfaf0@judfilm.net>
	<BAY102-DAV33427B2A8FEF67D567A2A86FF0@phx.gbl>
Message-ID: <Pine.LNX.4.64.0812052218520.18680@localhost.localdomain>

On Sat, 6 Dec 2008, Allan McRae wrote:

> Jud wrote:
>> Hi,
>> 
>> Dan suggested I send this to the pacman-dev list.
>> 
>> After completing some research and asking alot of questions I present
>> some minor changes to PKGBUILD.proto supplied as a .diff to be merged
>> after your approval. I believe it helps the intended audience create a
>> better PKGBUILD in less time according to the latest Arch Packaging
>> Standards.
>> 
>> Cheers
>> Jud
>> 
>> 
>> Inline:
>> --- PKGBUILD.proto	2008-12-05 23:32:33.000005000 +1000
>> +++ PKGBUILD.proto.new	2008-12-05 23:37:45.374547000 +1000
>> @@ -3,13 +3,15 @@
>>  # NOTE: Please fill out the license field for your package! If it is
>> unknown, # then please put 'unknown'.
>>  -# Contributor: Your Name <youremail at domain.com>
>> +
>> +# Contributor: Your Name <youremail at domain.com> # Use dots only to
>> reduce spam +
>> 
>
> I'm sure people can figure that out for themselves....
>
>>  pkgname=NAME
>> -pkgver=VERSION
>> +pkgver=VERSION # Note: if pkgver is '0.99-10' then use an underscore,
>> i.e. '0.99_10' pkgrel=1
>>  pkgdesc=""
>> -arch=()
>> -url=""
>> +url="http://ADDRESS/"
>> +arch=('i686' 'x86_64')

By convention, the arch field goes right after the pkgdesc


>>  license=('GPL')
>>  groups=()
>>  depends=()
>> @@ -20,17 +22,13 @@
>>  replaces=()
>>  backup=()
>>  options=()
>> -install=
>> -source=($pkgname-$pkgver.tar.gz)
>> -noextract=()

Why did you removed the noextract field? Was it done by mistake?

>> -md5sums=() #generate with 'makepkg -g'
>> +install=(${pkgname}.install)
>> 
>
> I really dislike the brackets there.  install holds a value not an array much 
> like pkgname, pkgrel.
>
>> +source=(http://ADDRESS/TO/FILE/${pkgname}-${pkgver}.tar.gz)
>> +md5sums=() # Generate with 'makepkg -g'
>>   build() {
>> -  cd "$srcdir/$pkgname-$pkgver"
>> -
>> -  ./configure --prefix=/usr
>> +  cd ${srcdir}${pkgname}-${pkgver}

you forgot a /


>> +  ./configure --prefix=usr
>>    make || return 1
>> -  make DESTDIR="$pkgdir/" install
>> +  make DESTDIR=${pkgdir} install || return 1
>>  }
>> -
>> -# vim:set ts=2 sw=2 et:
>>   ------------------------------------------------------------------------


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From sega01 at gmail.com  Sat Dec  6 08:00:29 2008
From: sega01 at gmail.com (Teran McKinney)
Date: Sat, 6 Dec 2008 13:00:29 +0000
Subject: [pacman-dev] PKGBUILD.proto
In-Reply-To: <Pine.LNX.4.64.0812052218520.18680@localhost.localdomain>
References: <20081206121205.264bfaf0@judfilm.net>
	<BAY102-DAV33427B2A8FEF67D567A2A86FF0@phx.gbl>
	<Pine.LNX.4.64.0812052218520.18680@localhost.localdomain>
Message-ID: <b245135e0812060500v50b495ebpf6b3b7101e4e72e6@mail.gmail.com>

Seems a little more concise to me, but I personally think that the
install script should be commented out (if it has a default value).
Maybe arch=('any') should be default? Would make life easier for
non-i686/x86_64 maintainers, and is generally much more truthful.

Cheers,
Teran (sega01)

On Sat, Dec 6, 2008 at 03:25, Eric B?langer <belanger at astro.umontreal.ca> wrote:
> On Sat, 6 Dec 2008, Allan McRae wrote:
>
>> Jud wrote:
>>>
>>> Hi,
>>>
>>> Dan suggested I send this to the pacman-dev list.
>>>
>>> After completing some research and asking alot of questions I present
>>> some minor changes to PKGBUILD.proto supplied as a .diff to be merged
>>> after your approval. I believe it helps the intended audience create a
>>> better PKGBUILD in less time according to the latest Arch Packaging
>>> Standards.
>>>
>>> Cheers
>>> Jud
>>>
>>>
>>> Inline:
>>> --- PKGBUILD.proto      2008-12-05 23:32:33.000005000 +1000
>>> +++ PKGBUILD.proto.new  2008-12-05 23:37:45.374547000 +1000
>>> @@ -3,13 +3,15 @@
>>>  # NOTE: Please fill out the license field for your package! If it is
>>> unknown, # then please put 'unknown'.
>>>  -# Contributor: Your Name <youremail at domain.com>
>>> +
>>> +# Contributor: Your Name <youremail at domain.com> # Use dots only to
>>> reduce spam +
>>>
>>
>> I'm sure people can figure that out for themselves....
>>
>>>  pkgname=NAME
>>> -pkgver=VERSION
>>> +pkgver=VERSION # Note: if pkgver is '0.99-10' then use an underscore,
>>> i.e. '0.99_10' pkgrel=1
>>>  pkgdesc=""
>>> -arch=()
>>> -url=""
>>> +url="http://ADDRESS/"
>>> +arch=('i686' 'x86_64')
>
> By convention, the arch field goes right after the pkgdesc
>
>
>>>  license=('GPL')
>>>  groups=()
>>>  depends=()
>>> @@ -20,17 +22,13 @@
>>>  replaces=()
>>>  backup=()
>>>  options=()
>>> -install=
>>> -source=($pkgname-$pkgver.tar.gz)
>>> -noextract=()
>
> Why did you removed the noextract field? Was it done by mistake?
>
>>> -md5sums=() #generate with 'makepkg -g'
>>> +install=(${pkgname}.install)
>>>
>>
>> I really dislike the brackets there.  install holds a value not an array
>> much like pkgname, pkgrel.
>>
>>> +source=(http://ADDRESS/TO/FILE/${pkgname}-${pkgver}.tar.gz)
>>> +md5sums=() # Generate with 'makepkg -g'
>>>  build() {
>>> -  cd "$srcdir/$pkgname-$pkgver"
>>> -
>>> -  ./configure --prefix=/usr
>>> +  cd ${srcdir}${pkgname}-${pkgver}
>
> you forgot a /
>
>
>>> +  ./configure --prefix=usr
>>>   make || return 1
>>> -  make DESTDIR="$pkgdir/" install
>>> +  make DESTDIR=${pkgdir} install || return 1
>>>  }
>>> -
>>> -# vim:set ts=2 sw=2 et:
>>>  ------------------------------------------------------------------------
>
>
> --
> This message has been scanned for viruses and
> dangerous content by MailScanner, and is
> believed to be clean.
>
> _______________________________________________
> pacman-dev mailing list
> pacman-dev at archlinux.org
> http://archlinux.org/mailman/listinfo/pacman-dev
>

From allan at archlinux.org  Sat Dec  6 09:37:22 2008
From: allan at archlinux.org (Allan McRae)
Date: Sun, 07 Dec 2008 00:37:22 +1000
Subject: [pacman-dev] [Fwd: [arch-general] PKGBUILD and DOS line endings]
Message-ID: <BAY102-DAV89FA49C215929396E7A6086FF0@phx.gbl>

Should we be handling this issue?

Allan

-------- Original Message --------
Subject: 	[arch-general] PKGBUILD and DOS line endings
Date: 	Sat, 6 Dec 2008 15:32:26 +0100
From: 	Alessandro Doro <ordo.ad at gmail.com>
Reply-To: 	General Discusson about Arch Linux <arch-general at archlinux.org>
To: 	arch-general at archlinux.org



I found in AUR a PKGBUILD with DOS line endings.
See what happens:

$ makepkg
: command not found

==> ERROR: An unknown error has occured. Exiting...


makepkg simply sources "$BUILDSCRIPT" and bash treats '\r' literally.

The PKGBUILD maintainer is surely to be blamed and I don't think that
`makepkg' should try to catch this error (sorry for some innocent user).

Just wanted to expose this "feature".

bye






From markc at renta.net  Sat Dec  6 10:04:50 2008
From: markc at renta.net (Mark Constable)
Date: Sun, 7 Dec 2008 01:04:50 +1000
Subject: [pacman-dev] bzip2 package compression
Message-ID: <200812070104.50208.markc@renta.net>

I changed these in /etc/makepkg.conf to bz2 variants ages
ago and assumed that the resulting packages would be bzip2
compressed. When I double checked the packages they were
still only gzip compressed even though the extension was
tar.bz2. Everything works so I didn't pick it up but I'd be
interested in the bandwidth saving from bzip2 compression.

 PKGEXT='.pkg.tar.bz2'
 SRCEXT='.src.tar.bz2'
 DB_COMPRESSION='bz2'

Is there a reason that a -cjf is not used in makepkg ?

% grep bsdtar /usr/bin/makepkg
                        cmd="bsdtar -x -f $file" ;;
if ! bsdtar -czf "$pkg_file" $comp_files *; then
        bsdtar -xOf "$old_file" .PKGINFO > "$pkginfo" || continue
if ! bsdtar -czLf "$pkg_file" ${pkgname}; then

--markc


From dpmcgee at gmail.com  Sat Dec  6 12:48:44 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Sat, 6 Dec 2008 11:48:44 -0600
Subject: [pacman-dev] [Fwd: [arch-general] PKGBUILD and DOS line endings]
In-Reply-To: <BAY102-DAV89FA49C215929396E7A6086FF0@phx.gbl>
References: <BAY102-DAV89FA49C215929396E7A6086FF0@phx.gbl>
Message-ID: <449c10960812060948i2c3c6d68paa6dd8e7add58bfc@mail.gmail.com>

On Sat, Dec 6, 2008 at 8:37 AM, Allan McRae <allan at archlinux.org> wrote:
> Should we be handling this issue?

I feel like a slightly more descriptive failure message wouldn't hurt.
Is it easy to handle? Someone would need to do some playing.

-Dan

> -------- Original Message --------
> Subject:        [arch-general] PKGBUILD and DOS line endings
> Date:   Sat, 6 Dec 2008 15:32:26 +0100
> From:   Alessandro Doro <ordo.ad at gmail.com>
> Reply-To:       General Discusson about Arch Linux
> <arch-general at archlinux.org>
> To:     arch-general at archlinux.org
>
>
>
> I found in AUR a PKGBUILD with DOS line endings.
> See what happens:
>
> $ makepkg
> : command not found
>
> ==> ERROR: An unknown error has occured. Exiting...
>
>
> makepkg simply sources "$BUILDSCRIPT" and bash treats '\r' literally.
>
> The PKGBUILD maintainer is surely to be blamed and I don't think that
> `makepkg' should try to catch this error (sorry for some innocent user).
>
> Just wanted to expose this "feature".
>
> bye
>
>
>
>
>
> _______________________________________________
> pacman-dev mailing list
> pacman-dev at archlinux.org
> http://archlinux.org/mailman/listinfo/pacman-dev
>

From dpmcgee at gmail.com  Sat Dec  6 12:57:12 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Sat, 6 Dec 2008 11:57:12 -0600
Subject: [pacman-dev] bzip2 package compression
In-Reply-To: <200812070104.50208.markc@renta.net>
References: <200812070104.50208.markc@renta.net>
Message-ID: <449c10960812060957vdb468ecs5ba132010f1c89af@mail.gmail.com>

On Sat, Dec 6, 2008 at 9:04 AM, Mark Constable <markc at renta.net> wrote:
> I changed these in /etc/makepkg.conf to bz2 variants ages
> ago and assumed that the resulting packages would be bzip2
> compressed. When I double checked the packages they were
> still only gzip compressed even though the extension was
> tar.bz2. Everything works so I didn't pick it up but I'd be
> interested in the bandwidth saving from bzip2 compression.
>
>  PKGEXT='.pkg.tar.bz2'
>  SRCEXT='.src.tar.bz2'
>  DB_COMPRESSION='bz2'
>
> Is there a reason that a -cjf is not used in makepkg ?

	local TAR_OPT
	case "$PKGEXT" in
		*tar.gz)  TAR_OPT="z" ;;
		*tar.bz2) TAR_OPT="j" ;;
		*) warning "$(gettext "'%s' is not a valid archive extension.")" \
		"$PKGEXT" ;;
	esac

The current code is a bit smarter and compresses correctly depending
on the extension.

-Dan

From aaronmgriffin at gmail.com  Sat Dec  6 19:06:37 2008
From: aaronmgriffin at gmail.com (Aaron Griffin)
Date: Sat, 6 Dec 2008 18:06:37 -0600
Subject: [pacman-dev] [Fwd: [arch-general] PKGBUILD and DOS line endings]
In-Reply-To: <449c10960812060948i2c3c6d68paa6dd8e7add58bfc@mail.gmail.com>
References: <BAY102-DAV89FA49C215929396E7A6086FF0@phx.gbl>
	<449c10960812060948i2c3c6d68paa6dd8e7add58bfc@mail.gmail.com>
Message-ID: <d64a48980812061606y506e2656teef17c6a8dd42680@mail.gmail.com>

On Sat, Dec 6, 2008 at 11:48 AM, Dan McGee <dpmcgee at gmail.com> wrote:
> On Sat, Dec 6, 2008 at 8:37 AM, Allan McRae <allan at archlinux.org> wrote:
>> Should we be handling this issue?
>
> I feel like a slightly more descriptive failure message wouldn't hurt.
> Is it easy to handle? Someone would need to do some playing.

I don't think this is easy to handle, as it's at the bash level. You
could maybe run "file" on the PKGBUILD to check...

$ file testdos
testdos: ASCII text, with CRLF line terminators

$ file testunix
testunix: ASCII text

From markc at renta.net  Sat Dec  6 21:29:37 2008
From: markc at renta.net (Mark Constable)
Date: Sun, 7 Dec 2008 12:29:37 +1000
Subject: [pacman-dev] bzip2 package compression
In-Reply-To: <449c10960812060957vdb468ecs5ba132010f1c89af@mail.gmail.com>
References: <200812070104.50208.markc@renta.net>
	<449c10960812060957vdb468ecs5ba132010f1c89af@mail.gmail.com>
Message-ID: <200812071229.37570.markc@renta.net>

On 2008-12-07, Dan McGee wrote:
> > Is there a reason that a -cjf is not used in makepkg ?
> ...
> The current code is a bit smarter and compresses correctly
> depending on the extension.

Ah cool, I presume this is in your Git repo?

If so, do you think your current git repo is "stable
enough" to give this a whirl with the few dozen folks
using the daily kde-svn packages?

--markc


From dpmcgee at gmail.com  Sat Dec  6 22:05:58 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Sat, 6 Dec 2008 21:05:58 -0600
Subject: [pacman-dev] bzip2 package compression
In-Reply-To: <200812071229.37570.markc@renta.net>
References: <200812070104.50208.markc@renta.net>
	<449c10960812060957vdb468ecs5ba132010f1c89af@mail.gmail.com>
	<200812071229.37570.markc@renta.net>
Message-ID: <449c10960812061905v25746161n8607d78e0394fd0@mail.gmail.com>

On Sat, Dec 6, 2008 at 8:29 PM, Mark Constable <markc at renta.net> wrote:
> On 2008-12-07, Dan McGee wrote:
>> > Is there a reason that a -cjf is not used in makepkg ?
>> ...
>> The current code is a bit smarter and compresses correctly
>> depending on the extension.
>
> Ah cool, I presume this is in your Git repo?
>
> If so, do you think your current git repo is "stable
> enough" to give this a whirl with the few dozen folks
> using the daily kde-svn packages?

Yeah this has probably been in GIT for some time.

All of us that are regular contributors to pacman are already using
pacman-git, so yeah, it better be stable. If you do go this route, be
sure to report problems to the list or in Flyspray (set the reported
version to 'git').

-Dan

From jud at judfilm.net  Sat Dec  6 22:58:38 2008
From: jud at judfilm.net (Jud)
Date: Sun, 7 Dec 2008 13:58:38 +1000
Subject: [pacman-dev] PKGBUILD.proto - Take2
Message-ID: <20081207135838.49912e8a@judfilm.net>

Hi,

Edit: Take2, changed the install line

Dan suggested I send this to the pacman-dev list.

After completing some research and asking alot of questions I present
some minor changes to PKGBUILD.proto supplied as a .diff to be merged
after your approval. I believe it helps the intended audience create a
better PKGBUILD in less time according to the latest Arch Packaging
Standards.

Cheers
Jud

Inline:
--- PKGBUILD.proto	2008-12-05 23:32:33.000005000 +1000
+++ PKGBUILD.proto.new	2008-12-07 13:49:43.889569135 +1000
@@ -3,13 +3,15 @@
 # NOTE: Please fill out the license field for your package! If it is
unknown, # then please put 'unknown'.
 
+
 # Contributor: Your Name <youremail at domain.com>
+
 pkgname=NAME
-pkgver=VERSION
+pkgver=VERSION # Note: if pkgver is '0.99-10' then use an underscore,
i.e. '0.99_10' pkgrel=1
 pkgdesc=""
-arch=()
-url=""
+url="http://ADDRESS/"
+arch=('i686' 'x86_64')
 license=('GPL')
 groups=()
 depends=()
@@ -20,17 +22,13 @@
 replaces=()
 backup=()
 options=()
-install=
-source=($pkgname-$pkgver.tar.gz)
-noextract=()
-md5sums=() #generate with 'makepkg -g'
+install=${pkgname}.install
+source=(http://ADDRESS/TO/FILE/${pkgname}-${pkgver}.tar.gz)
+md5sums=() # Generate with 'makepkg -g'
 
 build() {
-  cd "$srcdir/$pkgname-$pkgver"
-
-  ./configure --prefix=/usr
+  cd ${srcdir}${pkgname}-${pkgver}
+  ./configure --prefix=usr
   make || return 1
-  make DESTDIR="$pkgdir/" install
+  make DESTDIR=${pkgdir} install || return 1
 }
-
-# vim:set ts=2 sw=2 et:
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PKGBUILD.proto.diff
Type: text/x-patch
Size: 1094 bytes
Desc: not available
URL: <http://archlinux.org/pipermail/pacman-dev/attachments/20081207/8eaee355/attachment.bin>

From belanger at ASTRO.UMontreal.CA  Sat Dec  6 22:45:15 2008
From: belanger at ASTRO.UMontreal.CA (=?ISO-8859-1?Q?Eric_B=E9langer?=)
Date: Sat, 6 Dec 2008 22:45:15 -0500 (EST)
Subject: [pacman-dev] PKGBUILD.proto - Take2
In-Reply-To: <20081207135838.49912e8a@judfilm.net>
References: <20081207135838.49912e8a@judfilm.net>
Message-ID: <Pine.LNX.4.64.0812062241370.8601@localhost.localdomain>

On Sun, 7 Dec 2008, Jud wrote:

> Hi,
>
> Edit: Take2, changed the install line
>
> Dan suggested I send this to the pacman-dev list.
>
> After completing some research and asking alot of questions I present
> some minor changes to PKGBUILD.proto supplied as a .diff to be merged
> after your approval. I believe it helps the intended audience create a
> better PKGBUILD in less time according to the latest Arch Packaging
> Standards.
>
> Cheers
> Jud
>
> Inline:
> --- PKGBUILD.proto	2008-12-05 23:32:33.000005000 +1000
> +++ PKGBUILD.proto.new	2008-12-07 13:49:43.889569135 +1000
> @@ -3,13 +3,15 @@
> # NOTE: Please fill out the license field for your package! If it is
> unknown, # then please put 'unknown'.
>
> +
> # Contributor: Your Name <youremail at domain.com>
> +
> pkgname=NAME
> -pkgver=VERSION
> +pkgver=VERSION # Note: if pkgver is '0.99-10' then use an underscore,
> i.e. '0.99_10' pkgrel=1
> pkgdesc=""
> -arch=()
> -url=""
> +url="http://ADDRESS/"
> +arch=('i686' 'x86_64')
> license=('GPL')
> groups=()
> depends=()
> @@ -20,17 +22,13 @@
> replaces=()
> backup=()
> options=()
> -install=
> -source=($pkgname-$pkgver.tar.gz)
> -noextract=()
> -md5sums=() #generate with 'makepkg -g'
> +install=${pkgname}.install
> +source=(http://ADDRESS/TO/FILE/${pkgname}-${pkgver}.tar.gz)
> +md5sums=() # Generate with 'makepkg -g'
>
> build() {
> -  cd "$srcdir/$pkgname-$pkgver"
> -
> -  ./configure --prefix=/usr
> +  cd ${srcdir}${pkgname}-${pkgver}
> +  ./configure --prefix=usr
>   make || return 1
> -  make DESTDIR="$pkgdir/" install
> +  make DESTDIR=${pkgdir} install || return 1
> }
> -
> -# vim:set ts=2 sw=2 et:
>
>

Please do the fixes I mentionned here: 
http://archlinux.org/pipermail/pacman-dev/2008-December/007740.html

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From allan at archlinux.org  Sat Dec  6 23:07:47 2008
From: allan at archlinux.org (Allan McRae)
Date: Sun, 07 Dec 2008 14:07:47 +1000
Subject: [pacman-dev] PKGBUILD.proto - Take2
In-Reply-To: <20081207135838.49912e8a@judfilm.net>
References: <20081207135838.49912e8a@judfilm.net>
Message-ID: <BAY102-DAV36597D1CEA1A71DF8156C86FC0@phx.gbl>

Jud wrote:
> Hi,
>
> Edit: Take2, changed the install line
>
> Dan suggested I send this to the pacman-dev list.
>
> After completing some research and asking alot of questions I present
> some minor changes to PKGBUILD.proto supplied as a .diff to be merged
> after your approval. I believe it helps the intended audience create a
> better PKGBUILD in less time according to the latest Arch Packaging
> Standards.
>
> Cheers
> Jud
>
> Inline:
> --- PKGBUILD.proto	2008-12-05 23:32:33.000005000 +1000
> +++ PKGBUILD.proto.new	2008-12-07 13:49:43.889569135 +1000
> @@ -3,13 +3,15 @@
>  # NOTE: Please fill out the license field for your package! If it is
> unknown, # then please put 'unknown'.
>  
> +
>  # Contributor: Your Name <youremail at domain.com>
> +
>  pkgname=NAME
> -pkgver=VERSION
> +pkgver=VERSION # Note: if pkgver is '0.99-10' then use an underscore,
> i.e. '0.99_10' pkgrel=1
>   

Can we make this shorter.  I.e "# Note: Cannot contain hyphens."   This 
really should get documented in the man page.

>  pkgdesc=""
> -arch=()
> -url=""
> +url="http://ADDRESS/"
> +arch=('i686' 'x86_64')
>   

Re-ordering why?  Also, adding Arch's standard arch values is distro 
specific.  We try and avoid that in the pacman code.

>  license=('GPL')
>  groups=()
>  depends=()
> @@ -20,17 +22,13 @@
>  replaces=()
>  backup=()
>  options=()
> -install=
> -source=($pkgname-$pkgver.tar.gz)
> -noextract=()
> -md5sums=() #generate with 'makepkg -g'
> +install=${pkgname}.install
> +source=(http://ADDRESS/TO/FILE/${pkgname}-${pkgver}.tar.gz)
> +md5sums=() # Generate with 'makepkg -g'
>   

Why remove noextract?  The prototype should have every field.

>  
>  build() {
> -  cd "$srcdir/$pkgname-$pkgver"
> -
> -  ./configure --prefix=/usr
> +  cd ${srcdir}${pkgname}-${pkgver}
>   

Still missing a / in the middle of this line and the quotes are needed 
for people who build in directories with spaces.

> +  ./configure --prefix=usr
>    make || return 1
> -  make DESTDIR="$pkgdir/" install
> +  make DESTDIR=${pkgdir} install || return 1
>   

The trailing slash was added here deliberately as some makefiles suck.  
Also, quotes are needed.

>  }
> -
> -# vim:set ts=2 sw=2 et:
>   

Allan




From dpmcgee at gmail.com  Sat Dec  6 23:47:38 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Sat, 6 Dec 2008 22:47:38 -0600
Subject: [pacman-dev] PKGBUILD.proto - Take2
In-Reply-To: <BAY102-DAV36597D1CEA1A71DF8156C86FC0@phx.gbl>
References: <20081207135838.49912e8a@judfilm.net>
	<BAY102-DAV36597D1CEA1A71DF8156C86FC0@phx.gbl>
Message-ID: <449c10960812062047k22afd039v8923094949047862@mail.gmail.com>

I agree with Allan on pretty much everything he said below. I'll add a
few things too.

On Sat, Dec 6, 2008 at 10:07 PM, Allan McRae <allan at archlinux.org> wrote:
> Jud wrote:
>> --- PKGBUILD.proto      2008-12-05 23:32:33.000005000 +1000
>> +++ PKGBUILD.proto.new  2008-12-07 13:49:43.889569135 +1000
>> @@ -3,13 +3,15 @@
>>  # NOTE: Please fill out the license field for your package! If it is
>> unknown, # then please put 'unknown'.
>>  +
>>  # Contributor: Your Name <youremail at domain.com>
>> +
>>  pkgname=NAME
>> -pkgver=VERSION
>> +pkgver=VERSION # Note: if pkgver is '0.99-10' then use an underscore,
>> i.e. '0.99_10' pkgrel=1
>>
>
> Can we make this shorter.  I.e "# Note: Cannot contain hyphens."   This
> really should get documented in the man page.
Agreed on both counts. Patches welcome. :P

>>  pkgdesc=""
>> -arch=()
>> -url=""
>> +url="http://ADDRESS/"
>> +arch=('i686' 'x86_64')
>>
>
> Re-ordering why?  Also, adding Arch's standard arch values is distro
> specific.  We try and avoid that in the pacman code.
Mostly agreed. This is a shell script so order really doesn't matter.
I feel like default arch values might be nice to show what we are
going for, however. I'm really impartial on this though, and I'd defer
to Allan on this decision as I've made him Mr. Makepkg. :)

>>  license=('GPL')
>>  groups=()
>>  depends=()
>> @@ -20,17 +22,13 @@
>>  replaces=()
>>  backup=()
>>  options=()
>> -install=
>> -source=($pkgname-$pkgver.tar.gz)
>> -noextract=()
>> -md5sums=() #generate with 'makepkg -g'
>> +install=${pkgname}.install
>> +source=(http://ADDRESS/TO/FILE/${pkgname}-${pkgver}.tar.gz)
>> +md5sums=() # Generate with 'makepkg -g'
>>
>
> Why remove noextract?  The prototype should have every field.
Agreed.

>>   build() {
>> -  cd "$srcdir/$pkgname-$pkgver"
>> -
>> -  ./configure --prefix=/usr
>> +  cd ${srcdir}${pkgname}-${pkgver}
>>
>
> Still missing a / in the middle of this line and the quotes are needed for
> people who build in directories with spaces.
And why the 6 unnecessary characters? {}{}{} is a waste IMO as you
didn't really give a good reason to change what is already there.
Please leave this line as-is.

>> +  ./configure --prefix=usr
>>   make || return 1
>> -  make DESTDIR="$pkgdir/" install
>> +  make DESTDIR=${pkgdir} install || return 1
>>
>
> The trailing slash was added here deliberately as some makefiles suck.
>  Also, quotes are needed.
Yep.

We may sound harsh but we try to do reviews of every patch- don't get
discouraged, but please try and incorporate our feedback.

-Dan

From dan at archlinux.org  Sun Dec  7 00:33:21 2008
From: dan at archlinux.org (Dan McGee)
Date: Sat,  6 Dec 2008 23:33:21 -0600
Subject: [pacman-dev] [PATCH] makepkg: ensure PKGBUILD does not contain CRLF
	characters
Message-ID: <1228628001-21945-1-git-send-email-dan@archlinux.org>

Do a simple check before sourcing the file to ensure we are a valid bash
script.

Signed-off-by: Dan McGee <dan at archlinux.org>
---
 scripts/makepkg.sh.in |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in
index b889625..179746d 100644
--- a/scripts/makepkg.sh.in
+++ b/scripts/makepkg.sh.in
@@ -1354,6 +1354,12 @@ if [ ! -f "$BUILDSCRIPT" ]; then
 		# PKGBUILD passed through a pipe
 		BUILDSCRIPT=/dev/stdin
 	fi
+else
+	crlftest=$(file $BUILDSCRIPT | grep -F 'CRLF' || true)
+	if [ "$crlftest" != "" ]; then
+		error "$(gettext "%s contains CRLF characters and cannot be sourced.")" "$BUILDSCRIPT"
+		exit 1
+	fi
 fi
 
 source "$BUILDSCRIPT"
-- 
1.6.0.4


From jud at judfilm.net  Sun Dec  7 01:51:07 2008
From: jud at judfilm.net (Jud)
Date: Sun, 7 Dec 2008 16:51:07 +1000
Subject: [pacman-dev] PKGBUILD.proto - Take2
In-Reply-To: <449c10960812062047k22afd039v8923094949047862@mail.gmail.com>
References: <20081207135838.49912e8a@judfilm.net>
	<BAY102-DAV36597D1CEA1A71DF8156C86FC0@phx.gbl>
	<449c10960812062047k22afd039v8923094949047862@mail.gmail.com>
Message-ID: <20081207165107.28bd5fb6@judfilm.net>

On Sat, 6 Dec 2008 22:47:38 -0600
"Dan McGee" <dpmcgee at gmail.com> wrote:

> I agree with Allan on pretty much everything he said below. I'll add a
> few things too.
> 
> On Sat, Dec 6, 2008 at 10:07 PM, Allan McRae <allan at archlinux.org>
> wrote:
> > Jud wrote:
> >> --- PKGBUILD.proto      2008-12-05 23:32:33.000005000 +1000
> >> +++ PKGBUILD.proto.new  2008-12-07 13:49:43.889569135 +1000
> >> @@ -3,13 +3,15 @@
> >>  # NOTE: Please fill out the license field for your package! If it
> >> is unknown, # then please put 'unknown'.
> >>  +
> >>  # Contributor: Your Name <youremail at domain.com>
> >> +
> >>  pkgname=NAME
> >> -pkgver=VERSION
> >> +pkgver=VERSION # Note: if pkgver is '0.99-10' then use an
> >> underscore, i.e. '0.99_10' pkgrel=1
> >>
> >
> > Can we make this shorter.  I.e "# Note: Cannot contain hyphens."
> > This really should get documented in the man page.
> Agreed on both counts. Patches welcome. :P
> 
> >>  pkgdesc=""
> >> -arch=()
> >> -url=""
> >> +url="http://ADDRESS/"
> >> +arch=('i686' 'x86_64')
> >>
> >
> > Re-ordering why?  Also, adding Arch's standard arch values is distro
> > specific.  We try and avoid that in the pacman code.
> Mostly agreed. This is a shell script so order really doesn't matter.
> I feel like default arch values might be nice to show what we are
> going for, however. I'm really impartial on this though, and I'd defer
> to Allan on this decision as I've made him Mr. Makepkg. :)
> 
> >>  license=('GPL')
> >>  groups=()
> >>  depends=()
> >> @@ -20,17 +22,13 @@
> >>  replaces=()
> >>  backup=()
> >>  options=()
> >> -install=
> >> -source=($pkgname-$pkgver.tar.gz)
> >> -noextract=()
> >> -md5sums=() #generate with 'makepkg -g'
> >> +install=${pkgname}.install
> >> +source=(http://ADDRESS/TO/FILE/${pkgname}-${pkgver}.tar.gz)
> >> +md5sums=() # Generate with 'makepkg -g'
> >>
> >
> > Why remove noextract?  The prototype should have every field.
> Agreed.
> 
> >>   build() {
> >> -  cd "$srcdir/$pkgname-$pkgver"
> >> -
> >> -  ./configure --prefix=/usr
> >> +  cd ${srcdir}${pkgname}-${pkgver}
> >>
> >
> > Still missing a / in the middle of this line and the quotes are
> > needed for people who build in directories with spaces.
> And why the 6 unnecessary characters? {}{}{} is a waste IMO as you
> didn't really give a good reason to change what is already there.
> Please leave this line as-is.
> 
> >> +  ./configure --prefix=usr
> >>   make || return 1
> >> -  make DESTDIR="$pkgdir/" install
> >> +  make DESTDIR=${pkgdir} install || return 1
> >>
> >
> > The trailing slash was added here deliberately as some makefiles
> > suck. Also, quotes are needed.
> Yep.
> 
> We may sound harsh but we try to do reviews of every patch- don't get
> discouraged, but please try and incorporate our feedback.
> 
> -Dan
> _______________________________________________
> pacman-dev mailing list
> pacman-dev at archlinux.org
> http://archlinux.org/mailman/listinfo/pacman-dev

Thanks everyone for the quick replies. My only intention is to make
Arch and pacman better.

Everything added in is from other PKGBUILDs and what others have said to
do over the past few weeks, as you could guess it has become very
confusing. So I am glad to get this all straightened out and everyone
can learn and make Arch and Pacman better. Personally, I couldn't give a
c#%p what is in the PKGBUILD as long as everyone (Devs included) are on
the same page.


arch=() # For Arch Linux: arch=('i686' 'x86_64')
Would this line be ok since I'd say the over whelming usage would be
Arch (and derivatives) based?


url line
Only following the "Options and Directives" from the "PKGBUILD(5)
Manual Page" Which I believe makes sense if you consider the old "Who?
What? Where? When?" [1]. Also I think it reads and looks much better.

If pacman is trying to be Distro independent wouldn't it be
better to change line 9 from PKGBUILD.5.txt "PKGBUILD - Arch Linux
package build description file"? I would be happy to help re-write
the whole file.


Do these following lines have the same result?
cd "$srcdir/$pkgname-$pkgver"
cd ${srcdir}/${pkgname}-${pkgver}

From the PKGBUILDs I have studied (over 500 in the past few weeks) I
would say it would be about 50/50 split (some only have 
'cd $srcdir/$pkgname-$pkgver')

Anyways I'm not phased which way the knife falls and I hope we can
sort this all out quickly.

Cheers
Jud


[1] http://en.wikipedia.org/wiki/5_Ws

From allan at archlinux.org  Sun Dec  7 02:11:31 2008
From: allan at archlinux.org (Allan McRae)
Date: Sun, 07 Dec 2008 17:11:31 +1000
Subject: [pacman-dev] PKGBUILD.proto - Take2
In-Reply-To: <20081207165107.28bd5fb6@judfilm.net>
References: <20081207135838.49912e8a@judfilm.net>	<BAY102-DAV36597D1CEA1A71DF8156C86FC0@phx.gbl>	<449c10960812062047k22afd039v8923094949047862@mail.gmail.com>
	<20081207165107.28bd5fb6@judfilm.net>
Message-ID: <BAY102-DAV7B8CF8E1CA25A7B032FE986FC0@phx.gbl>

Jud wrote:
> Thanks everyone for the quick replies. My only intention is to make
> Arch and pacman better.
>
> Everything added in is from other PKGBUILDs and what others have said to
> do over the past few weeks, as you could guess it has become very
> confusing. So I am glad to get this all straightened out and everyone
> can learn and make Arch and Pacman better. Personally, I couldn't give a
> c#%p what is in the PKGBUILD as long as everyone (Devs included) are on
> the same page.
>   

Stylistic guidelines are almost never agreed on by anyone and are not 
particularly enforceable.  As an example, search for C code formating 
guidelines on the web...

>
> arch=() # For Arch Linux: arch=('i686' 'x86_64')
> Would this line be ok since I'd say the over whelming usage would be
> Arch (and derivatives) based?
>
>   

I would like it left as just "arch=()". There is an example in the 
PKGBUILD man page.

> url line
> Only following the "Options and Directives" from the "PKGBUILD(5)
> Manual Page" Which I believe makes sense if you consider the old "Who?
> What? Where? When?" [1]. Also I think it reads and looks much better.
>
> If pacman is trying to be Distro independent wouldn't it be
> better to change line 9 from PKGBUILD.5.txt "PKGBUILD - Arch Linux
> package build description file"? I would be happy to help re-write
> the whole file.
>
>   

Yes, that should be fixed.  I do not think it needs that much of a 
rewrite though.

> Do these following lines have the same result?
> cd "$srcdir/$pkgname-$pkgver"
> cd ${srcdir}/${pkgname}-${pkgver}
>   

Well, the quotes in the top line allow $srcdir to have spaces.  
Otherwise they are the some thing.  The {}'s are redundant in this case.

> >From the PKGBUILDs I have studied (over 500 in the past few weeks) I
> would say it would be about 50/50 split (some only have 
> 'cd $srcdir/$pkgname-$pkgver')
>   

I have to admit that I do not use quotes because I don't care about 
people who use spaces in directory names...  But the prototype probably 
should keep the quotes in.

> Anyways I'm not phased which way the knife falls and I hope we can
> sort this all out quickly.
>
> Cheers
> Jud

Allan



From geoffroy.carrier at koon.fr  Sun Dec  7 03:43:55 2008
From: geoffroy.carrier at koon.fr (Geoffroy Carrier)
Date: Sun, 7 Dec 2008 09:43:55 +0100
Subject: [pacman-dev] PKGBUILD.proto - Take2
In-Reply-To: <BAY102-DAV7B8CF8E1CA25A7B032FE986FC0@phx.gbl>
References: <20081207135838.49912e8a@judfilm.net>
	<BAY102-DAV36597D1CEA1A71DF8156C86FC0@phx.gbl>
	<449c10960812062047k22afd039v8923094949047862@mail.gmail.com>
	<20081207165107.28bd5fb6@judfilm.net>
	<BAY102-DAV7B8CF8E1CA25A7B032FE986FC0@phx.gbl>
Message-ID: <c9084260812070043x1a5b170ajd9d531f1fa05a481@mail.gmail.com>

On Sun, Dec 7, 2008 at 08:11, Allan McRae <allan at archlinux.org> wrote:
> I would like it left as just "arch=()". There is an example in the PKGBUILD
> man page.
Generic concern: I often like to start from the PKGBUILD.proto.
For now it only has comments on the first lines, which can be
destroyed with V4jd in vim, and in the md5sums line, which I destroy
entirely too (and generate with makepkg -g >> PKGBUILD).
PLEASE, PLEASE don't add documentation that will take time to remove
and provide indications that will only be useful on the first uses.
I'd rather see them in a HOWTO.

-- 
Geoffroy Carrier

From allan at archlinux.org  Sun Dec  7 05:33:29 2008
From: allan at archlinux.org (Allan McRae)
Date: Sun, 07 Dec 2008 20:33:29 +1000
Subject: [pacman-dev] [PATCH] makepkg: ensure PKGBUILD does not contain
 CRLF	characters
In-Reply-To: <1228628001-21945-1-git-send-email-dan@archlinux.org>
References: <1228628001-21945-1-git-send-email-dan@archlinux.org>
Message-ID: <BAY102-DAV31572D0BFC210DDA5713086FC0@phx.gbl>

Dan McGee wrote:
> Do a simple check before sourcing the file to ensure we are a valid bash
> script.
>
> Signed-off-by: Dan McGee <dan at archlinux.org>
> ---
>  scripts/makepkg.sh.in |    6 ++++++
>  1 files changed, 6 insertions(+), 0 deletions(-)
>
> diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in
> index b889625..179746d 100644
> --- a/scripts/makepkg.sh.in
> +++ b/scripts/makepkg.sh.in
> @@ -1354,6 +1354,12 @@ if [ ! -f "$BUILDSCRIPT" ]; then
>  		# PKGBUILD passed through a pipe
>  		BUILDSCRIPT=/dev/stdin
>  	fi
> +else
> +	crlftest=$(file $BUILDSCRIPT | grep -F 'CRLF' || true)
> +	if [ "$crlftest" != "" ]; then
> +		error "$(gettext "%s contains CRLF characters and cannot be sourced.")" "$BUILDSCRIPT"
> +		exit 1
> +	fi
>  fi
>  
>  source "$BUILDSCRIPT"
>   

Do you want to attempt a dos2unix there (with the error message 
downgraded to a warning) before failing?

Allan




From allan at archlinux.org  Sun Dec  7 06:19:00 2008
From: allan at archlinux.org (Allan McRae)
Date: Sun,  7 Dec 2008 21:19:00 +1000
Subject: [pacman-dev] [PATCH] makepkg: Introduce restrict option
Message-ID: <1228648740-11620-1-git-send-email-allan@archlinux.org>

The restrict option, combined with the RESTRICTED_FILES variable,
allows makepkg to automatically remove common confliting files
(e.g. /usr/share/info/dir).

Signed-off-by: Allan McRae <allan at archlinux.org>
---
 doc/makepkg.conf.5.txt |    8 ++++++++
 etc/makepkg.conf.in    |    7 +++++--
 scripts/makepkg.sh.in  |    8 ++++++--
 3 files changed, 19 insertions(+), 4 deletions(-)

diff --git a/doc/makepkg.conf.5.txt b/doc/makepkg.conf.5.txt
index 18dbf35..84e8192 100644
--- a/doc/makepkg.conf.5.txt
+++ b/doc/makepkg.conf.5.txt
@@ -125,6 +125,9 @@ Options
 	*zipman*;;
 		Compress manual (man and info) pages with gzip.
 
+	*restrict*;;
+		Remove restricted files from the package.
+
 **INTEGRITY_CHECK=(**check1 ...**)**::
 	File integrity checks to use. Multiple checks may be specified; this
 	affects both generation and checking. The current valid options are:
@@ -150,6 +153,11 @@ Options
 	to this array. *NOTE:* Do not add the leading slash to the directory
 	name.
 
+**RESTRICTED_FILES=(**usr/{,share}/info/dir ...**)**::
+	If "restrict" is specified in the OPTIONS array, this variable will
+	instruct makepkg which files to remove from the package. This is
+	useful for index files that are added to by multiple packages.
+
 **PKGDEST=**"/path/to/folder"::
 	If this value is not set, packages will by default be placed in the
 	current directory (location of the linkman:PKGBUILD[5]). Many people
diff --git a/etc/makepkg.conf.in b/etc/makepkg.conf.in
index 82722be..545a215 100644
--- a/etc/makepkg.conf.in
+++ b/etc/makepkg.conf.in
@@ -58,7 +58,7 @@ BUILDENV=(fakeroot !distcc color !ccache !xdelta)
 #   These are default values for the options=() settings
 #########################################################################
 #
-# Default: OPTIONS=(strip !docs libtool emptydirs zipman)
+# Default: OPTIONS=(strip !docs libtool emptydirs zipman restrict)
 #  A negated option will do the opposite of the comments below.
 #
 #-- strip:     Strip symbols from binaries/libraries
@@ -66,8 +66,9 @@ BUILDENV=(fakeroot !distcc color !ccache !xdelta)
 #-- libtool:   Leave libtool (.la) files in packages
 #-- emptydirs: Leave empty directories in packages
 #-- zipman:    Compress manual (man and info) pages with gzip
+#-- restrict:  Remove files sepecified below from package
 #
-OPTIONS=(strip !docs libtool emptydirs zipman)
+OPTIONS=(strip !docs libtool emptydirs zipman restrict)
 
 #-- File integrity checks to use. Valid: md5, sha1, sha256, sha384, sha512
 INTEGRITY_CHECK=(md5)
@@ -77,6 +78,8 @@ MAN_DIRS=({usr{,/local}{,/share},opt/*}/{man,info})
 DOC_DIRS=(usr/{,share/}{doc,gtk-doc} opt/*/{doc,gtk-doc})
 #-- Directories to be searched for the strip option (if option set correctly above)
 STRIP_DIRS=(bin lib sbin usr/{bin,lib,sbin,local/{bin,lib,sbin}} opt/*/{bin,lib,sbin})
+#-- Files to be removed from all packages
+RESTRICTED_FILES=(usr/{,share}/info/dir)
 
 #########################################################################
 # PACKAGE OUTPUT
diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in
index bf6524d..ea27883 100644
--- a/scripts/makepkg.sh.in
+++ b/scripts/makepkg.sh.in
@@ -41,7 +41,7 @@ confdir='@sysconfdir@'
 startdir="$PWD"
 srcdir="$startdir/src"
 pkgdir="$startdir/pkg"
-packaging_options=('strip' 'docs' 'libtool' 'emptydirs' 'zipman')
+packaging_options=('strip' 'docs' 'libtool' 'emptydirs' 'zipman' 'restrict')
 other_options=('ccache' 'distcc' 'makeflags' 'force')
 readonly -a packaging_options other_options
 
@@ -792,7 +792,6 @@ tidy_install() {
 		done
 	fi
 
-
 	if [ "$(check_option strip)" = "y" ]; then
 		msg2 "$(gettext "Stripping debugging symbols from binaries and libraries...")"
 		local binary
@@ -818,6 +817,11 @@ tidy_install() {
 		find . ! -type d -name "*.la" -exec rm -f -- '{}' \;
 	fi
 
+	if [ "$(check_option restrict)" = "y" -a -n "RESTRICTED_FILES" ]; then
+		msg2 "$(gettext "Removing restricted files...")"
+		rm -f ${RESTRICTED_FILES[@]}
+	fi
+
 	if [ "$(check_option emptydirs)" = "n" ]; then
 		msg2 "$(gettext "Removing empty directories...")"
 		find . -depth -type d -empty -delete
-- 
1.6.0.4


From allan at archlinux.org  Sun Dec  7 06:23:12 2008
From: allan at archlinux.org (Allan McRae)
Date: Sun, 07 Dec 2008 21:23:12 +1000
Subject: [pacman-dev] [PATCH] makepkg: Introduce restrict option
In-Reply-To: <1228648740-11620-1-git-send-email-allan@archlinux.org>
References: <1228648740-11620-1-git-send-email-allan@archlinux.org>
Message-ID: <BAY102-DAV791E7BBA92DC9A6A4748286FC0@phx.gbl>

Allan McRae wrote:
> The restrict option, combined with the RESTRICTED_FILES variable,
> allows makepkg to automatically remove common confliting files
> (e.g. /usr/share/info/dir).
>
> Signed-off-by: Allan McRae <allan at archlinux.org>
> ---
>   

Bonus points if someone comes up with a better name for the options.....

Note that this patch made off my working branch which has a couple of 
patches which have not made mainline yet and overlap with some parts of 
code adjusted here.

>  doc/makepkg.conf.5.txt |    8 ++++++++
>  etc/makepkg.conf.in    |    7 +++++--
>  scripts/makepkg.sh.in  |    8 ++++++--
>  3 files changed, 19 insertions(+), 4 deletions(-)
>
> diff --git a/doc/makepkg.conf.5.txt b/doc/makepkg.conf.5.txt
> index 18dbf35..84e8192 100644
> --- a/doc/makepkg.conf.5.txt
> +++ b/doc/makepkg.conf.5.txt
> @@ -125,6 +125,9 @@ Options
>  	*zipman*;;
>  		Compress manual (man and info) pages with gzip.
>  
> +	*restrict*;;
> +		Remove restricted files from the package.
> +
>  **INTEGRITY_CHECK=(**check1 ...**)**::
>  	File integrity checks to use. Multiple checks may be specified; this
>  	affects both generation and checking. The current valid options are:
> @@ -150,6 +153,11 @@ Options
>  	to this array. *NOTE:* Do not add the leading slash to the directory
>  	name.
>  
> +**RESTRICTED_FILES=(**usr/{,share}/info/dir ...**)**::
> +	If "restrict" is specified in the OPTIONS array, this variable will
> +	instruct makepkg which files to remove from the package. This is
> +	useful for index files that are added to by multiple packages.
> +
>  **PKGDEST=**"/path/to/folder"::
>  	If this value is not set, packages will by default be placed in the
>  	current directory (location of the linkman:PKGBUILD[5]). Many people
> diff --git a/etc/makepkg.conf.in b/etc/makepkg.conf.in
> index 82722be..545a215 100644
> --- a/etc/makepkg.conf.in
> +++ b/etc/makepkg.conf.in
> @@ -58,7 +58,7 @@ BUILDENV=(fakeroot !distcc color !ccache !xdelta)
>  #   These are default values for the options=() settings
>  #########################################################################
>  #
> -# Default: OPTIONS=(strip !docs libtool emptydirs zipman)
> +# Default: OPTIONS=(strip !docs libtool emptydirs zipman restrict)
>  #  A negated option will do the opposite of the comments below.
>  #
>  #-- strip:     Strip symbols from binaries/libraries
> @@ -66,8 +66,9 @@ BUILDENV=(fakeroot !distcc color !ccache !xdelta)
>  #-- libtool:   Leave libtool (.la) files in packages
>  #-- emptydirs: Leave empty directories in packages
>  #-- zipman:    Compress manual (man and info) pages with gzip
> +#-- restrict:  Remove files sepecified below from package
>  #
> -OPTIONS=(strip !docs libtool emptydirs zipman)
> +OPTIONS=(strip !docs libtool emptydirs zipman restrict)
>  
>  #-- File integrity checks to use. Valid: md5, sha1, sha256, sha384, sha512
>  INTEGRITY_CHECK=(md5)
> @@ -77,6 +78,8 @@ MAN_DIRS=({usr{,/local}{,/share},opt/*}/{man,info})
>  DOC_DIRS=(usr/{,share/}{doc,gtk-doc} opt/*/{doc,gtk-doc})
>  #-- Directories to be searched for the strip option (if option set correctly above)
>  STRIP_DIRS=(bin lib sbin usr/{bin,lib,sbin,local/{bin,lib,sbin}} opt/*/{bin,lib,sbin})
> +#-- Files to be removed from all packages
> +RESTRICTED_FILES=(usr/{,share}/info/dir)
>  
>  #########################################################################
>  # PACKAGE OUTPUT
> diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in
> index bf6524d..ea27883 100644
> --- a/scripts/makepkg.sh.in
> +++ b/scripts/makepkg.sh.in
> @@ -41,7 +41,7 @@ confdir='@sysconfdir@'
>  startdir="$PWD"
>  srcdir="$startdir/src"
>  pkgdir="$startdir/pkg"
> -packaging_options=('strip' 'docs' 'libtool' 'emptydirs' 'zipman')
> +packaging_options=('strip' 'docs' 'libtool' 'emptydirs' 'zipman' 'restrict')
>  other_options=('ccache' 'distcc' 'makeflags' 'force')
>  readonly -a packaging_options other_options
>  
> @@ -792,7 +792,6 @@ tidy_install() {
>  		done
>  	fi
>  
> -
>  	if [ "$(check_option strip)" = "y" ]; then
>  		msg2 "$(gettext "Stripping debugging symbols from binaries and libraries...")"
>  		local binary
> @@ -818,6 +817,11 @@ tidy_install() {
>  		find . ! -type d -name "*.la" -exec rm -f -- '{}' \;
>  	fi
>  
> +	if [ "$(check_option restrict)" = "y" -a -n "RESTRICTED_FILES" ]; then
> +		msg2 "$(gettext "Removing restricted files...")"
> +		rm -f ${RESTRICTED_FILES[@]}
> +	fi
> +
>  	if [ "$(check_option emptydirs)" = "n" ]; then
>  		msg2 "$(gettext "Removing empty directories...")"
>  		find . -depth -type d -empty -delete
>   




From dpmcgee at gmail.com  Sun Dec  7 09:35:54 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Sun, 7 Dec 2008 08:35:54 -0600
Subject: [pacman-dev] [PATCH] makepkg: ensure PKGBUILD does not contain
	CRLF characters
In-Reply-To: <BAY102-DAV31572D0BFC210DDA5713086FC0@phx.gbl>
References: <1228628001-21945-1-git-send-email-dan@archlinux.org>
	<BAY102-DAV31572D0BFC210DDA5713086FC0@phx.gbl>
Message-ID: <449c10960812070635q797cdae3j58ec07eab970d9bc@mail.gmail.com>

On Sun, Dec 7, 2008 at 4:33 AM, Allan McRae <allan at archlinux.org> wrote:
> Dan McGee wrote:
>>
>> Do a simple check before sourcing the file to ensure we are a valid bash
>> script.
>>
>> Signed-off-by: Dan McGee <dan at archlinux.org>
>> ---
>>  scripts/makepkg.sh.in |    6 ++++++
>>  1 files changed, 6 insertions(+), 0 deletions(-)
>>
>> diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in
>> index b889625..179746d 100644
>> --- a/scripts/makepkg.sh.in
>> +++ b/scripts/makepkg.sh.in
>> @@ -1354,6 +1354,12 @@ if [ ! -f "$BUILDSCRIPT" ]; then
>>                # PKGBUILD passed through a pipe
>>                BUILDSCRIPT=/dev/stdin
>>        fi
>> +else
>> +       crlftest=$(file $BUILDSCRIPT | grep -F 'CRLF' || true)
>> +       if [ "$crlftest" != "" ]; then
>> +               error "$(gettext "%s contains CRLF characters and cannot
>> be sourced.")" "$BUILDSCRIPT"
>> +               exit 1
>> +       fi
>>  fi
>>   source "$BUILDSCRIPT"
>>
>
> Do you want to attempt a dos2unix there (with the error message downgraded
> to a warning) before failing?

I'm not sure everyone would have dos2unix installed, so that makes me
shy away from this a bit. In addition, I feel like actually modifying
the PKGBUILD should be a rare case. We do it in the case of VCS
PKGBUILDs, but that is it, and I'd rather not destroy someone's work.

-Dan

From dpmcgee at gmail.com  Sun Dec  7 16:18:32 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Sun, 7 Dec 2008 15:18:32 -0600
Subject: [pacman-dev] GPG work
Message-ID: <449c10960812071318v7a7fb427nb19f4f5726496234@mail.gmail.com>

I did quite a bit more work with GPG today. I wrapped my head around
GPGME, which presents a nice C interface to the GPG stuff so we are
now a lot closer to a working implementation:
http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=shortlog;h=refs/heads/newgpg

>From the script side of things, I didn't change much. The libalpm code
has changed considerably, and there is still a lot of room for
improvement. Let me know if you guys have questions.

-Dan

From allan at archlinux.org  Sun Dec  7 21:35:41 2008
From: allan at archlinux.org (Allan McRae)
Date: Mon, 08 Dec 2008 12:35:41 +1000
Subject: [pacman-dev] makepkg: package splitting
Message-ID: <BAY102-DAV5C6EAD449827BDCFC3F5086FD0@phx.gbl>

I have finished implementing the basics of package splitting in 
makepkg.  Everything works as expected when just making packages and 
capturing logs but there is some small output issues that need tidying 
up and checking that other functionality work (e.g. repackaging is an 
issue....)

Anyway you can see the work on my "splitpkg" branch:
http://dev.archlinux.org/~allan/gitweb/gitweb.cgi?p=pacman.git;a=shortlog;h=refs/heads/splitpkg

For your interest, I have attached the same package written in three 
different ways to show you what makepkg can now do:

PKGBUILD.1 - Traditional build
PKGBUILD.2 - Splitting the packaging step from build step.  Limits 
fakeroot usage
PKGBUILD.3 - Package splitting


-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: PKGBUILD.1
URL: <http://archlinux.org/pipermail/pacman-dev/attachments/20081208/c8c9e822/attachment.diff>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: PKGBUILD.2
URL: <http://archlinux.org/pipermail/pacman-dev/attachments/20081208/c8c9e822/attachment-0001.diff>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: PKGBUILD.3
URL: <http://archlinux.org/pipermail/pacman-dev/attachments/20081208/c8c9e822/attachment-0002.diff>

From dan at archlinux.org  Sun Dec  7 23:13:16 2008
From: dan at archlinux.org (Dan McGee)
Date: Sun,  7 Dec 2008 23:13:16 -0500 (EST)
Subject: [pacman-dev] [GIT] The official pacman repository branch, maint,
	updated. v3.2.1-25-g78cf32e
Message-ID: <20081208041317.079642A1F2@gerolde.archlinux.org>

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "The official pacman repository".

The branch, maint has been updated
       via  78cf32e194a1a58c6a7ee3d1c10623e668be71d6 (commit)
       via  59776ef306935399b2bf1e9cbdd7a7bfa6df1b49 (commit)
       via  b373b1d16b6235bef2e34a9a21e043418222a813 (commit)
      from  a1f7c83dbf3bce492163362d2896e3a4176be616 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
-----------------------------------------------------------------------

Summary of changes:
 contrib/pactree       |    3 +--
 doc/PKGBUILD.5.txt    |    2 +-
 doc/repo-add.8.txt    |    4 ++--
 scripts/makepkg.sh.in |    4 ++++
 4 files changed, 8 insertions(+), 5 deletions(-)


hooks/post-receive
--
The official pacman repository

From dan at archlinux.org  Sun Dec  7 23:13:17 2008
From: dan at archlinux.org (Dan McGee)
Date: Sun,  7 Dec 2008 23:13:17 -0500 (EST)
Subject: [pacman-dev] [GIT] The official pacman repository branch, master,
	updated. v3.2.1-59-gbd62827
Message-ID: <20081208041317.1E1B72A1D6@gerolde.archlinux.org>

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "The official pacman repository".

The branch, master has been updated
       via  bd628274cc8db81704860e69894fcd217c2068d0 (commit)
       via  9ae7eb1292a42eb568fd982f32d3dca5b9794a2d (commit)
       via  818fae320f22ea758765a55b3a3a98e2f0a66a45 (commit)
       via  69be73f68cc770ddcbe388af96e32f8bba7886f7 (commit)
       via  78cf32e194a1a58c6a7ee3d1c10623e668be71d6 (commit)
       via  59776ef306935399b2bf1e9cbdd7a7bfa6df1b49 (commit)
       via  b373b1d16b6235bef2e34a9a21e043418222a813 (commit)
      from  61c6552862345cb155903cd1566f1cef5c527a94 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
commit bd628274cc8db81704860e69894fcd217c2068d0
Merge: 9ae7eb1292a42eb568fd982f32d3dca5b9794a2d 78cf32e194a1a58c6a7ee3d1c10623e668be71d6
Author: Dan McGee <dan at archlinux.org>
Date:   Sun Dec 7 22:12:17 2008 -0600

    Merge branch 'maint'

commit 9ae7eb1292a42eb568fd982f32d3dca5b9794a2d
Author: Giovanni Scafora <linuxmania at gmail.com>
Date:   Thu Dec 4 21:03:31 2008 -0600

    Update Italian translation
    
    Signed-off-by: Dan McGee <dan at archlinux.org>

commit 818fae320f22ea758765a55b3a3a98e2f0a66a45
Author: Dan McGee <dan at archlinux.org>
Date:   Sat Dec 6 23:32:34 2008 -0600

    makepkg: ensure PKGBUILD does not contain CRLF characters
    
    Do a simple check before sourcing the file to ensure we are a valid bash
    script.
    
    Signed-off-by: Dan McGee <dan at archlinux.org>

commit 69be73f68cc770ddcbe388af96e32f8bba7886f7
Author: Allan McRae <allan at archlinux.org>
Date:   Fri Nov 14 03:08:52 2008 +1000

    makepkg: several small bits of tidying
    
    1. Do not warn people about missing arch if they are using --ignorearch.
    
    2. Remove unneed reference to bug report about using fakeroot as little
       as possible.  We want to do that, bug report of not.
    
    3. Removes superfluous warning given when building as root. The user
       has already used the "--asroot" flag.
    
    4. Move comment about skipping warning message to above where it occurs
    
    5. Do not warn about skipping source retreval, integrety checks and
       extraction when using --repackage
    
    6. Do not warn about skipping build when using --repackage
    
    7. Move comment about fakeroot usage to above test condition
    
    Signed-off-by: Allan McRae <allan at archlinux.org>
    Signed-off-by: Dan McGee <dan at archlinux.org>

-----------------------------------------------------------------------

Summary of changes:
 contrib/pactree       |    3 +-
 doc/PKGBUILD.5.txt    |    2 +-
 doc/repo-add.8.txt    |    4 +-
 po/it.po              |  141 +++++++++++++++++++++++--------------------------
 scripts/makepkg.sh.in |   41 +++++++--------
 5 files changed, 90 insertions(+), 101 deletions(-)


hooks/post-receive
--
The official pacman repository

From dpmcgee at gmail.com  Sun Dec  7 23:30:31 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Sun, 7 Dec 2008 22:30:31 -0600
Subject: [pacman-dev] pacman 3.2.2 release
Message-ID: <449c10960812072030y5d091b32md682bcec2c67f8e4@mail.gmail.com>

I'd like to do a 3.2.2 release sometime soon (this week, more than
likely). Let me know if there are any outstanding patches for this
release that aren't already in the maint branch as of tonight. I will
put a call for updated translations out in the coming few days.

-Dan

From jud at judfilm.net  Mon Dec  8 02:20:32 2008
From: jud at judfilm.net (Jud)
Date: Mon, 8 Dec 2008 17:20:32 +1000
Subject: [pacman-dev] PKGBUILD.proto - Take3
Message-ID: <20081208172032.6845854a@judfilm.net>

Hi,

Edit: Take3, suggested changes - thanks everyone!

Dan suggested I send this to the pacman-dev list.

After completing some research and asking alot of questions I present
some minor changes to PKGBUILD.proto supplied as a .diff to be merged
after your approval. I believe it helps the intended audience create a
better PKGBUILD in less time according to the latest Arch Packaging
Standards.

Cheers
Jud


Inline:
--- PKGBUILD.proto	2008-12-05 23:32:33.000005000 +1000
+++ PKGBUILD.proto.new	2008-12-08 17:17:15.536302100 +1000
@@ -4,12 +4,13 @@
 # then please put 'unknown'.
 
 # Contributor: Your Name <youremail at domain.com>
+
 pkgname=NAME
-pkgver=VERSION
+pkgver=VERSION # Note: Cannot contain hyphens
 pkgrel=1
 pkgdesc=""
+url="http://ADDRESS/"
 arch=()
-url=""
 license=('GPL')
 groups=()
 depends=()
@@ -20,17 +21,14 @@
 replaces=()
 backup=()
 options=()
-install=
-source=($pkgname-$pkgver.tar.gz)
+install=$pkgname.install
+source=(http://ADDRESS/TO/FILE/$pkgname-$pkgver.tar.gz)
 noextract=()
-md5sums=() #generate with 'makepkg -g'
+md5sums=() # Generate with 'makepkg -g'
 
 build() {
   cd "$srcdir/$pkgname-$pkgver"
-
   ./configure --prefix=/usr
   make || return 1
-  make DESTDIR="$pkgdir/" install
+  make DESTDIR="$pkgdir/" install || return 1
 }
-
-# vim:set ts=2 sw=2 et:
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PKGBUILD.proto.diff
Type: text/x-patch
Size: 881 bytes
Desc: not available
URL: <http://archlinux.org/pipermail/pacman-dev/attachments/20081208/1a83bd58/attachment.bin>

From gerbra at archlinux.de  Mon Dec  8 05:55:53 2008
From: gerbra at archlinux.de (Gerhard Brauer)
Date: Mon, 8 Dec 2008 11:55:53 +0100
Subject: [pacman-dev] GPG work
In-Reply-To: <449c10960812071318v7a7fb427nb19f4f5726496234@mail.gmail.com>
References: <449c10960812071318v7a7fb427nb19f4f5726496234@mail.gmail.com>
Message-ID: <20081208115553.3c2fd3d9@tux1.brauer.lan>

Am Sun, 7 Dec 2008 15:18:32 -0600
schrieb "Dan McGee" <dpmcgee at gmail.com>:

> I did quite a bit more work with GPG today. I wrapped my head around
> GPGME, which presents a nice C interface to the GPG stuff so we are
> now a lot closer to a working implementation:
> http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=shortlog;h=refs/heads/newgpg
> 
> >From the script side of things, I didn't change much. The libalpm
> >code
> has changed considerably, and there is still a lot of room for
> improvement. Let me know if you guys have questions.

With heads/newgpg pacman doesn't check or find the .sig Files. If
starting with --debug i got these debug messages:

debug: md5(/var/cache/pacman/pkg/abook-0.5.6-3-i686.pkg.tar.gz) =79777684f62164 934a1264df66b8fdc6
debug: returning error 35 from gpgme_init : signature directory not configured correctly
debug: installing packages
debug: found cached pkg: /var/cache/pacman/pkg/abook-0.5.6-3-i686.pkg.tar.gz
debug: loading target '/var/cache/pacman/pkg/abook-0.5.6-3-i686.pkg.tar.gz'
debug: no package signature file found

Where or what have i to configure as the "gpgme_init : signature directory"?
My public key is in /root/.gnupg/pubring.gpg. I tried it also with /tmp/testing.gpg but the same error.
AFAI could read the code this may belongs to commit:
http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=commit;h=1a286336147c7d3af42041d26205b9ca3980f459
I see a prog gpgme-config, but don't see what i could do with ;-)

Help ;-)

> -Dan

Gerhard

From dpmcgee at gmail.com  Mon Dec  8 07:34:06 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Mon, 8 Dec 2008 06:34:06 -0600
Subject: [pacman-dev] GPG work
In-Reply-To: <20081208115553.3c2fd3d9@tux1.brauer.lan>
References: <449c10960812071318v7a7fb427nb19f4f5726496234@mail.gmail.com>
	<20081208115553.3c2fd3d9@tux1.brauer.lan>
Message-ID: <449c10960812080434w6ba33471k9fd55ae681b69cf3@mail.gmail.com>

On Mon, Dec 8, 2008 at 4:55 AM, Gerhard Brauer <gerbra at archlinux.de> wrote:
> Am Sun, 7 Dec 2008 15:18:32 -0600
> schrieb "Dan McGee" <dpmcgee at gmail.com>:
>
>> I did quite a bit more work with GPG today. I wrapped my head around
>> GPGME, which presents a nice C interface to the GPG stuff so we are
>> now a lot closer to a working implementation:
>> http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=shortlog;h=refs/heads/newgpg
>>
>> >From the script side of things, I didn't change much. The libalpm
>> >code
>> has changed considerably, and there is still a lot of room for
>> improvement. Let me know if you guys have questions.
>
> With heads/newgpg pacman doesn't check or find the .sig Files. If
> starting with --debug i got these debug messages:
>
> debug: md5(/var/cache/pacman/pkg/abook-0.5.6-3-i686.pkg.tar.gz) =79777684f62164 934a1264df66b8fdc6
> debug: returning error 35 from gpgme_init : signature directory not configured correctly
> debug: installing packages
> debug: found cached pkg: /var/cache/pacman/pkg/abook-0.5.6-3-i686.pkg.tar.gz
> debug: loading target '/var/cache/pacman/pkg/abook-0.5.6-3-i686.pkg.tar.gz'
> debug: no package signature file found
>
> Where or what have i to configure as the "gpgme_init : signature directory"?
> My public key is in /root/.gnupg/pubring.gpg. I tried it also with /tmp/testing.gpg but the same error.
> AFAI could read the code this may belongs to commit:
> http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=commit;h=1a286336147c7d3af42041d26205b9ca3980f459
> I see a prog gpgme-config, but don't see what i could do with ;-)
>
> Help ;-)

I didn't promise this worked out of the box- I just meant that it was
a better start than the other code. You're either going to have to
know C and understand what is going on (and fix it), or wait for it to
be in a better state of completion.

-Dan

From sega01 at gmail.com  Mon Dec  8 08:00:36 2008
From: sega01 at gmail.com (Teran McKinney)
Date: Mon, 8 Dec 2008 13:00:36 +0000
Subject: [pacman-dev] GPG work
In-Reply-To: <449c10960812080434w6ba33471k9fd55ae681b69cf3@mail.gmail.com>
References: <449c10960812071318v7a7fb427nb19f4f5726496234@mail.gmail.com>
	<20081208115553.3c2fd3d9@tux1.brauer.lan>
	<449c10960812080434w6ba33471k9fd55ae681b69cf3@mail.gmail.com>
Message-ID: <b245135e0812080500s345f706dw98d32443bb7515a8@mail.gmail.com>

I like the idea of GPG signed repositories, but they are just about
useless if they are signing MD5s. MD5 is very insecure, but good for
normal file integrity checking. Can Pacman use SHA-256 or similiar?
Another thing to watch out for is malicious publication of old
repositories with old and vulnerable packages that have the force
option set. I've thought briefly on how to circumvent this, but not
enough to have a method I would purpose.

Thanks,
Teran

On Mon, Dec 8, 2008 at 12:34, Dan McGee <dpmcgee at gmail.com> wrote:
> On Mon, Dec 8, 2008 at 4:55 AM, Gerhard Brauer <gerbra at archlinux.de> wrote:
>> Am Sun, 7 Dec 2008 15:18:32 -0600
>> schrieb "Dan McGee" <dpmcgee at gmail.com>:
>>
>>> I did quite a bit more work with GPG today. I wrapped my head around
>>> GPGME, which presents a nice C interface to the GPG stuff so we are
>>> now a lot closer to a working implementation:
>>> http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=shortlog;h=refs/heads/newgpg
>>>
>>> >From the script side of things, I didn't change much. The libalpm
>>> >code
>>> has changed considerably, and there is still a lot of room for
>>> improvement. Let me know if you guys have questions.
>>
>> With heads/newgpg pacman doesn't check or find the .sig Files. If
>> starting with --debug i got these debug messages:
>>
>> debug: md5(/var/cache/pacman/pkg/abook-0.5.6-3-i686.pkg.tar.gz) =79777684f62164 934a1264df66b8fdc6
>> debug: returning error 35 from gpgme_init : signature directory not configured correctly
>> debug: installing packages
>> debug: found cached pkg: /var/cache/pacman/pkg/abook-0.5.6-3-i686.pkg.tar.gz
>> debug: loading target '/var/cache/pacman/pkg/abook-0.5.6-3-i686.pkg.tar.gz'
>> debug: no package signature file found
>>
>> Where or what have i to configure as the "gpgme_init : signature directory"?
>> My public key is in /root/.gnupg/pubring.gpg. I tried it also with /tmp/testing.gpg but the same error.
>> AFAI could read the code this may belongs to commit:
>> http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=commit;h=1a286336147c7d3af42041d26205b9ca3980f459
>> I see a prog gpgme-config, but don't see what i could do with ;-)
>>
>> Help ;-)
>
> I didn't promise this worked out of the box- I just meant that it was
> a better start than the other code. You're either going to have to
> know C and understand what is going on (and fix it), or wait for it to
> be in a better state of completion.
>
> -Dan
> _______________________________________________
> pacman-dev mailing list
> pacman-dev at archlinux.org
> http://archlinux.org/mailman/listinfo/pacman-dev
>

From dpmcgee at gmail.com  Mon Dec  8 08:08:20 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Mon, 8 Dec 2008 07:08:20 -0600
Subject: [pacman-dev] GPG work
In-Reply-To: <b245135e0812080500s345f706dw98d32443bb7515a8@mail.gmail.com>
References: <449c10960812071318v7a7fb427nb19f4f5726496234@mail.gmail.com>
	<20081208115553.3c2fd3d9@tux1.brauer.lan>
	<449c10960812080434w6ba33471k9fd55ae681b69cf3@mail.gmail.com>
	<b245135e0812080500s345f706dw98d32443bb7515a8@mail.gmail.com>
Message-ID: <449c10960812080508q790bf7c4ud178139694701177@mail.gmail.com>

On Mon, Dec 8, 2008 at 7:00 AM, Teran McKinney <sega01 at gmail.com> wrote:
> I like the idea of GPG signed repositories, but they are just about
> useless if they are signing MD5s. MD5 is very insecure, but good for
> normal file integrity checking. Can Pacman use SHA-256 or similiar?
> Another thing to watch out for is malicious publication of old
> repositories with old and vulnerable packages that have the force
> option set. I've thought briefly on how to circumvent this, but not
> enough to have a method I would purpose.

I think you misunderstood completely- try reading this first:
http://archlinux.org/pipermail/arch-dev-public/2008-December/009244.html

We sign *packages*, not repositories. Will this damn thing about MD5
please die? "Fixing" that still fixes nothing, and I'll pay one
million USD to someone that can actually forge a package with a given
MD5.

I believe I addressed the old repositories question there as well- we
will eventually have to sign databases too. A lot of thought was done
in this report:
http://www.cs.arizona.edu/people/justin/packagemanagersecurity/

-Dan

From ngaba at bibl.u-szeged.hu  Mon Dec  8 15:37:54 2008
From: ngaba at bibl.u-szeged.hu (Nagy Gabor)
Date: Mon, 08 Dec 2008 21:37:54 +0100
Subject: [pacman-dev] pacman 3.2.2 release
In-Reply-To: <449c10960812072030y5d091b32md682bcec2c67f8e4@mail.gmail.com>
References: <449c10960812072030y5d091b32md682bcec2c67f8e4@mail.gmail.com>
Message-ID: <1228768674.3712.12.camel@tandk.math.u-szeged.hu>

> I'd like to do a 3.2.2 release sometime soon (this week, more than
> likely). Let me know if there are any outstanding patches for this
> release that aren't already in the maint branch as of tonight. I will
> put a call for updated translations out in the coming few days.
> 

I have some pending patches in my working branch, maybe the last one
(fix for FS#12059) could appear in 3.2.2 (I am not sure, because the
patch introduces a new error type, which is an API extension).

Bye

http://repo.or.cz/w/pacman-ng.git?a=shortlog;h=refs/heads/working



From yunzheng.hu at gmail.com  Mon Dec  8 09:22:27 2008
From: yunzheng.hu at gmail.com (Yun Zheng Hu)
Date: Mon, 8 Dec 2008 15:22:27 +0100
Subject: [pacman-dev] [PATCH] Added the function parse_options to
	replace getopt
In-Reply-To: <BAY102-DAV720DCFA5BD7A81DDA252B86040@phx.gbl>
References: <1227138210-32437-1-git-send-email-yunzheng.hu@gmail.com>
	<BAY102-DAV720DCFA5BD7A81DDA252B86040@phx.gbl>
Message-ID: <df84014e0812080622v47d6bd88w79908aa5231ea1a1@mail.gmail.com>

>
> Just letting the submitter know that this patch is being looked at.  I have
> a getopt branch on my git repo where this will receive more testing before
> it gets pulled in.  From my current testing, it looks good and we seem to
> have lost no functionality.
>

Do you think this can make it to the pacman 3.2.3 release?

From dpmcgee at gmail.com  Mon Dec  8 09:31:56 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Mon, 8 Dec 2008 08:31:56 -0600
Subject: [pacman-dev] [PATCH] Added the function parse_options to
	replace getopt
In-Reply-To: <df84014e0812080622v47d6bd88w79908aa5231ea1a1@mail.gmail.com>
References: <1227138210-32437-1-git-send-email-yunzheng.hu@gmail.com>
	<BAY102-DAV720DCFA5BD7A81DDA252B86040@phx.gbl>
	<df84014e0812080622v47d6bd88w79908aa5231ea1a1@mail.gmail.com>
Message-ID: <449c10960812080631w41945303v6d44d366b819040e@mail.gmail.com>

On Mon, Dec 8, 2008 at 8:22 AM, Yun Zheng Hu <yunzheng.hu at gmail.com> wrote:
>>
>> Just letting the submitter know that this patch is being looked at.  I have
>> a getopt branch on my git repo where this will receive more testing before
>> it gets pulled in.  From my current testing, it looks good and we seem to
>> have lost no functionality.
>>
>
> Do you think this can make it to the pacman 3.2.3 release?

We (Allan and I) talked about this last night a bit. I would like to
get it in, but with a maint release especially, we don't want to have
regressions.

-Dan

From allan at archlinux.org  Mon Dec  8 09:35:13 2008
From: allan at archlinux.org (Allan McRae)
Date: Tue, 09 Dec 2008 00:35:13 +1000
Subject: [pacman-dev] [PATCH] Added the function parse_options
 to	replace getopt
In-Reply-To: <449c10960812080631w41945303v6d44d366b819040e@mail.gmail.com>
References: <1227138210-32437-1-git-send-email-yunzheng.hu@gmail.com>	<BAY102-DAV720DCFA5BD7A81DDA252B86040@phx.gbl>	<df84014e0812080622v47d6bd88w79908aa5231ea1a1@mail.gmail.com>
	<449c10960812080631w41945303v6d44d366b819040e@mail.gmail.com>
Message-ID: <BAY102-DAV1270BB22AB145720612A8886FD0@phx.gbl>

Dan McGee wrote:
> On Mon, Dec 8, 2008 at 8:22 AM, Yun Zheng Hu <yunzheng.hu at gmail.com> wrote:
>   
>>> Just letting the submitter know that this patch is being looked at.  I have
>>> a getopt branch on my git repo where this will receive more testing before
>>> it gets pulled in.  From my current testing, it looks good and we seem to
>>> have lost no functionality.
>>>
>>>       
>> Do you think this can make it to the pacman 3.2.3 release?
>>     
>
> We (Allan and I) talked about this last night a bit. I would like to
> get it in, but with a maint release especially, we don't want to have
> regressions.
>   

I will try and find time to give it a thorough review in the next couple 
of days and see if we can get it in.

Allan



From louipc.ist at gmail.com  Mon Dec  8 10:00:19 2008
From: louipc.ist at gmail.com (Loui Chang)
Date: Mon, 8 Dec 2008 10:00:19 -0500
Subject: [pacman-dev] GPG work
In-Reply-To: <449c10960812080508q790bf7c4ud178139694701177@mail.gmail.com>
References: <449c10960812071318v7a7fb427nb19f4f5726496234@mail.gmail.com>
	<20081208115553.3c2fd3d9@tux1.brauer.lan>
	<449c10960812080434w6ba33471k9fd55ae681b69cf3@mail.gmail.com>
	<b245135e0812080500s345f706dw98d32443bb7515a8@mail.gmail.com>
	<449c10960812080508q790bf7c4ud178139694701177@mail.gmail.com>
Message-ID: <20081208150019.GM517@lynn.lan>

On Mon, Dec 08, 2008 at 07:08:20AM -0600, Dan McGee wrote:
> We sign *packages*, not repositories. Will this damn thing about MD5
> please die? "Fixing" that still fixes nothing, and I'll pay one
> million USD to someone that can actually forge a package with a given
> MD5.
 
Hah hah! I have my work ahead of me!


From dpmcgee at gmail.com  Mon Dec  8 10:04:56 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Mon, 8 Dec 2008 09:04:56 -0600
Subject: [pacman-dev] GPG work
In-Reply-To: <449c10960812080508q790bf7c4ud178139694701177@mail.gmail.com>
References: <449c10960812071318v7a7fb427nb19f4f5726496234@mail.gmail.com>
	<20081208115553.3c2fd3d9@tux1.brauer.lan>
	<449c10960812080434w6ba33471k9fd55ae681b69cf3@mail.gmail.com>
	<b245135e0812080500s345f706dw98d32443bb7515a8@mail.gmail.com>
	<449c10960812080508q790bf7c4ud178139694701177@mail.gmail.com>
Message-ID: <449c10960812080704x6040fdaav192e28bdfdfaebd7@mail.gmail.com>

On Mon, Dec 8, 2008 at 7:08 AM, Dan McGee <dpmcgee at gmail.com> wrote:
> On Mon, Dec 8, 2008 at 7:00 AM, Teran McKinney <sega01 at gmail.com> wrote:
>> I like the idea of GPG signed repositories, but they are just about
>> useless if they are signing MD5s. MD5 is very insecure, but good for
>> normal file integrity checking. Can Pacman use SHA-256 or similiar?
>> Another thing to watch out for is malicious publication of old
>> repositories with old and vulnerable packages that have the force
>> option set. I've thought briefly on how to circumvent this, but not
>> enough to have a method I would purpose.
>
> I think you misunderstood completely- try reading this first:
> http://archlinux.org/pipermail/arch-dev-public/2008-December/009244.html

And sorry about this- I thought I had cross-posted this message to
this list, so now I see why it maybe wasn't clear the route we were
taking. Let me know if you have questions.

-Dan

From aaronmgriffin at gmail.com  Mon Dec  8 11:36:13 2008
From: aaronmgriffin at gmail.com (Aaron Griffin)
Date: Mon, 8 Dec 2008 10:36:13 -0600
Subject: [pacman-dev] GPG work
In-Reply-To: <20081208150019.GM517@lynn.lan>
References: <449c10960812071318v7a7fb427nb19f4f5726496234@mail.gmail.com>
	<20081208115553.3c2fd3d9@tux1.brauer.lan>
	<449c10960812080434w6ba33471k9fd55ae681b69cf3@mail.gmail.com>
	<b245135e0812080500s345f706dw98d32443bb7515a8@mail.gmail.com>
	<449c10960812080508q790bf7c4ud178139694701177@mail.gmail.com>
	<20081208150019.GM517@lynn.lan>
Message-ID: <d64a48980812080836j2a5d9b73x19b025d685a8100f@mail.gmail.com>

On Mon, Dec 8, 2008 at 9:00 AM, Loui Chang <louipc.ist at gmail.com> wrote:
> On Mon, Dec 08, 2008 at 07:08:20AM -0600, Dan McGee wrote:
>> We sign *packages*, not repositories. Will this damn thing about MD5
>> please die? "Fixing" that still fixes nothing, and I'll pay one
>> million USD to someone that can actually forge a package with a given
>> MD5.
>
> Hah hah! I have my work ahead of me!

Forcing md5sum collisions requires arbitrary null padding. tar can (I
think) support this, but not if it's compressed. You can't arbitrarily
put nulls in the middle of a gzip'd stream...

From aaronmgriffin at gmail.com  Mon Dec  8 11:38:32 2008
From: aaronmgriffin at gmail.com (Aaron Griffin)
Date: Mon, 8 Dec 2008 10:38:32 -0600
Subject: [pacman-dev] [PATCH] makepkg: Introduce restrict option
In-Reply-To: <BAY102-DAV791E7BBA92DC9A6A4748286FC0@phx.gbl>
References: <1228648740-11620-1-git-send-email-allan@archlinux.org>
	<BAY102-DAV791E7BBA92DC9A6A4748286FC0@phx.gbl>
Message-ID: <d64a48980812080838s2c558336t115655d38bfe5183@mail.gmail.com>

On Sun, Dec 7, 2008 at 5:23 AM, Allan McRae <allan at archlinux.org> wrote:
> Allan McRae wrote:
>>
>> The restrict option, combined with the RESTRICTED_FILES variable,
>> allows makepkg to automatically remove common confliting files
>> (e.g. /usr/share/info/dir).
>>
>> Signed-off-by: Allan McRae <allan at archlinux.org>
>> ---
>>
>
> Bonus points if someone comes up with a better name for the options.....
>
> Note that this patch made off my working branch which has a couple of
> patches which have not made mainline yet and overlap with some parts of code
> adjusted here.

And bonus bonus points it you can somehow combine this with the
!libtool option - it makes it way more generic and all that. Though
now that I think about it, it may break crap with the KDE packages.

From aaronmgriffin at gmail.com  Mon Dec  8 11:50:43 2008
From: aaronmgriffin at gmail.com (Aaron Griffin)
Date: Mon, 8 Dec 2008 10:50:43 -0600
Subject: [pacman-dev] makepkg: package splitting
In-Reply-To: <BAY102-DAV5C6EAD449827BDCFC3F5086FD0@phx.gbl>
References: <BAY102-DAV5C6EAD449827BDCFC3F5086FD0@phx.gbl>
Message-ID: <d64a48980812080850u48ebf2c4i7386c18c74380861@mail.gmail.com>

On Sun, Dec 7, 2008 at 8:35 PM, Allan McRae <allan at archlinux.org> wrote:
> I have finished implementing the basics of package splitting in makepkg.
>  Everything works as expected when just making packages and capturing logs
> but there is some small output issues that need tidying up and checking that
> other functionality work (e.g. repackaging is an issue....)

I like the PKGBUILD format. It seems nice. What happens, though, with
packages that have goofy chars in the name? say... libsigc++ - let's
pretend I wanted to split that. Do we have a workaround?

From serg.partizan at gmail.com  Mon Dec  8 15:19:53 2008
From: serg.partizan at gmail.com (serg)
Date: Mon, 8 Dec 2008 22:19:53 +0200
Subject: [pacman-dev] [translation] small fix for Russian translation
Message-ID: <d0a78d890812081219w202c0372w8939b834cf07c3f6@mail.gmail.com>

pacman/po/ru.po
972c972
< msgstr "??????:"
---
> msgstr "??????:"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://archlinux.org/pipermail/pacman-dev/attachments/20081208/3904478a/attachment.htm>

From allan at archlinux.org  Mon Dec  8 20:46:08 2008
From: allan at archlinux.org (Allan McRae)
Date: Tue, 09 Dec 2008 11:46:08 +1000
Subject: [pacman-dev] makepkg: package splitting
In-Reply-To: <d64a48980812080850u48ebf2c4i7386c18c74380861@mail.gmail.com>
References: <BAY102-DAV5C6EAD449827BDCFC3F5086FD0@phx.gbl>
	<d64a48980812080850u48ebf2c4i7386c18c74380861@mail.gmail.com>
Message-ID: <BAY102-DAV12AAECEB921CB242A6A05F86FA0@phx.gbl>

Aaron Griffin wrote:
> On Sun, Dec 7, 2008 at 8:35 PM, Allan McRae <allan at archlinux.org> wrote:
>   
>> I have finished implementing the basics of package splitting in makepkg.
>>  Everything works as expected when just making packages and capturing logs
>> but there is some small output issues that need tidying up and checking that
>> other functionality work (e.g. repackaging is an issue....)
>>     
>
> I like the PKGBUILD format. It seems nice. What happens, though, with
> packages that have goofy chars in the name? say... libsigc++ - let's
> pretend I wanted to split that. Do we have a workaround

Do we need a workaround?

test.sh:
#! /bin/bash

a+() {
    echo "Function with goofy chars works"
}

a+
[EOF]

 > ./test.sh
Function with goofy chars works






From mad at wol.de  Tue Dec  9 06:30:56 2008
From: mad at wol.de (Marc - A. Dahlhaus [ Administration | Westermann GmbH ])
Date: Tue, 09 Dec 2008 12:30:56 +0100
Subject: [pacman-dev] makepkg: package splitting
In-Reply-To: <493C87FD.8080501@archlinux.org>
References: <493C87FD.8080501@archlinux.org>
Message-ID: <1228822256.16706.12.camel@marc>

Am Montag, den 08.12.2008, 12:35 +1000 schrieb Allan McRae:
> I have finished implementing the basics of package splitting in 
> makepkg.  Everything works as expected when just making packages and 
> capturing logs but there is some small output issues that need tidying 
> up and checking that other functionality work (e.g. repackaging is an 
> issue....)

Hello Allan,


i've tested the script and found some bugs:

The sourcing of makepkg.conf is in a wrong location (far to late).
If you display the usage via --help you get no PKGBUILD mentioned in
usages output. If you specify another buildscript via -p you get that
mentioned in usage but it gets overwritten by the contents of
makepkg.conf when it gets sourced in later before the actual build is
done.

If you give -c to makepkg the pkgdirs of the splitpackages don't get
erased.

But the actual build and logging went fine, thank you :)

Hope that helps,

Marc


From allan at archlinux.org  Tue Dec  9 06:57:41 2008
From: allan at archlinux.org (Allan McRae)
Date: Tue, 09 Dec 2008 21:57:41 +1000
Subject: [pacman-dev] makepkg: package splitting
In-Reply-To: <1228822256.16706.12.camel@marc>
References: <493C87FD.8080501@archlinux.org> <1228822256.16706.12.camel@marc>
Message-ID: <BAY102-DAV57CA001EB7078EB145A7E86FA0@phx.gbl>

Marc - A. Dahlhaus [ Administration | Westermann GmbH ] wrote:
> Am Montag, den 08.12.2008, 12:35 +1000 schrieb Allan McRae:
>   
>> I have finished implementing the basics of package splitting in 
>> makepkg.  Everything works as expected when just making packages and 
>> capturing logs but there is some small output issues that need tidying 
>> up and checking that other functionality work (e.g. repackaging is an 
>> issue....)
>>     
>
> Hello Allan,
>
>
> i've tested the script and found some bugs:
>
> The sourcing of makepkg.conf is in a wrong location (far to late).
> If you display the usage via --help you get no PKGBUILD mentioned in
> usages output. If you specify another buildscript via -p you get that
> mentioned in usage but it gets overwritten by the contents of
> makepkg.conf when it gets sourced in later before the actual build is
> done.
>   

Good catch....  I have no idea what caused that!  Should not be herd to 
track down though.

> If you give -c to makepkg the pkgdirs of the splitpackages don't get
> erased.
>   

Someone is either running makepkg as root or without fakeroot...   
Anyway, thats an easy fix.

> But the actual build and logging went fine, thank you :)
>   

Good to see the bits I tested actually working!  Thanks for your help.

Allan




From mad at wol.de  Tue Dec  9 07:23:07 2008
From: mad at wol.de (Marc - A. Dahlhaus [ Administration | Westermann GmbH ])
Date: Tue, 09 Dec 2008 13:23:07 +0100
Subject: [pacman-dev] makepkg: package splitting
In-Reply-To: <493E5D35.5050702@archlinux.org>
References: <493C87FD.8080501@archlinux.org>
	<1228822256.16706.12.camel@marc>  <493E5D35.5050702@archlinux.org>
Message-ID: <1228825387.16706.20.camel@marc>

Am Dienstag, den 09.12.2008, 21:57 +1000 schrieb Allan McRae:
> Marc - A. Dahlhaus [ Administration | Westermann GmbH ] wrote:
> > Am Montag, den 08.12.2008, 12:35 +1000 schrieb Allan McRae:
> >   
> >> I have finished implementing the basics of package splitting in 
> >> makepkg.  Everything works as expected when just making packages and 
> >> capturing logs but there is some small output issues that need tidying 
> >> up and checking that other functionality work (e.g. repackaging is an 
> >> issue....)
> >>     
> >
> > Hello Allan,
> >
> >
> > i've tested the script and found some bugs:
> >
> > The sourcing of makepkg.conf is in a wrong location (far to late).
> > If you display the usage via --help you get no PKGBUILD mentioned in
> > usages output. If you specify another buildscript via -p you get that
> > mentioned in usage but it gets overwritten by the contents of
> > makepkg.conf when it gets sourced in later before the actual build is
> > done.
> >   
> 
> Good catch....  I have no idea what caused that!  Should not be herd to 
> track down though.
> 
> > If you give -c to makepkg the pkgdirs of the splitpackages don't get
> > erased.
> >   
> 
> Someone is either running makepkg as root or without fakeroot...   
> Anyway, thats an easy fix.

I'll tested that again and i forgot to change two of the DESTDIR
locations to the new home of the pkgdirs... So the missing cleanup was
actually a stupid user error...

> > But the actual build and logging went fine, thank you :)
> >   
> 
> Good to see the bits I tested actually working!  Thanks for your help.
> 
> Allan


From allan at archlinux.org  Tue Dec  9 07:28:43 2008
From: allan at archlinux.org (Allan McRae)
Date: Tue, 09 Dec 2008 22:28:43 +1000
Subject: [pacman-dev] makepkg: package splitting
In-Reply-To: <BAY102-DAV57CA001EB7078EB145A7E86FA0@phx.gbl>
References: <493C87FD.8080501@archlinux.org> <1228822256.16706.12.camel@marc>
	<BAY102-DAV57CA001EB7078EB145A7E86FA0@phx.gbl>
Message-ID: <BAY102-DAV11F8DB1DBBA5942F0FD4A286FA0@phx.gbl>

Allan McRae wrote:
> Marc - A. Dahlhaus [ Administration | Westermann GmbH ] wrote:
>> Am Montag, den 08.12.2008, 12:35 +1000 schrieb Allan McRae:
>>  
>>> I have finished implementing the basics of package splitting in 
>>> makepkg.  Everything works as expected when just making packages and 
>>> capturing logs but there is some small output issues that need 
>>> tidying up and checking that other functionality work (e.g. 
>>> repackaging is an issue....)
>>>     
>>
>> Hello Allan,
>>
>>
>> i've tested the script and found some bugs:
>>
>> The sourcing of makepkg.conf is in a wrong location (far to late).
>> If you display the usage via --help you get no PKGBUILD mentioned in
>> usages output. If you specify another buildscript via -p you get that
>> mentioned in usage but it gets overwritten by the contents of
>> makepkg.conf when it gets sourced in later before the actual build is
>> done.
>>   
>
> Good catch....  I have no idea what caused that!  Should not be herd 
> to track down though.
>

This was actually not my fault!  It is an issue with the current master 
branch.  Separate post on its way...

>> If you give -c to makepkg the pkgdirs of the splitpackages don't get
>> erased.
>>   
>
> Someone is either running makepkg as root or without fakeroot...   
> Anyway, thats an easy fix.
>

Fixed.

Allan




From allan at archlinux.org  Tue Dec  9 07:32:29 2008
From: allan at archlinux.org (Allan McRae)
Date: Tue, 09 Dec 2008 22:32:29 +1000
Subject: [pacman-dev] makepkg: package splitting
In-Reply-To: <1228825387.16706.20.camel@marc>
References: <493C87FD.8080501@archlinux.org>	<1228822256.16706.12.camel@marc>
	<493E5D35.5050702@archlinux.org> <1228825387.16706.20.camel@marc>
Message-ID: <BAY102-DAV1382308888E15E2EE316FF86FA0@phx.gbl>

Marc - A. Dahlhaus [ Administration | Westermann GmbH ] wrote:
> Am Dienstag, den 09.12.2008, 21:57 +1000 schrieb Allan McRae:
>   
>> Marc - A. Dahlhaus [ Administration | Westermann GmbH ] wrote:
>>     
>>     
>>> If you give -c to makepkg the pkgdirs of the splitpackages don't get
>>> erased.
>>>   
>>>       
>> Someone is either running makepkg as root or without fakeroot...   
>> Anyway, thats an easy fix.
>>     
>
> I'll tested that again and i forgot to change two of the DESTDIR
> locations to the new home of the pkgdirs... So the missing cleanup was
> actually a stupid user error...
>   

You should only be doing something like "make DESTDIR=$pkgdir install".  
Makepkg points $pkgdir in the right place for each split package.  (I 
will get around to documenting all this at some stage)

Allan




From allan at archlinux.org  Tue Dec  9 07:41:18 2008
From: allan at archlinux.org (Allan McRae)
Date: Tue, 09 Dec 2008 22:41:18 +1000
Subject: [pacman-dev] Order of option parsing and sourcing makepkg.conf
Message-ID: <BAY102-DAV6716983E81A8B8184036F86FA0@phx.gbl>

Hi,

With the commit which enables us to specify the makepkg config file on 
the command-line 
(http://projects.archlinux.org/?p=pacman.git;a=commitdiff;h=4b183bf9), 
the sourcing of the config file gets moved after the parsing of the 
options.  This creates problems with the --help flag as it needs to know 
what BUILDSCRIPT is defined as.  It also causes problems when specifying 
a different BUILDSCRIPT with -p as this gets set during option parsing 
then gets overwritten with the makepkg config file gets sourced.

I don't see a nice fix for this.  Sourcing the conf file before option 
parsing and then again after if it is changed seems not good to me.  So 
can we back that patch out at least temporarily.

Note that this does not effect maint so the 3.2.2 release will be bug 
free as usual.

Allan




From allan at archlinux.org  Tue Dec  9 07:55:00 2008
From: allan at archlinux.org (Allan McRae)
Date: Tue, 09 Dec 2008 22:55:00 +1000
Subject: [pacman-dev] Order of option parsing and sourcing makepkg.conf
In-Reply-To: <BAY102-DAV6716983E81A8B8184036F86FA0@phx.gbl>
References: <BAY102-DAV6716983E81A8B8184036F86FA0@phx.gbl>
Message-ID: <BAY102-DAV7449810D3BA002DA9BC1986FA0@phx.gbl>

Allan McRae wrote:
> Hi,
>
> With the commit which enables us to specify the makepkg config file on 
> the command-line 
> (http://projects.archlinux.org/?p=pacman.git;a=commitdiff;h=4b183bf9), 
> the sourcing of the config file gets moved after the parsing of the 
> options.  This creates problems with the --help flag as it needs to 
> know what BUILDSCRIPT is defined as.  It also causes problems when 
> specifying a different BUILDSCRIPT with -p as this gets set during 
> option parsing then gets overwritten with the makepkg config file gets 
> sourced.
>
> I don't see a nice fix for this.  Sourcing the conf file before option 
> parsing and then again after if it is changed seems not good to me.  
> So can we back that patch out at least temporarily.
>
> Note that this does not effect maint so the 3.2.2 release will be bug 
> free as usual.

Here is a link to the original discussion of this patch:
http://archlinux.org/pipermail/pacman-dev/2008-August/007321.html

Allan



From dpmcgee at gmail.com  Tue Dec  9 09:40:40 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Tue, 9 Dec 2008 08:40:40 -0600
Subject: [pacman-dev] Order of option parsing and sourcing makepkg.conf
In-Reply-To: <BAY102-DAV7449810D3BA002DA9BC1986FA0@phx.gbl>
References: <BAY102-DAV6716983E81A8B8184036F86FA0@phx.gbl>
	<BAY102-DAV7449810D3BA002DA9BC1986FA0@phx.gbl>
Message-ID: <449c10960812090640iaf0ce18t806c896a293f3f79@mail.gmail.com>

On Tue, Dec 9, 2008 at 6:55 AM, Allan McRae <allan at archlinux.org> wrote:
> Allan McRae wrote:
>>
>> Hi,
>>
>> With the commit which enables us to specify the makepkg config file on the
>> command-line
>> (http://projects.archlinux.org/?p=pacman.git;a=commitdiff;h=4b183bf9), the
>> sourcing of the config file gets moved after the parsing of the options.
>>  This creates problems with the --help flag as it needs to know what
>> BUILDSCRIPT is defined as.  It also causes problems when specifying a
>> different BUILDSCRIPT with -p as this gets set during option parsing then
>> gets overwritten with the makepkg config file gets sourced.
>>
>> I don't see a nice fix for this.  Sourcing the conf file before option
>> parsing and then again after if it is changed seems not good to me.  So can
>> we back that patch out at least temporarily.
>>
>> Note that this does not effect maint so the 3.2.2 release will be bug free
>> as usual.
>
> Here is a link to the original discussion of this patch:
> http://archlinux.org/pipermail/pacman-dev/2008-August/007321.html

Maybe we should reconsider moving BUILDSCRIPT out of makepkg and into
makepkg.conf? This was commit 9c9e18ef32c0cf3, it feels like eons ago.
But let's be honest- how often is someone going to change the name of
PKGBUILD or should even care about it? If we at least initially define
it in makepkg, and allow for overrides in makepkg.conf, it probably
isn't the end of the world.

-Dan

From allan at archlinux.org  Tue Dec  9 09:55:24 2008
From: allan at archlinux.org (Allan McRae)
Date: Wed, 10 Dec 2008 00:55:24 +1000
Subject: [pacman-dev] Order of option parsing and sourcing makepkg.conf
In-Reply-To: <449c10960812090640iaf0ce18t806c896a293f3f79@mail.gmail.com>
References: <BAY102-DAV6716983E81A8B8184036F86FA0@phx.gbl>	<BAY102-DAV7449810D3BA002DA9BC1986FA0@phx.gbl>
	<449c10960812090640iaf0ce18t806c896a293f3f79@mail.gmail.com>
Message-ID: <BAY102-DAV3E6C6B3B481D5A1BB321086FA0@phx.gbl>

Dan McGee wrote:
> On Tue, Dec 9, 2008 at 6:55 AM, Allan McRae <allan at archlinux.org> wrote:
>   
>> Allan McRae wrote:
>>     
>>> Hi,
>>>
>>> With the commit which enables us to specify the makepkg config file on the
>>> command-line
>>> (http://projects.archlinux.org/?p=pacman.git;a=commitdiff;h=4b183bf9), the
>>> sourcing of the config file gets moved after the parsing of the options.
>>>  This creates problems with the --help flag as it needs to know what
>>> BUILDSCRIPT is defined as.  It also causes problems when specifying a
>>> different BUILDSCRIPT with -p as this gets set during option parsing then
>>> gets overwritten with the makepkg config file gets sourced.
>>>
>>> I don't see a nice fix for this.  Sourcing the conf file before option
>>> parsing and then again after if it is changed seems not good to me.  So can
>>> we back that patch out at least temporarily.
>>>
>>> Note that this does not effect maint so the 3.2.2 release will be bug free
>>> as usual.
>>>       
>> Here is a link to the original discussion of this patch:
>> http://archlinux.org/pipermail/pacman-dev/2008-August/007321.html
>>     
>
> Maybe we should reconsider moving BUILDSCRIPT out of makepkg and into
> makepkg.conf? This was commit 9c9e18ef32c0cf3, it feels like eons ago.
> But let's be honest- how often is someone going to change the name of
> PKGBUILD or should even care about it? If we at least initially define
> it in makepkg, and allow for overrides in makepkg.conf, it probably
> isn't the end of the world.
>   

Well, that would be the easy fix I didn't see...   I have no objections 
to moving BUILDSCRIPT back into makepkg.  I would be surprised if anyone 
ever changed it.  And we provide the "-p" flag if someone wants to 
override it anyway. 

Allan




From aaronmgriffin at gmail.com  Tue Dec  9 11:00:47 2008
From: aaronmgriffin at gmail.com (Aaron Griffin)
Date: Tue, 9 Dec 2008 10:00:47 -0600
Subject: [pacman-dev] makepkg: package splitting
In-Reply-To: <BAY102-DAV12AAECEB921CB242A6A05F86FA0@phx.gbl>
References: <BAY102-DAV5C6EAD449827BDCFC3F5086FD0@phx.gbl>
	<d64a48980812080850u48ebf2c4i7386c18c74380861@mail.gmail.com>
	<BAY102-DAV12AAECEB921CB242A6A05F86FA0@phx.gbl>
Message-ID: <d64a48980812090800s2354084ayf23b861fc3f2abca@mail.gmail.com>

On Mon, Dec 8, 2008 at 7:46 PM, Allan McRae <allan at archlinux.org> wrote:
> Aaron Griffin wrote:
>>
>> On Sun, Dec 7, 2008 at 8:35 PM, Allan McRae <allan at archlinux.org> wrote:
>>
>>>
>>> I have finished implementing the basics of package splitting in makepkg.
>>>  Everything works as expected when just making packages and capturing
>>> logs
>>> but there is some small output issues that need tidying up and checking
>>> that
>>> other functionality work (e.g. repackaging is an issue....)
>>>
>>
>> I like the PKGBUILD format. It seems nice. What happens, though, with
>> packages that have goofy chars in the name? say... libsigc++ - let's
>> pretend I wanted to split that. Do we have a workaround
>
> Do we need a workaround?
>
> test.sh:
> #! /bin/bash
>
> a+() {
>   echo "Function with goofy chars works"
> }
>
> a+
> [EOF]
>
>> ./test.sh
> Function with goofy chars works

Well, color me stupid! I thought bash functions used C restrictions...
alphanumeric and underscore.

From aaronmgriffin at gmail.com  Tue Dec  9 11:08:37 2008
From: aaronmgriffin at gmail.com (Aaron Griffin)
Date: Tue, 9 Dec 2008 10:08:37 -0600
Subject: [pacman-dev] Order of option parsing and sourcing makepkg.conf
In-Reply-To: <BAY102-DAV3E6C6B3B481D5A1BB321086FA0@phx.gbl>
References: <BAY102-DAV6716983E81A8B8184036F86FA0@phx.gbl>
	<BAY102-DAV7449810D3BA002DA9BC1986FA0@phx.gbl>
	<449c10960812090640iaf0ce18t806c896a293f3f79@mail.gmail.com>
	<BAY102-DAV3E6C6B3B481D5A1BB321086FA0@phx.gbl>
Message-ID: <d64a48980812090808n65ee80ebo4248d37bee87a41c@mail.gmail.com>

On Tue, Dec 9, 2008 at 8:55 AM, Allan McRae <allan at archlinux.org> wrote:
> Dan McGee wrote:
>>
>> On Tue, Dec 9, 2008 at 6:55 AM, Allan McRae <allan at archlinux.org> wrote:
>>
>>>
>>> Allan McRae wrote:
>>>
>>>>
>>>> Hi,
>>>>
>>>> With the commit which enables us to specify the makepkg config file on
>>>> the
>>>> command-line
>>>> (http://projects.archlinux.org/?p=pacman.git;a=commitdiff;h=4b183bf9),
>>>> the
>>>> sourcing of the config file gets moved after the parsing of the options.
>>>>  This creates problems with the --help flag as it needs to know what
>>>> BUILDSCRIPT is defined as.  It also causes problems when specifying a
>>>> different BUILDSCRIPT with -p as this gets set during option parsing
>>>> then
>>>> gets overwritten with the makepkg config file gets sourced.
>>>>
>>>> I don't see a nice fix for this.  Sourcing the conf file before option
>>>> parsing and then again after if it is changed seems not good to me.  So
>>>> can
>>>> we back that patch out at least temporarily.
>>>>
>>>> Note that this does not effect maint so the 3.2.2 release will be bug
>>>> free
>>>> as usual.
>>>>
>>>
>>> Here is a link to the original discussion of this patch:
>>> http://archlinux.org/pipermail/pacman-dev/2008-August/007321.html
>>>
>>
>> Maybe we should reconsider moving BUILDSCRIPT out of makepkg and into
>> makepkg.conf? This was commit 9c9e18ef32c0cf3, it feels like eons ago.
>> But let's be honest- how often is someone going to change the name of
>> PKGBUILD or should even care about it? If we at least initially define
>> it in makepkg, and allow for overrides in makepkg.conf, it probably
>> isn't the end of the world.
>>
>
> Well, that would be the easy fix I didn't see...   I have no objections to
> moving BUILDSCRIPT back into makepkg.  I would be surprised if anyone ever
> changed it.  And we provide the "-p" flag if someone wants to override it
> anyway.

Just an FYI for if you do this. The db-scripts uses $BUILDSCRIPT, so I
will need to define it myself before you upgrade pacman on gerolde.

From dpmcgee at gmail.com  Tue Dec  9 13:20:50 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Tue, 9 Dec 2008 12:20:50 -0600
Subject: [pacman-dev] Order of option parsing and sourcing makepkg.conf
In-Reply-To: <d64a48980812090808n65ee80ebo4248d37bee87a41c@mail.gmail.com>
References: <BAY102-DAV6716983E81A8B8184036F86FA0@phx.gbl>
	<BAY102-DAV7449810D3BA002DA9BC1986FA0@phx.gbl>
	<449c10960812090640iaf0ce18t806c896a293f3f79@mail.gmail.com>
	<BAY102-DAV3E6C6B3B481D5A1BB321086FA0@phx.gbl>
	<d64a48980812090808n65ee80ebo4248d37bee87a41c@mail.gmail.com>
Message-ID: <449c10960812091020n37ce4a3eoe944ec4e755f094d@mail.gmail.com>

On Tue, Dec 9, 2008 at 10:08 AM, Aaron Griffin <aaronmgriffin at gmail.com> wrote:
> On Tue, Dec 9, 2008 at 8:55 AM, Allan McRae <allan at archlinux.org> wrote:
>> Dan McGee wrote:
>>>
>>> On Tue, Dec 9, 2008 at 6:55 AM, Allan McRae <allan at archlinux.org> wrote:
>>>
>>>>
>>>> Allan McRae wrote:
>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> With the commit which enables us to specify the makepkg config file on
>>>>> the
>>>>> command-line
>>>>> (http://projects.archlinux.org/?p=pacman.git;a=commitdiff;h=4b183bf9),
>>>>> the
>>>>> sourcing of the config file gets moved after the parsing of the options.
>>>>>  This creates problems with the --help flag as it needs to know what
>>>>> BUILDSCRIPT is defined as.  It also causes problems when specifying a
>>>>> different BUILDSCRIPT with -p as this gets set during option parsing
>>>>> then
>>>>> gets overwritten with the makepkg config file gets sourced.
>>>>>
>>>>> I don't see a nice fix for this.  Sourcing the conf file before option
>>>>> parsing and then again after if it is changed seems not good to me.  So
>>>>> can
>>>>> we back that patch out at least temporarily.
>>>>>
>>>>> Note that this does not effect maint so the 3.2.2 release will be bug
>>>>> free
>>>>> as usual.
>>>>>
>>>>
>>>> Here is a link to the original discussion of this patch:
>>>> http://archlinux.org/pipermail/pacman-dev/2008-August/007321.html
>>>>
>>>
>>> Maybe we should reconsider moving BUILDSCRIPT out of makepkg and into
>>> makepkg.conf? This was commit 9c9e18ef32c0cf3, it feels like eons ago.
>>> But let's be honest- how often is someone going to change the name of
>>> PKGBUILD or should even care about it? If we at least initially define
>>> it in makepkg, and allow for overrides in makepkg.conf, it probably
>>> isn't the end of the world.
>>>
>>
>> Well, that would be the easy fix I didn't see...   I have no objections to
>> moving BUILDSCRIPT back into makepkg.  I would be surprised if anyone ever
>> changed it.  And we provide the "-p" flag if someone wants to override it
>> anyway.
>
> Just an FYI for if you do this. The db-scripts uses $BUILDSCRIPT, so I
> will need to define it myself before you upgrade pacman on gerolde.

As long as no one merges/deletes things from makepkg.conf you're fine. :)

-Dan

From smurfd at gmail.com  Wed Dec 10 04:12:08 2008
From: smurfd at gmail.com (smurfd)
Date: Wed, 10 Dec 2008 10:12:08 +0100
Subject: [pacman-dev] pacman summary!
Message-ID: <fafd59260812100112k6b90e390p4fd7c28a6639d56e@mail.gmail.com>

Hi,
i have a suggestion,
like the suggestion i gave to the portage developers for the gentoo
"package" manager.

It would be really nice, if after a "pacman -Suy" or just a simple "pacman
-S <packagelist>" that you got a summary of all the messages that has been
spit out on the screen while doing the upgrade of packages.

If you upgrade several packages, say 100 of them or so, it could be hard to
read all the notices that every package spits out when beeing installed.

i mean stuff like :
*>>> Updating module dependencies. Please wait ...
>>> MKINITCPIO SETUP
>>> ----------------
>>> If you use LVM2, Encrypted root or software RAID,
>>> Ensure you enable support in /etc/mkinitcpio.conf .
>>> More information about mkinitcpio setup can be found here:
>>> http://wiki.archlinux.org/index.php/Mkinitcpio

>>> Generating initial ramdisk, using mkinitcpio.  Please wait...
*
Hope you like the idea,
and i dont think it should be so hard to apply.
cut all of the >>> things to a temp file and after doing the last upgrade
you cat whatever is in that temp file.. or something like that.

Best regards
/smurfd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://archlinux.org/pipermail/pacman-dev/attachments/20081210/5b555bda/attachment-0001.htm>

From allan at archlinux.org  Wed Dec 10 04:20:05 2008
From: allan at archlinux.org (Allan McRae)
Date: Wed, 10 Dec 2008 19:20:05 +1000
Subject: [pacman-dev] pacman summary!
In-Reply-To: <fafd59260812100112k6b90e390p4fd7c28a6639d56e@mail.gmail.com>
References: <fafd59260812100112k6b90e390p4fd7c28a6639d56e@mail.gmail.com>
Message-ID: <BAY102-DAV129CA131A5342AC3B6993686FB0@phx.gbl>

smurfd wrote:
> Hi,
> i have a suggestion,
> like the suggestion i gave to the portage developers for the gentoo 
> "package" manager.
>
> It would be really nice, if after a "pacman -Suy" or just a simple 
> "pacman -S <packagelist>" that you got a summary of all the messages 
> that has been spit out on the screen while doing the upgrade of packages.
>
> If you upgrade several packages, say 100 of them or so, it could be 
> hard to read all the notices that every package spits out when beeing 
> installed.
>
> i mean stuff like :
> />>> Updating module dependencies. Please wait ...
> >>> MKINITCPIO SETUP
> >>> ----------------
> >>> If you use LVM2, Encrypted root or software RAID,
> >>> Ensure you enable support in /etc/mkinitcpio.conf .
> >>> More information about mkinitcpio setup can be found here:
> >>> http://wiki.archlinux.org/index.php/Mkinitcpio
>
> >>> Generating initial ramdisk, using mkinitcpio.  Please wait...
> /
> Hope you like the idea,
> and i dont think it should be so hard to apply.
> cut all of the >>> things to a temp file and after doing the last 
> upgrade you cat whatever is in that temp file.. or something like that.
>
> Best regards
> /smurfd

So looking at /var/log/pacman.log is not enough?

Allan




From smurfd at gmail.com  Wed Dec 10 04:44:56 2008
From: smurfd at gmail.com (smurfd)
Date: Wed, 10 Dec 2008 10:44:56 +0100
Subject: [pacman-dev] pacman summary!
In-Reply-To: <BAY102-DAV129CA131A5342AC3B6993686FB0@phx.gbl>
References: <fafd59260812100112k6b90e390p4fd7c28a6639d56e@mail.gmail.com>
	<BAY102-DAV129CA131A5342AC3B6993686FB0@phx.gbl>
Message-ID: <fafd59260812100144m5d7342d0oa457638dd8a29664@mail.gmail.com>

ah okey, didnt know that that existed...
cool.
thanks.

On Wed, Dec 10, 2008 at 10:20 AM, Allan McRae <allan at archlinux.org> wrote:

> smurfd wrote:
>
>> Hi,
>> i have a suggestion,
>> like the suggestion i gave to the portage developers for the gentoo
>> "package" manager.
>>
>> It would be really nice, if after a "pacman -Suy" or just a simple "pacman
>> -S <packagelist>" that you got a summary of all the messages that has been
>> spit out on the screen while doing the upgrade of packages.
>>
>> If you upgrade several packages, say 100 of them or so, it could be hard
>> to read all the notices that every package spits out when beeing installed.
>>
>> i mean stuff like :
>> />>> Updating module dependencies. Please wait ...
>> >>> MKINITCPIO SETUP
>> >>> ----------------
>> >>> If you use LVM2, Encrypted root or software RAID,
>> >>> Ensure you enable support in /etc/mkinitcpio.conf .
>> >>> More information about mkinitcpio setup can be found here:
>> >>> http://wiki.archlinux.org/index.php/Mkinitcpio
>>
>> >>> Generating initial ramdisk, using mkinitcpio.  Please wait...
>> /
>> Hope you like the idea,
>> and i dont think it should be so hard to apply.
>> cut all of the >>> things to a temp file and after doing the last upgrade
>> you cat whatever is in that temp file.. or something like that.
>>
>> Best regards
>> /smurfd
>>
>
> So looking at /var/log/pacman.log is not enough?
>
> Allan
>
>
>
> _______________________________________________
> pacman-dev mailing list
> pacman-dev at archlinux.org
> http://archlinux.org/mailman/listinfo/pacman-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://archlinux.org/pipermail/pacman-dev/attachments/20081210/74e1ee87/attachment.htm>

From vmiklos at frugalware.org  Wed Dec 10 09:59:06 2008
From: vmiklos at frugalware.org (Miklos Vajna)
Date: Wed, 10 Dec 2008 15:59:06 +0100
Subject: [pacman-dev] makepkg: package splitting
In-Reply-To: <493C87FD.8080501@archlinux.org>
References: <493C87FD.8080501@archlinux.org>
Message-ID: <20081210145906.GN5691@genesis.frugalware.org>

On Mon, Dec 08, 2008 at 12:35:41PM +1000, Allan McRae <allan at archlinux.org> wrote:
> pkgname=('abs-core' 'abs-readme')
> pkgver=2.3
> pkgrel=1
> arch=('i686' 'x86_64')
> url="http://projects.archlinux.org/git/?p=abs.git"
> license=('GPL')
> source=(ftp://ftp.archlinux.org/other/abs/abs-${pkgver}.tar.gz)
> md5sums=('d6fd791aa487ba8bb5ff48c3ace20592')
> 
> build() {
>   cd ${srcdir}/abs
>   make CONFDIR=/etc/ || return 1
>   
>   # change ABS tags for x86_64 to correct values
>   if [ "$CARCH" = "x86_64" ]; then
>     sed -i "s|i686|x86_64|g" ${pkgdir}/etc/abs.conf
>   fi
> }
> 
> package_abs-core() {  
>   pkgdesc="Utilities to download and work with the Arch Build System (ABS)"
>   depends=('bash' 'rsync')
>   backup=(etc/abs.conf)
> 
>   cd ${srcdir}/abs
>   make CONFDIR=/etc/ DESTDIR=${pkgdir} install || return 1
> }
> 
> package_abs-readme() {
>   pkgdesc="ABS base directory and README file"
>   depends=('abs')
>   install=abs.install
> 
>   # Add readme file, and make base /var/abs path
>   install -dm0755 ${pkgdir}/var/abs/local/
>   install -Dm0644 ${srcdir}/abs/README ${pkgdir}/var/abs/README
> }

How does this allow one to define package-dependent dependencies?
Typically foo-python will have to depend on python, but foo won't.

Thanks.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://archlinux.org/pipermail/pacman-dev/attachments/20081210/b9ac56cb/attachment.pgp>

From vmiklos at frugalware.org  Wed Dec 10 10:00:57 2008
From: vmiklos at frugalware.org (Miklos Vajna)
Date: Wed, 10 Dec 2008 16:00:57 +0100
Subject: [pacman-dev] makepkg: package splitting
In-Reply-To: <20081210145906.GN5691@genesis.frugalware.org>
References: <493C87FD.8080501@archlinux.org>
	<20081210145906.GN5691@genesis.frugalware.org>
Message-ID: <20081210150057.GO5691@genesis.frugalware.org>

On Wed, Dec 10, 2008 at 03:59:06PM +0100, Miklos Vajna <vmiklos at frugalware.org> wrote:
> How does this allow one to define package-dependent dependencies?
> Typically foo-python will have to depend on python, but foo won't.

Aah, sorry for the spam. I overlooked the contents of the functions.

Sorry again.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://archlinux.org/pipermail/pacman-dev/attachments/20081210/2dbc818e/attachment.pgp>

From aaronmgriffin at gmail.com  Fri Dec 12 01:02:19 2008
From: aaronmgriffin at gmail.com (aaronmgriffin at gmail.com)
Date: Fri, 12 Dec 2008 00:02:19 -0600
Subject: [pacman-dev] [PATCH] Make the repo-add quiet flag less quiet
Message-ID: <1229061739-14685-1-git-send-email-aaronmgriffin@gmail.com>

From: Aaron Griffin <aaronmgriffin at gmail.com>

Considering one can easily run:
   repo-add .... >/dev/null
to get only warnings and errors, the -q flag is mostly useless.

Make the -q flag silence only level 2 messages

Signed-off-by: Aaron Griffin <aaronmgriffin at gmail.com>
---
 scripts/repo-add.sh.in |    5 ++---
 1 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/scripts/repo-add.sh.in b/scripts/repo-add.sh.in
index 4f9639c..c89e2a5 100644
--- a/scripts/repo-add.sh.in
+++ b/scripts/repo-add.sh.in
@@ -34,7 +34,6 @@ REPO_DB_FILE=""
 umask 0022
 
 msg() {
-	[ $QUIET -ne 0 ] && return
 	local mesg=$1; shift
 	printf "==> ${mesg}\n" "$@" >&1
 }
@@ -68,8 +67,8 @@ repo-remove will update a package database by removing the package name\n\
 specified on the command line from the given repo database. Multiple\n\
 packages to remove can be specified on the command line.\n\n")"
 	printf "$(gettext "\
-The -q/--quiet flag to either program will force silent running except\n\
-in the case of warnings or errors.\n\n")"
+Use the -q/--quiet flag to minimize output to basic messages, warnings,\n\
+and errors\n\n")"
 	echo "$(gettext "Example:  repo-add /path/to/repo.db.tar.gz pacman-3.0.0.pkg.tar.gz")"
 	echo "$(gettext "Example:  repo-remove /path/to/repo.db.tar.gz kernel26")"
 }
-- 
1.6.0.5


From dpmcgee at gmail.com  Fri Dec 12 01:10:29 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Fri, 12 Dec 2008 00:10:29 -0600
Subject: [pacman-dev] [PATCH] Make the repo-add quiet flag less quiet
In-Reply-To: <1229061739-14685-1-git-send-email-aaronmgriffin@gmail.com>
References: <1229061739-14685-1-git-send-email-aaronmgriffin@gmail.com>
Message-ID: <449c10960812112210w6697c7f3n8365e0abc6bb533@mail.gmail.com>

On Fri, Dec 12, 2008 at 12:02 AM,  <aaronmgriffin at gmail.com> wrote:
> From: Aaron Griffin <aaronmgriffin at gmail.com>
>
> Considering one can easily run:
>   repo-add .... >/dev/null
> to get only warnings and errors, the -q flag is mostly useless.
>
> Make the -q flag silence only level 2 messages
Seems fine to me, if no one else has objections.

>
> Signed-off-by: Aaron Griffin <aaronmgriffin at gmail.com>
> ---
>  scripts/repo-add.sh.in |    5 ++---
>  1 files changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/scripts/repo-add.sh.in b/scripts/repo-add.sh.in
> index 4f9639c..c89e2a5 100644
> --- a/scripts/repo-add.sh.in
> +++ b/scripts/repo-add.sh.in
> @@ -34,7 +34,6 @@ REPO_DB_FILE=""
>  umask 0022
>
>  msg() {
> -       [ $QUIET -ne 0 ] && return
>        local mesg=$1; shift
>        printf "==> ${mesg}\n" "$@" >&1
>  }
> @@ -68,8 +67,8 @@ repo-remove will update a package database by removing the package name\n\
>  specified on the command line from the given repo database. Multiple\n\
>  packages to remove can be specified on the command line.\n\n")"
>        printf "$(gettext "\
> -The -q/--quiet flag to either program will force silent running except\n\
> -in the case of warnings or errors.\n\n")"
> +Use the -q/--quiet flag to minimize output to basic messages, warnings,\n\
> +and errors\n\n")"
>        echo "$(gettext "Example:  repo-add /path/to/repo.db.tar.gz pacman-3.0.0.pkg.tar.gz")"
>        echo "$(gettext "Example:  repo-remove /path/to/repo.db.tar.gz kernel26")"
>  }
> --
> 1.6.0.5
>
> _______________________________________________
> pacman-dev mailing list
> pacman-dev at archlinux.org
> http://archlinux.org/mailman/listinfo/pacman-dev
>

From aaronmgriffin at gmail.com  Fri Dec 12 01:11:31 2008
From: aaronmgriffin at gmail.com (Aaron Griffin)
Date: Fri, 12 Dec 2008 00:11:31 -0600
Subject: [pacman-dev] [PATCH] Make the repo-add quiet flag less quiet
In-Reply-To: <449c10960812112210w6697c7f3n8365e0abc6bb533@mail.gmail.com>
References: <1229061739-14685-1-git-send-email-aaronmgriffin@gmail.com>
	<449c10960812112210w6697c7f3n8365e0abc6bb533@mail.gmail.com>
Message-ID: <d64a48980812112211l4d9a63a7g373a51185043b1cc@mail.gmail.com>

On Fri, Dec 12, 2008 at 12:10 AM, Dan McGee <dpmcgee at gmail.com> wrote:
> On Fri, Dec 12, 2008 at 12:02 AM,  <aaronmgriffin at gmail.com> wrote:
>> From: Aaron Griffin <aaronmgriffin at gmail.com>
>>
>> Considering one can easily run:
>>   repo-add .... >/dev/null
>> to get only warnings and errors, the -q flag is mostly useless.
>>
>> Make the -q flag silence only level 2 messages
> Seems fine to me, if no one else has objections.

Note that the docs changed in there, and they're gettext strings

>
>>
>> Signed-off-by: Aaron Griffin <aaronmgriffin at gmail.com>
>> ---
>>  scripts/repo-add.sh.in |    5 ++---
>>  1 files changed, 2 insertions(+), 3 deletions(-)
>>
>> diff --git a/scripts/repo-add.sh.in b/scripts/repo-add.sh.in
>> index 4f9639c..c89e2a5 100644
>> --- a/scripts/repo-add.sh.in
>> +++ b/scripts/repo-add.sh.in
>> @@ -34,7 +34,6 @@ REPO_DB_FILE=""
>>  umask 0022
>>
>>  msg() {
>> -       [ $QUIET -ne 0 ] && return
>>        local mesg=$1; shift
>>        printf "==> ${mesg}\n" "$@" >&1
>>  }
>> @@ -68,8 +67,8 @@ repo-remove will update a package database by removing the package name\n\
>>  specified on the command line from the given repo database. Multiple\n\
>>  packages to remove can be specified on the command line.\n\n")"
>>        printf "$(gettext "\
>> -The -q/--quiet flag to either program will force silent running except\n\
>> -in the case of warnings or errors.\n\n")"
>> +Use the -q/--quiet flag to minimize output to basic messages, warnings,\n\
>> +and errors\n\n")"
>>        echo "$(gettext "Example:  repo-add /path/to/repo.db.tar.gz pacman-3.0.0.pkg.tar.gz")"
>>        echo "$(gettext "Example:  repo-remove /path/to/repo.db.tar.gz kernel26")"
>>  }
>> --
>> 1.6.0.5

From gerbra at archlinux.de  Sat Dec 13 07:23:08 2008
From: gerbra at archlinux.de (Gerhard Brauer)
Date: Sat, 13 Dec 2008 13:23:08 +0100
Subject: [pacman-dev] GPG work
In-Reply-To: <449c10960812080434w6ba33471k9fd55ae681b69cf3@mail.gmail.com>
References: <449c10960812071318v7a7fb427nb19f4f5726496234@mail.gmail.com>
	<20081208115553.3c2fd3d9@tux1.brauer.lan>
	<449c10960812080434w6ba33471k9fd55ae681b69cf3@mail.gmail.com>
Message-ID: <20081213132308.5e8c5b4e@tux1.brauer.lan>

Am Mon, 8 Dec 2008 06:34:06 -0600
schrieb "Dan McGee" <dpmcgee at gmail.com>:

Hi Dan, Hi Developers!

> I didn't promise this worked out of the box- I just meant that it was
> a better start than the other code. You're either going to have to
> know C and understand what is going on (and fix it), or wait for it to
> be in a better state of completion.

I know i make myself very unpopulary with this...

Anything new which could be tested?

As far as i see we have a very good start shortly after the discussion
about "signing". Now a stagnation. I'm aware that the pacman developers
are also short in time. Should maybe the git head from Dan's repo
(newgpg) posted on arch public-devel or arch-general so that C
programmer could have look?

Or what is with the idea to start first with package database signing?
If package signing is hard to implement maybe we shouldn't make the
second step before the first.

> -Dan

Regards
	Gerhard

From dpmcgee at gmail.com  Sat Dec 13 09:18:15 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Sat, 13 Dec 2008 08:18:15 -0600
Subject: [pacman-dev] GPG work
In-Reply-To: <20081213132308.5e8c5b4e@tux1.brauer.lan>
References: <449c10960812071318v7a7fb427nb19f4f5726496234@mail.gmail.com>
	<20081208115553.3c2fd3d9@tux1.brauer.lan>
	<449c10960812080434w6ba33471k9fd55ae681b69cf3@mail.gmail.com>
	<20081213132308.5e8c5b4e@tux1.brauer.lan>
Message-ID: <449c10960812130618g1d9625d8y4aba0df77b37e9d5@mail.gmail.com>

On Sat, Dec 13, 2008 at 6:23 AM, Gerhard Brauer <gerbra at archlinux.de> wrote:
> As far as i see we have a very good start shortly after the discussion
> about "signing". Now a stagnation. I'm aware that the pacman developers
> are also short in time. Should maybe the git head from Dan's repo
> (newgpg) posted on arch public-devel or arch-general so that C
> programmer could have look?

Simple fact- there is no one that is both interested AND can program in C.

If I attached a $500 bounty, we might get results. Until then, we just
don't have enough people in the community actually interested in
working on pacman- just those that ask for new features. This code has
been posted on every single mailing list, every developer has seen it,
etc. and yet we still have no one else working on it. I can't quit my
day job.

-Dan

From thomas at archlinux.org  Sat Dec 13 10:45:29 2008
From: thomas at archlinux.org (=?ISO-8859-15?Q?Thomas_B=E4chler?=)
Date: Sat, 13 Dec 2008 16:45:29 +0100
Subject: [pacman-dev] GPG work
In-Reply-To: <d64a48980812080836j2a5d9b73x19b025d685a8100f@mail.gmail.com>
References: <449c10960812071318v7a7fb427nb19f4f5726496234@mail.gmail.com>
	<20081208115553.3c2fd3d9@tux1.brauer.lan>
	<449c10960812080434w6ba33471k9fd55ae681b69cf3@mail.gmail.com>
	<b245135e0812080500s345f706dw98d32443bb7515a8@mail.gmail.com>
	<449c10960812080508q790bf7c4ud178139694701177@mail.gmail.com>
	<20081208150019.GM517@lynn.lan>
	<d64a48980812080836j2a5d9b73x19b025d685a8100f@mail.gmail.com>
Message-ID: <4943D899.1020708@archlinux.org>

Aaron Griffin schrieb:
> Forcing md5sum collisions requires arbitrary null padding. tar can (I
> think) support this, but not if it's compressed. You can't arbitrarily
> put nulls in the middle of a gzip'd stream...

You can put arbitrary data at the end of a gzip archive. This has been 
used here: 
http://web.archive.org/web/20011212144451/fatphil.org/maths/illegal1.html

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 260 bytes
Desc: OpenPGP digital signature
URL: <http://archlinux.org/pipermail/pacman-dev/attachments/20081213/ed3c8cb6/attachment.pgp>

From sivertb at stud.ntnu.no  Sat Dec 13 11:35:34 2008
From: sivertb at stud.ntnu.no (Sivert Berg)
Date: Sat, 13 Dec 2008 17:35:34 +0100
Subject: [pacman-dev] [PATCH] Added a new database backend
Message-ID: <1229186134-993-1-git-send-email-sivertb@stud.ntnu.no>

This is an attempt to add a one-file-per-database backend. This leads to
significatly faster loading of the database when the database is not
cached. As you can see from the patch minimal changes to the rest of
ALPM was needed. The only change was to make reading of the changelog
files happen through the backend. The database is basically a scaled
down ext2-like filesystem. To make corruption detection possible in an
early phase I have planned to add a checksum, but have not come that far
yet. I have been using it on my laptop for a week or so now, and
everything seems to be working as expected. I'm sorry it became such a
large patch, but about half of it is mostly old code (be_packed.c is
essentially be_files.c with fopen fprintf etc changed with new
functions).

Signed-off-by: Sivert Berg <sivertb at stud.ntnu.no>
---
 configure.ac            |    7 +
 lib/libalpm/Makefile.am |    2 +-
 lib/libalpm/be_files.c  |   21 ++
 lib/libalpm/be_packed.c |  845 +++++++++++++++++++++++++++++++++++++++++++++++
 lib/libalpm/be_packed.h |  757 ++++++++++++++++++++++++++++++++++++++++++
 lib/libalpm/db.h        |    3 +
 lib/libalpm/package.c   |   12 +-
 src/util/Makefile.am    |    4 +-
 src/util/convdb.c       |  236 +++++++++++++
 9 files changed, 1876 insertions(+), 11 deletions(-)
 create mode 100644 lib/libalpm/be_packed.c
 create mode 100644 lib/libalpm/be_packed.h
 create mode 100644 src/util/convdb.c

diff --git a/configure.ac b/configure.ac
index c7841b8..6e28ac9 100644
--- a/configure.ac
+++ b/configure.ac
@@ -73,6 +73,12 @@ AC_ARG_WITH(root-dir,
 	AS_HELP_STRING([--with-root-dir=path], [set the location of pacman's root operating directory]),
 	[ROOTDIR=$withval], [ROOTDIR=/])
 
+# Help line for database backend
+AC_ARG_WITH(backend,
+	AS_HELP_STRING([--with-backend=backend], [choose which backend to use. You can choose between 'files' and 'packed', it defaults to files]),
+	[BACKEND=$withval], [BACKEND=files])
+AC_SUBST(BACKEND)
+
 # Help line for package extension
 AC_ARG_WITH(pkg-ext,
 	AS_HELP_STRING([--with-pkg-ext=ext], [set the file extension used by packages]),
@@ -362,6 +368,7 @@ ${PACKAGE_NAME}:
     package extension      : ${PKGEXT}
     source pkg extension   : ${SRCEXT}
     database extension     : ${DBEXT}
+    database backend       : ${BACKEND}
 
   Compilation options:
     Run make in doc/ dir   : ${wantdoc}
diff --git a/lib/libalpm/Makefile.am b/lib/libalpm/Makefile.am
index 871855e..79b9d26 100644
--- a/lib/libalpm/Makefile.am
+++ b/lib/libalpm/Makefile.am
@@ -25,7 +25,7 @@ libalpm_la_SOURCES = \
 	alpm.h alpm.c \
 	alpm_list.h alpm_list.c \
 	backup.h backup.c \
-	be_files.c \
+	be_ at BACKEND@.c \
 	be_package.c \
 	cache.h cache.c \
 	conflict.h conflict.c \
diff --git a/lib/libalpm/be_files.c b/lib/libalpm/be_files.c
index b9ff646..81d7a17 100644
--- a/lib/libalpm/be_files.c
+++ b/lib/libalpm/be_files.c
@@ -874,4 +874,25 @@ int _alpm_db_remove(pmdb_t *db, pmpkg_t *info)
 	return(ret);
 }
 
+void *_alpm_db_changelog_open(pmdb_t *db, pmpkg_t *pkg)
+{
+	char clfile[PATH_MAX];
+	snprintf(clfile, PATH_MAX, "%s/%s/%s-%s/changelog",
+			alpm_option_get_dbpath(),
+			alpm_db_get_name(handle->db_local),
+			alpm_pkg_get_name(pkg),
+			alpm_pkg_get_version(pkg));
+	return fopen(clfile, "r");
+}
+
+int _alpm_db_changelog_close(pmdb_t *db, void *stream)
+{
+	return fclose(stream);
+}
+
+size_t _alpm_db_changelog_read(pmdb_t *db, const void *pf, void *ptr, size_t size)
+{
+	return fread(ptr, 1, size, pf);
+}
+
 /* vim: set ts=2 sw=2 noet: */
diff --git a/lib/libalpm/be_packed.c b/lib/libalpm/be_packed.c
new file mode 100644
index 0000000..3e4ea68
--- /dev/null
+++ b/lib/libalpm/be_packed.c
@@ -0,0 +1,845 @@
+/*
+ *  be_packed.c
+ *
+ *  Copyright (c) 2006 by Christian Hamar <krics at linuxforum.hu>
+ *  Copyright (c) 2006 by Miklos Vajna <vmiklos at frugalware.org>
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation; either version 2 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "config.h"
+
+#include <unistd.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <errno.h>
+#include <string.h>
+#include <stdint.h> /* uintmax_t, intmax_t */
+#include <sys/stat.h>
+#include <dirent.h>
+#include <ctype.h>
+#include <time.h>
+#include <utime.h>
+#include <limits.h> /* PATH_MAX */
+#include <locale.h> /* setlocale */
+
+/* libalpm */
+#include "db.h"
+#include "alpm_list.h"
+#include "cache.h"
+#include "log.h"
+#include "util.h"
+#include "alpm.h"
+#include "handle.h"
+#include "package.h"
+#include "delta.h"
+#include "deps.h"
+#include "dload.h"
+
+#include "be_packed.h"
+
+/** Update a package database
+ * @param force if true, then forces the update, otherwise update only in case
+ * the database isn't up to date
+ * @param db pointer to the package database to update
+ * @return 0 on success, > 0 on error (pm_errno is set accordingly), < 0 if up
+ * to date
+ */
+int SYMEXPORT alpm_db_update(int force, pmdb_t *db)
+{
+	char *dbfile, *dbfilepath;
+	time_t newmtime = 0;
+	char dbpath[PATH_MAX];
+	char dbname[PATH_MAX];
+	char buffer[1024];
+	size_t len;
+	struct stat st;
+	struct utimbuf utb;
+
+	int ret;
+
+	ALPM_LOG_FUNC;
+
+	/* Sanity checks */
+	ASSERT(handle != NULL, RET_ERR(PM_ERR_HANDLE_NULL, -1));
+	ASSERT(db != NULL && db != handle->db_local, RET_ERR(PM_ERR_WRONG_ARGS, -1));
+	/* Verify we are in a transaction.  This is done _mainly_ because we need a DB
+	 * lock - if we update without a db lock, we may kludge some other pacman
+	 * process that _has_ a lock.
+	 */
+	ASSERT(handle->trans != NULL, RET_ERR(PM_ERR_TRANS_NULL, -1));
+	ASSERT(handle->trans->state == STATE_INITIALIZED, RET_ERR(PM_ERR_TRANS_NOT_INITIALIZED, -1));
+	ASSERT(handle->trans->type == PM_TRANS_TYPE_SYNC, RET_ERR(PM_ERR_TRANS_TYPE, -1));
+
+	if(!alpm_list_find_ptr(handle->dbs_sync, db)) {
+		RET_ERR(PM_ERR_DB_NOT_FOUND, -1);
+	}
+
+	strncpy(dbname, db->path, PATH_MAX - 3);
+	/* Strip of the trailing / */
+	dbname[strlen(dbname) - 1] = '\0';
+	strcat(dbname, ".db");
+	stat(dbname, &st);
+
+	if(!force) {
+		/* get the lastupdate time */
+		if(st.st_mtime == 0) {
+			_alpm_log(PM_LOG_DEBUG, "failed to get last write time for %s\n",
+					db->treename);
+		}
+	}
+	else
+		st.st_mtime = 0;
+
+	len = strlen(db->treename) + strlen(DBEXT) + 1;
+	MALLOC(dbfile, len, RET_ERR(PM_ERR_MEMORY, -1));
+	sprintf(dbfile, "%s" DBEXT, db->treename);
+
+	strncpy(dbpath, alpm_option_get_dbpath(), PATH_MAX);
+	strcpy(dbpath + strlen(dbpath), "sync/");
+
+	ret = _alpm_download_single_file(dbfile, db->servers, dbpath,
+			st.st_mtime, &newmtime);
+	free(dbfile);
+
+	if(ret == 1) {
+		/* mtimes match, do nothing */
+		pm_errno = 0;
+		return(1);
+	} else if(ret == -1) {
+		/* pm_errno was set by the download code */
+		_alpm_log(PM_LOG_DEBUG, "failed to sync db: %s\n", alpm_strerrorlast());
+		return(-1);
+	} else {
+		/* remove the old db */
+		_alpm_log(PM_LOG_DEBUG, "removing database %s\n", db->path);
+		unlink(dbname);
+
+		/* cache needs to be rebuilt */
+		_alpm_db_free_pkgcache(db);
+
+		/* form the path to the db location */
+		len = strlen(dbpath) + strlen(db->treename) + strlen(DBEXT) + 1;
+		MALLOC(dbfilepath, len, RET_ERR(PM_ERR_MEMORY, -1));
+		sprintf(dbfilepath, "%s%s" DBEXT, dbpath, db->treename);
+
+		/* convert the database file */
+		snprintf(buffer, 1024, "convdb %s", dbfilepath);
+		system(buffer);
+
+		/* set the correct time on the new database */
+		utb.actime = newmtime;
+		utb.modtime = newmtime;
+		utime(dbname, &utb);
+
+		/* Reopen the database */
+		_pack_db_close(db->handle);
+		db->handle = _pack_db_open(dbname);
+
+		unlink(dbfilepath);
+		free(dbfilepath);
+	}
+
+	return(0);
+}
+
+int _alpm_db_open(pmdb_t *db)
+{
+	ALPM_LOG_FUNC;
+	char path[PATH_MAX];
+
+	if(db == NULL) {
+		RET_ERR(PM_ERR_DB_NULL, -1);
+	}
+
+	strncpy(path, db->path, PATH_MAX - 3);
+	/* Strip of the trailing / */
+	path[strlen(path) - 1] = '\0';
+	strcat(path, ".db");
+
+	_alpm_log(PM_LOG_DEBUG, "opening database from path '%s'\n", db->path);
+	db->handle = _pack_db_open(path);
+
+	if(db->handle == NULL) {
+		RET_ERR(PM_ERR_DB_OPEN, -1);
+	}
+
+	return(0);
+}
+
+void _alpm_db_close(pmdb_t *db)
+{
+	ALPM_LOG_FUNC;
+
+	if(db == NULL) {
+		return;
+	}
+
+	if(db->handle) {
+		_pack_db_close(db->handle);
+		db->handle = NULL;
+	}
+}
+
+static int splitname(const char *target, pmpkg_t *pkg)
+{
+	/* the format of a db entry is as follows:
+	 *    package-version-rel/
+	 * package name can contain hyphens, so parse from the back- go back
+	 * two hyphens and we have split the version from the name.
+	 */
+	char *tmp, *p, *q;
+
+	if(target == NULL || pkg == NULL) {
+		return(-1);
+	}
+	STRDUP(tmp, target, RET_ERR(PM_ERR_MEMORY, -1));
+	p = tmp + strlen(tmp);
+
+	/* do the magic parsing- find the beginning of the version string
+	 * by doing two iterations of same loop to lop off two hyphens */
+	for(q = --p; *q && *q != '-'; q--);
+	for(p = --q; *p && *p != '-'; p--);
+	if(*p != '-' || p == tmp) {
+		return(-1);
+	}
+
+	/* copy into fields and return */
+	if(pkg->version) {
+		FREE(pkg->version);
+	}
+	STRDUP(pkg->version, p+1, RET_ERR(PM_ERR_MEMORY, -1));
+	/* insert a terminator at the end of the name (on hyphen)- then copy it */
+	*p = '\0';
+	if(pkg->name) {
+		FREE(pkg->name);
+	}
+	STRDUP(pkg->name, tmp, RET_ERR(PM_ERR_MEMORY, -1));
+
+	free(tmp);
+	return(0);
+}
+
+int _alpm_db_populate(pmdb_t *db)
+{
+	int count = 0;
+	int i;
+	char name[PATH_MAX];
+	pack_dir_t *dir;
+
+	ALPM_LOG_FUNC;
+
+	ASSERT(db != NULL, RET_ERR(PM_ERR_DB_NULL, -1));
+
+	dir = _pack_open_dir(db->handle);
+
+	while(_pack_dir_next(db->handle, dir, name, PATH_MAX) != -1) {
+		pmpkg_t *pkg;
+
+		pkg = _alpm_pkg_new();
+		if(pkg == NULL) {
+			return(-1);
+		}
+
+		/* Deal only with desc entries, that way we're sure we do
+		 * not add the same package twice
+		 */
+		if(strstr(name, "/desc") == NULL)
+			continue;
+
+		for (i = 0; name[i] && name[i] != '/'; i++);
+		name[i] = '\0';
+
+		/* split the db entry name */
+		if(splitname(name, pkg) != 0) {
+			_alpm_log(PM_LOG_ERROR, _("invalid name for database entry '%s'\n"),
+					name);
+			_alpm_pkg_free(pkg);
+			continue;
+		}
+
+		/* explicitly read with only 'BASE' data, accessors will handle the rest */
+		if(_alpm_db_read(db, pkg, INFRQ_BASE) == -1) {
+			_alpm_log(PM_LOG_ERROR, _("corrupted database entry '%s'\n"), name);
+			_alpm_pkg_free(pkg);
+			continue;
+		}
+		pkg->origin = PKG_FROM_CACHE;
+		pkg->origin_data.db = db;
+		/* add to the collection */
+		_alpm_log(PM_LOG_FUNCTION, "adding '%s' to package cache for db '%s'\n",
+				pkg->name, db->treename);
+		db->pkgcache = alpm_list_add(db->pkgcache, pkg);
+		count++;
+	}
+
+	db->pkgcache = alpm_list_msort(db->pkgcache, count, _alpm_pkg_cmp);
+	return(count);
+}
+
+int _alpm_db_read(pmdb_t *db, pmpkg_t *info, pmdbinfrq_t inforeq)
+{
+	char line[513];
+	char pkgname[PATH_MAX];
+	pack_file_t *pf;
+	ALPM_LOG_FUNC;
+
+	if(db == NULL) {
+		RET_ERR(PM_ERR_DB_NULL, -1);
+	}
+
+	if(info == NULL || info->name == NULL || info->version == NULL) {
+		_alpm_log(PM_LOG_DEBUG, "invalid package entry provided to _alpm_db_read, skipping\n");
+		return(-1);
+	}
+
+	if(info->origin == PKG_FROM_FILE) {
+		_alpm_log(PM_LOG_DEBUG, "request to read database info for a file-based package '%s', skipping...\n", info->name);
+		return(-1);
+	}
+
+	/* bitmask logic here:
+	 * infolevel: 00001111
+	 * inforeq:   00010100
+	 * & result:  00000100
+	 * == to inforeq? nope, we need to load more info. */
+	if((info->infolevel & inforeq) == inforeq) {
+		/* already loaded this info, do nothing */
+		return(0);
+	}
+	_alpm_log(PM_LOG_FUNCTION, "loading package data for %s : level=0x%x\n",
+			info->name, inforeq);
+
+	/* clear out 'line', to be certain - and to make valgrind happy */
+	memset(line, 0, 513);
+	snprintf(pkgname, PATH_MAX, "%s-%s", info->name, info->version);
+
+	/* DESC */
+	if(inforeq & INFRQ_DESC) {
+		if((pf = _pack_open_file(db->handle, pkgname, "desc", PO_READ)) == NULL) {
+			_alpm_log(PM_LOG_ERROR, _("could not open package %s\n"), pkgname);
+			goto error;
+		}
+		while(!_pack_file_eof(db->handle, pf)) {
+			if(_pack_file_gets(db->handle, pf, line, 256) == 0) {
+				break;
+			}
+			_alpm_strtrim(line);
+			if(strcmp(line, "%NAME%") == 0) {
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				if(strcmp(_alpm_strtrim(line), info->name) != 0) {
+					_alpm_log(PM_LOG_ERROR, _("%s database is inconsistent: name "
+								"mismatch on package %s %s\n"), db->treename, info->name, line);
+				}
+			} else if(strcmp(line, "%VERSION%") == 0) {
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				if(strcmp(_alpm_strtrim(line), info->version) != 0) {
+					_alpm_log(PM_LOG_ERROR, _("%s database is inconsistent: version "
+								"mismatch on package %s\n"), db->treename, info->name);
+				}
+			} else if(strcmp(line, "%FILENAME%") == 0) {
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				STRDUP(info->filename, _alpm_strtrim(line), goto error);
+			} else if(strcmp(line, "%DESC%") == 0) {
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				STRDUP(info->desc, _alpm_strtrim(line), goto error);
+			} else if(strcmp(line, "%GROUPS%") == 0) {
+				while(_pack_file_gets(db->handle, pf, line, 512) &&
+						strlen(_alpm_strtrim(line))) {
+					char *linedup;
+					STRDUP(linedup, _alpm_strtrim(line), goto error);
+					info->groups = alpm_list_add(info->groups, linedup);
+				}
+			} else if(strcmp(line, "%URL%") == 0) {
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				STRDUP(info->url, _alpm_strtrim(line), goto error);
+			} else if(strcmp(line, "%LICENSE%") == 0) {
+				while(_pack_file_gets(db->handle, pf, line, 512) &&
+						strlen(_alpm_strtrim(line))) {
+					char *linedup;
+					STRDUP(linedup, _alpm_strtrim(line), goto error);
+					info->licenses = alpm_list_add(info->licenses, linedup);
+				}
+			} else if(strcmp(line, "%ARCH%") == 0) {
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				STRDUP(info->arch, _alpm_strtrim(line), goto error);
+			} else if(strcmp(line, "%BUILDDATE%") == 0) {
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				_alpm_strtrim(line);
+
+				char first = tolower(line[0]);
+				if(first > 'a' && first < 'z') {
+					struct tm tmp_tm = {0}; //initialize to null incase of failure
+					setlocale(LC_TIME, "C");
+					strptime(line, "%a %b %e %H:%M:%S %Y", &tmp_tm);
+					info->builddate = mktime(&tmp_tm);
+					setlocale(LC_TIME, "");
+				} else {
+					info->builddate = atol(line);
+				}
+			} else if(strcmp(line, "%INSTALLDATE%") == 0) {
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				_alpm_strtrim(line);
+
+				char first = tolower(line[0]);
+				if(first > 'a' && first < 'z') {
+					struct tm tmp_tm = {0}; //initialize to null incase of failure
+					setlocale(LC_TIME, "C");
+					strptime(line, "%a %b %e %H:%M:%S %Y", &tmp_tm);
+					info->installdate = mktime(&tmp_tm);
+					setlocale(LC_TIME, "");
+				} else {
+					info->installdate = atol(line);
+				}
+			} else if(strcmp(line, "%PACKAGER%") == 0) {
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				STRDUP(info->packager, _alpm_strtrim(line), goto error);
+			} else if(strcmp(line, "%REASON%") == 0) {
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				info->reason = atol(_alpm_strtrim(line));
+			} else if(strcmp(line, "%SIZE%") == 0 || strcmp(line, "%CSIZE%") == 0) {
+				/* NOTE: the CSIZE and SIZE fields both share the "size" field
+				 *       in the pkginfo_t struct.  This can be done b/c CSIZE
+				 *       is currently only used in sync databases, and SIZE is
+				 *       only used in local databases.
+				 */
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				info->size = atol(_alpm_strtrim(line));
+				/* also store this value to isize if isize is unset */
+				if(info->isize == 0) {
+					info->isize = info->size;
+				}
+			} else if(strcmp(line, "%ISIZE%") == 0) {
+				/* ISIZE (installed size) tag only appears in sync repositories,
+				 * not the local one. */
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				info->isize = atol(_alpm_strtrim(line));
+			} else if(strcmp(line, "%MD5SUM%") == 0) {
+				/* MD5SUM tag only appears in sync repositories,
+				 * not the local one. */
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				STRDUP(info->md5sum, _alpm_strtrim(line), goto error);
+			} else if(strcmp(line, "%REPLACES%") == 0) {
+				while(_pack_file_gets(db->handle, pf, line, 512) &&
+						strlen(_alpm_strtrim(line))) {
+					char *linedup;
+					STRDUP(linedup, _alpm_strtrim(line), goto error);
+					info->replaces = alpm_list_add(info->replaces, linedup);
+				}
+			} else if(strcmp(line, "%FORCE%") == 0) {
+				info->force = 1;
+			}
+		}
+		_pack_close_file(pf);
+		pf = NULL;
+	}
+
+	/* FILES */
+	if(inforeq & INFRQ_FILES) {
+		if((pf = _pack_open_file(db->handle, pkgname, "files", PO_READ)) == NULL) {
+			_alpm_log(PM_LOG_ERROR, _("could not open package %s\n"), pkgname);
+			goto error;
+		}
+		while(_pack_file_gets(db->handle, pf, line, 256) &&
+			strlen(_alpm_strtrim(line))) {
+			_alpm_strtrim(line);
+			if(strcmp(line, "%FILES%") == 0) {
+				while(_pack_file_gets(db->handle, pf, line, 512) &&
+						strlen(_alpm_strtrim(line))) {
+					char *linedup;
+					STRDUP(linedup, _alpm_strtrim(line), goto error);
+					info->files = alpm_list_add(info->files, linedup);
+				}
+			} else if(strcmp(line, "%BACKUP%") == 0) {
+				while(_pack_file_gets(db->handle, pf, line, 512) &&
+						strlen(_alpm_strtrim(line))) {
+					char *linedup;
+					STRDUP(linedup, _alpm_strtrim(line), goto error);
+					info->backup = alpm_list_add(info->backup, linedup);
+				}
+			}
+		}
+		_pack_close_file(pf);
+		pf = NULL;
+	}
+
+	/* DEPENDS */
+	if(inforeq & INFRQ_DEPENDS) {
+		if((pf = _pack_open_file(db->handle, pkgname, "depends", PO_READ)) == NULL) {
+			_alpm_log(PM_LOG_ERROR, _("could not open package %s\n"), pkgname);
+			goto error;
+		}
+		while(!_pack_file_eof(db->handle, pf)) {
+			_pack_file_gets(db->handle, pf, line, 255);
+			_alpm_strtrim(line);
+			if(strcmp(line, "%DEPENDS%") == 0) {
+				while(_pack_file_gets(db->handle, pf, line, 512) &&
+						strlen(_alpm_strtrim(line))) {
+					pmdepend_t *dep = _alpm_splitdep(_alpm_strtrim(line));
+					info->depends = alpm_list_add(info->depends, dep);
+				}
+			} else if(strcmp(line, "%OPTDEPENDS%") == 0) {
+				while(_pack_file_gets(db->handle, pf, line, 512) &&
+						strlen(_alpm_strtrim(line))) {
+					char *linedup;
+					STRDUP(linedup, _alpm_strtrim(line), goto error);
+					info->optdepends = alpm_list_add(info->optdepends, linedup);
+				}
+			} else if(strcmp(line, "%CONFLICTS%") == 0) {
+				while(_pack_file_gets(db->handle, pf, line, 512) &&
+						strlen(_alpm_strtrim(line))) {
+					char *linedup;
+					STRDUP(linedup, _alpm_strtrim(line), goto error);
+					info->conflicts = alpm_list_add(info->conflicts, linedup);
+				}
+			} else if(strcmp(line, "%PROVIDES%") == 0) {
+				while(_pack_file_gets(db->handle, pf, line, 512) &&
+						strlen(_alpm_strtrim(line))) {
+					char *linedup;
+					STRDUP(linedup, _alpm_strtrim(line), goto error);
+					info->provides = alpm_list_add(info->provides, linedup);
+				}
+			}
+		}
+		_pack_close_file(pf);
+		pf = NULL;
+	}
+
+	/* DELTAS */
+	if(inforeq & INFRQ_DELTAS) {
+		if((pf = _pack_open_file(db->handle, pkgname, "deltas", PO_READ)) != NULL) {
+			while(!_pack_file_eof(db->handle, pf)) {
+				_pack_file_gets(db->handle, pf, line, 255);
+				_alpm_strtrim(line);
+				if(strcmp(line, "%DELTAS%") == 0) {
+					while(_pack_file_gets(db->handle, pf, line, 512) &&
+							strlen(_alpm_strtrim(line))) {
+						pmdelta_t *delta = _alpm_delta_parse(line);
+						if(delta) {
+							info->deltas = alpm_list_add(info->deltas, delta);
+						}
+					}
+				}
+			}
+			_pack_close_file(pf);
+			pf = NULL;
+		}
+	}
+	/* INSTALL */
+	if(inforeq & INFRQ_SCRIPTLET) {
+		if((pf = _pack_open_file(db->handle, pkgname, "depends", PO_READ)) != NULL) {
+			info->scriptlet = 1;
+			_pack_close_file(pf);
+		}
+	}
+
+	/* internal */
+	info->infolevel |= inforeq;
+
+	return(0);
+
+error:
+	if(pf) {
+		_pack_close_file(pf);
+	}
+	return(-1);
+}
+
+int _alpm_db_write(pmdb_t *db, pmpkg_t *info, pmdbinfrq_t inforeq)
+{
+	char pkgname[PATH_MAX];
+	alpm_list_t *lp = NULL;
+	int retval = 0;
+	int local = 0;
+	pack_file_t *pf;
+	char buffer[1024];
+	size_t len;
+
+	ALPM_LOG_FUNC;
+
+	if(db == NULL || info == NULL) {
+		return(-1);
+	}
+
+	if(strcmp(db->treename, "local") == 0) {
+		local = 1;
+	}
+
+	snprintf(pkgname, PATH_MAX, "%s-%s", info->name, info->version);
+
+	/* DESC */
+	if(inforeq & INFRQ_DESC) {
+		_alpm_log(PM_LOG_DEBUG, "writing %s-%s DESC information back to db\n",
+				info->name, info->version);
+		if((pf = _pack_open_file(db->handle, pkgname, "desc", PO_WRITE)) == NULL) {
+			_alpm_log(PM_LOG_ERROR, _("could not open package %s\n"), pkgname);
+			retval = -1;
+			goto cleanup;
+		}
+
+		len = snprintf(buffer, 1024, "%%NAME%%\n%s\n\n"
+						"%%VERSION%%\n%s\n\n", info->name, info->version);
+		_pack_file_puts(db->handle, pf, buffer, len);
+		if(info->desc) {
+			len = snprintf(buffer, 1024, "%%DESC%%\n"
+							"%s\n\n", info->desc);
+			_pack_file_puts(db->handle, pf, buffer, len);
+		}
+		if(info->groups) {
+			_pack_file_puts(db->handle, pf, "%GROUPS%\n", 9);
+			for(lp = info->groups; lp; lp = lp->next) {
+				len = snprintf(buffer, 1024, "%s\n", (char *)lp->data);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			_pack_file_puts(db->handle, pf, "\n", 1);
+		}
+		if(info->replaces) {
+			_pack_file_puts(db->handle, pf, "%REPLACES%\n", 11);
+			for(lp = info->replaces; lp; lp = lp->next) {
+				len = snprintf(buffer, 1024, "%s\n", (char *)lp->data);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			len = snprintf(buffer, 1024, "\n");
+			_pack_file_puts(db->handle, pf, buffer, len);
+		}
+		if(info->force) {
+			len = snprintf(buffer, 1024, "%%FORCE%%\n\n");
+			_pack_file_puts(db->handle, pf, buffer, len);
+		}
+		if(local) {
+			if(info->url) {
+				len = snprintf(buffer, 1024, "%%URL%%\n"
+								"%s\n\n", info->url);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			if(info->licenses) {
+				_pack_file_puts(db->handle, pf, "%LICENSE%\n", 10);
+				for(lp = info->licenses; lp; lp = lp->next) {
+					len = snprintf(buffer, 1024, "%s\n", (char *)lp->data);
+					_pack_file_puts(db->handle, pf, buffer, len);
+				}
+				len = snprintf(buffer, 1024, "\n");
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			if(info->arch) {
+				len = snprintf(buffer, 1024, "%%ARCH%%\n"
+								"%s\n\n", info->arch);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			if(info->builddate) {
+				len = snprintf(buffer, 1024, "%%BUILDDATE%%\n"
+								"%ju\n\n", (uintmax_t)info->builddate);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			if(info->installdate) {
+				len = snprintf(buffer, 1024, "%%INSTALLDATE%%\n"
+								"%ju\n\n", (uintmax_t)info->installdate);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			if(info->packager) {
+				len = snprintf(buffer, 1024, "%%PACKAGER%%\n"
+								"%s\n\n", info->packager);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			if(info->isize) {
+				/* only write installed size, csize is irrelevant once installed */
+				len = snprintf(buffer, 1024, "%%SIZE%%\n"
+								"%ju\n\n", (intmax_t)info->isize);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			if(info->reason) {
+				len = snprintf(buffer, 1024, "%%REASON%%\n"
+								"%u\n\n", info->reason);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+		} else {
+			if(info->size) {
+				len = snprintf(buffer, 1024, "%%CSIZE%%\n"
+								"%ju\n\n", (intmax_t)info->size);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			if(info->isize) {
+				len = snprintf(buffer, 1024, "%%ISIZE%%\n"
+								"%ju\n\n", (intmax_t)info->isize);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			if(info->md5sum) {
+				len = snprintf(buffer, 1024, "%%MD5SUM%%\n"
+								"%s\n\n", info->md5sum);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+		}
+		_pack_close_file(pf);
+		pf = NULL;
+	}
+
+	/* FILES */
+	if(local && (inforeq & INFRQ_FILES)) {
+		_alpm_log(PM_LOG_DEBUG, "writing %s-%s FILES information back to db\n",
+				info->name, info->version);
+		if((pf = _pack_open_file(db->handle, pkgname, "files", PO_WRITE)) == NULL) {
+			_alpm_log(PM_LOG_ERROR, _("could not open package %s\n"), pkgname);
+			retval = -1;
+			goto cleanup;
+		}
+
+		if(info->files) {
+			len = snprintf(buffer, 1024, "%%FILES%%\n");
+			_pack_file_puts(db->handle, pf, buffer, len);
+			for(lp = info->files; lp; lp = lp->next) {
+				len = snprintf(buffer, 1024, "%s\n", (char *)lp->data);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			len = snprintf(buffer, 1024, "\n");
+			_pack_file_puts(db->handle, pf, buffer, len);
+		}
+		if(info->backup) {
+			len = snprintf(buffer, 1024, "%%BACKUP%%\n");
+			_pack_file_puts(db->handle, pf, buffer, len);
+			for(lp = info->backup; lp; lp = lp->next) {
+				len = snprintf(buffer, 1024, "%s\n", (char *)lp->data);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			len = snprintf(buffer, 1024, "\n");
+			_pack_file_puts(db->handle, pf, buffer, len);
+		}
+		_pack_close_file(pf);
+		pf = NULL;
+	}
+
+	/* DEPENDS */
+	if(inforeq & INFRQ_DEPENDS) {
+		_alpm_log(PM_LOG_DEBUG, "writing %s-%s DEPENDS information back to db\n",
+				info->name, info->version);
+		if((pf = _pack_open_file(db->handle, pkgname, "depends", PO_WRITE)) == NULL) {
+			_alpm_log(PM_LOG_ERROR, _("could not open package %s\n"), pkgname);
+			retval = -1;
+			goto cleanup;
+		}
+
+		if(info->depends) {
+			_pack_file_puts(db->handle, pf, "%DEPENDS%\n", 10);
+			for(lp = info->depends; lp; lp = lp->next) {
+				char *depstring = alpm_dep_get_string(lp->data);
+				len = snprintf(buffer, 1024, "%s\n", depstring);
+				_pack_file_puts(db->handle, pf, buffer, len);
+				free(depstring);
+			}
+			len = snprintf(buffer, 1024, "\n");
+			_pack_file_puts(db->handle, pf, buffer, len);
+		}
+		if(info->optdepends) {
+			_pack_file_puts(db->handle, pf, "%OPTDEPENDS%\n", 13);
+			for(lp = info->optdepends; lp; lp = lp->next) {
+				len = snprintf(buffer, 1024, "%s\n", (char *)lp->data);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			len = snprintf(buffer, 1024, "\n");
+			_pack_file_puts(db->handle, pf, buffer, len);
+		}
+		if(info->conflicts) {
+			_pack_file_puts(db->handle, pf, "%CONFLICTS%\n", 12);
+			for(lp = info->conflicts; lp; lp = lp->next) {
+				len = snprintf(buffer, 1024, "%s\n", (char *)lp->data);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			len = snprintf(buffer, 1024, "\n");
+			_pack_file_puts(db->handle, pf, buffer, len);
+		}
+		if(info->provides) {
+			_pack_file_puts(db->handle, pf, "%PROVIDES%\n", 11);
+			for(lp = info->provides; lp; lp = lp->next) {
+				len = snprintf(buffer, 1024, "%s\n", (char *)lp->data);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			len = snprintf(buffer, 1024, "\n");
+			_pack_file_puts(db->handle, pf, buffer, len);
+		}
+		_pack_close_file(pf);
+		pf = NULL;
+	}
+
+	/* INSTALL */
+	/* nothing needed here (script is automatically extracted) */
+
+cleanup:
+	if(pf) {
+		_pack_close_file(pf);
+	}
+
+	return(retval);
+}
+
+int _alpm_db_remove(pmdb_t *db, pmpkg_t *info)
+{
+	int ret = 0;
+	char pkgname[512];
+
+	ALPM_LOG_FUNC;
+
+	snprintf(pkgname, 512, "%s-%s", info->name, info->version);
+	_pack_package_remove(db->handle, pkgname);
+
+	if(db == NULL || info == NULL) {
+		RET_ERR(PM_ERR_DB_NULL, -1);
+	}
+
+	return(ret);
+}
+
+void *_alpm_db_changelog_open(pmdb_t *db, pmpkg_t *pkg)
+{
+	char name[512];
+	snprintf(name, 512, "%s-%s", pkg->name, pkg->version);
+	return _pack_open_file(db->handle, name, "changelog", PO_READ);
+}
+
+int _alpm_db_changelog_close(pmdb_t *db, void *stream)
+{
+	_pack_close_file(stream);
+	return 0;
+}
+
+size_t _alpm_db_changelog_read(pmdb_t *db, const void *pf, void *ptr, size_t size)
+{
+	return _pack_file_gets(db->handle, pf, ptr, size);
+}
+
+/* vim: set ts=2 sw=2 noet: */
diff --git a/lib/libalpm/be_packed.h b/lib/libalpm/be_packed.h
new file mode 100644
index 0000000..e118a3a
--- /dev/null
+++ b/lib/libalpm/be_packed.h
@@ -0,0 +1,757 @@
+/*
+ * Copyright (c) 2008  Sivert Berg
+ *
+ * Permission is hereby granted, free of charge, to any person
+ * obtaining a copy of this software and associated documentation
+ * files (the "Software"), to deal in the Software without
+ * restriction, including without limitation the rights to use,
+ * copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following
+ * conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
+ * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+ * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+ * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+
+/* _HOW IT WORKS_
+ *
+ * If you've ever looked at the ext2 filesystem you will find alot
+ * of similarities (among others the inode that I didn't have the
+ * imagination to find a new name for).
+ *
+ * _GROUP_
+ * Basicly the database is divided into groups. Each group got the
+ * same number of blocks and inodes.
+ *
+ * _INODE_
+ * Each file is assigned an unique inode. The inode got a list of
+ * blocks where the actual data for the file is stored. Worth noting
+ * is that the last entry in the block list is indirect. That means
+ * it points to a block that points to blocks.
+ */
+
+#include <stdint.h>
+#include <sys/mman.h>
+#include <stdlib.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <stdio.h>
+#include <string.h>
+#include <unistd.h>
+
+
+#ifndef BE_PACKED_H
+#define BE_PACKED_H
+
+#define BLOCK_SIZE 256
+#define GROUP_SIZE (BLOCK_SIZE * 4096)
+#define BLOCKS_PER_GROUP 3936
+#define INODES_PER_GROUP 1024
+
+#define MIN(a, b) (((a) < (b))?(a):(b))
+
+/* _pack_open_file flags */
+#define PO_READ 1
+#define PO_WRITE 2
+
+
+typedef struct __inode_t {
+	int32_t size;
+	int32_t block[7];
+} inode_t;
+
+typedef struct __group_desc_t {
+	uint32_t inodes_free;
+	uint32_t blocks_free;
+	uint32_t checksum;
+	inode_t i_table[INODES_PER_GROUP];
+	uint32_t bitmap[BLOCKS_PER_GROUP / 32];
+	char data[];
+} group_desc_t;
+
+typedef struct __package_t {
+	int32_t inode;
+	uint16_t rec_len;
+	uint16_t name_len;
+	char name[];
+} package_t;
+
+typedef struct __pack_db_t {
+	int fd;
+	int groups;
+  int read_only;
+	group_desc_t *map;
+} pack_db_t;
+
+typedef struct __pack_file_t {
+	int inode;
+	size_t pos;
+} pack_file_t;
+
+typedef struct __pack_dir_t {
+	size_t pos;
+} pack_dir_t;
+
+
+/**
+ ** GROUP FUNCTIONS
+ */
+/* Initilizes a new group */
+void _pack_init_group(group_desc_t *group)
+{
+	int i, j;
+
+	for (i = 0; i < INODES_PER_GROUP; i++) {
+		group->i_table[i].size = -1;
+		for (j = 0; j < 7; j++)
+			group->i_table[i].block[j] = -1;
+	}
+
+	for (i = 0; i < BLOCKS_PER_GROUP / 32; i++) {
+		group->bitmap[i] = 0;
+	}
+
+	group->inodes_free = INODES_PER_GROUP;
+	group->blocks_free = BLOCKS_PER_GROUP;
+	group->checksum = 0;
+}
+
+
+/* Adds a new group to the file */
+void _pack_new_group(pack_db_t *db)
+{
+	db->groups++;
+	ftruncate(db->fd, db->groups * GROUP_SIZE);
+	db->map = mremap(db->map, (db->groups - 1) * GROUP_SIZE,
+			db->groups * GROUP_SIZE, MREMAP_MAYMOVE);
+
+	_pack_init_group((group_desc_t*)((char*)db->map +
+				GROUP_SIZE * (db->groups - 1)));
+}
+
+
+/**
+ ** POINTER FUNCTIONS (return pointers to various structures)
+ */
+group_desc_t *_pack_get_group(pack_db_t *db, int group)
+{
+	return (group_desc_t*)((char*)db->map + group*GROUP_SIZE);
+}
+
+
+inode_t *_pack_get_inode(pack_db_t *db, int inode)
+{
+	int group = inode / INODES_PER_GROUP;
+	group_desc_t *groupd = _pack_get_group(db, group);
+	inode %= INODES_PER_GROUP;
+
+	return groupd->i_table + inode;
+}
+
+void *_pack_get_block(pack_db_t *db, int block)
+{
+	if (block == -1)
+		return NULL;
+
+	group_desc_t *group = _pack_get_group(db, block / BLOCKS_PER_GROUP);
+	return group->data + (block % BLOCKS_PER_GROUP) * BLOCK_SIZE;
+}
+
+
+/* Searches for a free block in the bitmap and marks the bit
+ * it finds as used.
+ */
+int _pack_check_bitmap(group_desc_t *group)
+{
+	int i = 0, j = 0, t = 0;
+
+	for (i = 0; i < BLOCKS_PER_GROUP / 32; i++) {
+		if (group->bitmap[i] != 0xffffffff) {
+			t = group->bitmap[i];
+			for (j = 0; j < 32 && (t & 1); j++, t >>= 1);
+			break;
+		}
+	}
+
+	group->bitmap[i] |= 1 << j;
+	return i * 32 + j;
+}
+
+
+/* Clears the n'th bit */
+int _pack_bitmap_free(group_desc_t *group, int n)
+{
+	int i = n / 32;
+	int j = n % 32;
+
+	group->bitmap[i] &= ~(i << j);
+	return 0;
+}
+
+
+/* Returns the number of a new unused block
+ * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP!
+ * that means ANY pointers into the database may be invalid
+ * after you've called this function.
+ */
+int _pack_get_free_block(pack_db_t *db)
+{
+	int i;
+	group_desc_t *group;
+	int ret;
+
+	for (i = 0; i < db->groups; i++) {
+		group = _pack_get_group(db, i);
+		if (group->blocks_free) {
+			group->blocks_free--;
+			ret = _pack_check_bitmap(group);
+			ret += i * BLOCKS_PER_GROUP;
+			return ret;;
+		}
+	}
+
+	_pack_new_group(db);
+	group = _pack_get_group(db, i);
+	group->blocks_free--;
+	ret = _pack_check_bitmap(group);
+	ret += i * BLOCKS_PER_GROUP;
+
+	return ret;
+}
+
+
+/* Frees the block given */
+int _pack_block_free(pack_db_t *db, int block)
+{
+	group_desc_t *group;
+	int g, b;
+
+	g = block / BLOCKS_PER_GROUP;
+	b = block & BLOCKS_PER_GROUP;
+
+	group = _pack_get_group(db, g);
+  group->blocks_free++;
+
+	return _pack_bitmap_free(group, b);
+}
+
+
+/* Returns the number of a new free inode.
+ * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP!
+ * that means ANY pointers into the database may be invalid
+ * after you've called this function.
+ */
+int _pack_get_free_inode(pack_db_t *db)
+{
+	int i = 0, j = 0;
+	group_desc_t *group;
+
+	for (i = 0; i < db->groups; i++) {
+		group = _pack_get_group(db, i);
+		if (group->inodes_free) {
+			group->inodes_free--;
+			for (j = 0; j < INODES_PER_GROUP; j++) {
+				if (group->i_table[j].size == -1) {
+					group->i_table[j].size = 0;
+					return i * INODES_PER_GROUP + j;
+				}
+			}
+		}
+	}
+
+	_pack_new_group(db);
+	group = _pack_get_group(db, i);
+	group->inodes_free--;
+
+	return i * INODES_PER_GROUP;
+}
+
+
+/* Frees an inode and all of the blocks it might be using */
+int _pack_inode_free(pack_db_t *db, int in)
+{
+	inode_t *inode = _pack_get_inode(db, in);
+  group_desc_t *group = _pack_get_group(db, in / INODES_PER_GROUP);
+	int i;
+	int block;
+	int32_t *blocks;
+	int size = BLOCK_SIZE / sizeof(uint32_t) - 1;
+
+	inode->size = -1;
+
+	for (i = 0; i < 6; i++) {
+		if (inode->block[i] != -1) {
+			_pack_block_free(db, inode->block[i]);
+			inode->block[i] = -1;
+		}
+	}
+
+	block = inode->block[6];
+	blocks = _pack_get_block(db, block);
+
+	while(blocks != NULL) {
+		for (i = 0; i < size; i++) {
+			if (blocks[i] != -1)
+				_pack_block_free(db, blocks[i]);
+		}
+
+		_pack_block_free(db, block);
+		block = blocks[size];
+		blocks = _pack_get_block(db, block);
+	}
+
+  group->inodes_free++;
+
+	return 0;
+}
+
+
+/* Returns the number of the block associated with the inode */
+int _pack_inode_get_block(pack_db_t *db, int i, int block)
+{
+	inode_t *inode = _pack_get_inode(db, i);
+	if (block < 6)
+		return inode->block[block];
+
+	block -= 6;
+	int size = BLOCK_SIZE / sizeof(uint32_t) - 1;
+	uint32_t *blocks = _pack_get_block(db, inode->block[6]);
+
+	for (;blocks != NULL && block >= size; block -= size)
+		blocks = _pack_get_block(db, blocks[size]);
+
+	if (blocks == NULL)
+		return -1;
+
+	return blocks[block];
+}
+
+
+/* Returns the number of the block block associated with the inode.
+ * If there is no such block, it gets allocated.
+ * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP!
+ * that means ANY pointers into the database may be invalid
+ * after you've called this function.
+ */
+int _pack_inode_add_block(pack_db_t *db, int in, int block)
+{
+	inode_t *inode = _pack_get_inode(db, in);
+	int tmp;
+
+	if (block < 6) {
+		if (inode->block[block] != -1)
+			return inode->block[block];
+
+		tmp = _pack_get_free_block(db);
+		inode = _pack_get_inode(db, in);
+		return inode->block[block] = tmp;
+	}
+
+	block -= 6;
+	int size = BLOCK_SIZE / sizeof(int32_t) - 1;
+	int i;
+	int old_block;
+	int32_t *blocks;
+
+	if (inode->block[6] == -1) {
+		tmp = _pack_get_free_block(db);
+		inode = _pack_get_inode(db, in);
+		inode->block[6] = tmp;
+		blocks = _pack_get_block(db, inode->block[6]);
+		for (i = 0; i < size + 1; i++)
+			blocks[i] = -1;
+	}
+	else
+		blocks = _pack_get_block(db, inode->block[6]);
+
+	old_block = inode->block[6];
+
+	for (;block >= size; block -= size) {
+		if (blocks[size] == -1) {
+			tmp = _pack_get_free_block(db);
+			blocks = _pack_get_block(db, old_block);
+			blocks[size] = tmp;
+			old_block = tmp;
+			blocks = _pack_get_block(db, tmp);
+			for (i = 0; i < size + 1; i++)
+				blocks[i] = -1;
+		}
+		else {
+			old_block = blocks[size];
+			blocks = _pack_get_block(db, blocks[size]);
+		}
+	}
+
+	if (blocks[block] == -1) {
+		tmp = _pack_get_free_block(db);
+		blocks = _pack_get_block(db, old_block);
+		blocks[block] = tmp;
+	}
+
+	return blocks[block];
+}
+
+/* Returns a pointer to the package with the name pkgname.
+ * Returns NULL if it's not found
+ */
+package_t *_pack_find_package(pack_db_t *db, const char *pkgname)
+{
+	inode_t *inode = _pack_get_inode(db, 0);
+	int s_len = strlen(pkgname);
+	package_t *pkg;
+	int block = 0;
+	int b_pos = 0;
+
+	pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, block));
+
+	while (block < (inode->size / BLOCK_SIZE)) {
+		if (pkg->inode != 0 &&
+				pkg->name_len == s_len &&
+				strncmp(pkgname, pkg->name, s_len) == 0)
+			break;
+
+		b_pos += pkg->rec_len;
+		pkg = (package_t*)((char*)pkg + pkg->rec_len);
+		if (b_pos >= BLOCK_SIZE) {
+			pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, ++block));
+			b_pos = 0;
+		}
+	}
+
+	return pkg;
+}
+
+/* Creates a new package with the name pkgname.
+ * Returns a pointer to the new package.
+ * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP!
+ * that means ANY pointers into the database may be invalid
+ * after you've called this function.
+ */
+package_t *_pack_add_package(pack_db_t *db, const char *pkgname)
+{
+	inode_t *inode = _pack_get_inode(db, 0);
+	int s_len = strlen(pkgname);
+	int len = s_len + sizeof(package_t);
+	package_t *pkg;
+	int block = 0;
+	int b_pos = 0;
+
+	pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, block));
+
+	while (pkg && block < (inode->size / BLOCK_SIZE)) {
+		if (pkg->rec_len >= len && pkg->inode == 0)
+			break;
+
+		b_pos += pkg->rec_len;
+		pkg = (package_t*)((char*)pkg + pkg->rec_len);
+		if (b_pos >= BLOCK_SIZE) {
+			pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, ++block));
+			b_pos = 0;
+		}
+	}
+
+	if (pkg == NULL) {
+		pkg = _pack_get_block(db, _pack_inode_add_block(db, 0, block));
+		pkg->rec_len = BLOCK_SIZE;
+		pkg->inode = 0;
+		inode = _pack_get_inode(db, 0);
+		inode->size += BLOCK_SIZE;
+	}
+
+	strncpy(pkg->name, pkgname, s_len);
+	if (pkg->rec_len > (len + sizeof(package_t) + 10)) {
+		package_t *foo = (package_t*)((char*)pkg + len);
+		foo->rec_len = pkg->rec_len - len;
+		foo->inode = 0;
+		pkg->rec_len = len;
+	}
+
+	pkg->inode = -1;
+	pkg->name_len = s_len;
+
+	return pkg;
+}
+
+
+/* Reads data from the file until buf is full or we hit a newline */
+int _pack_file_gets(pack_db_t *db, pack_file_t *file, char *buf, size_t size)
+{
+	inode_t *inode = _pack_get_inode(db, file->inode);
+	size_t i;
+	size_t b_pos = file->pos % BLOCK_SIZE;
+	const char *src = _pack_get_block(db,
+			_pack_inode_get_block(db, file->inode, file->pos / BLOCK_SIZE));
+	int newline = 0;
+
+	for (i = 0
+			;i < size && file->pos < inode->size && !newline
+			;i++, b_pos++, file->pos++) {
+		if (b_pos >= BLOCK_SIZE) {
+			src = _pack_get_block(db,
+					_pack_inode_get_block(db, file->inode, file->pos / BLOCK_SIZE));
+			b_pos = 0;
+		}
+
+		buf[i] = src[b_pos];
+
+		if (src[b_pos] == '\n')
+			newline = 1;
+	}
+
+	if (i < size - 1)
+		buf[i] = '\0';
+
+	return i;
+}
+
+
+/* Write the data in buf into the file
+ * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP!
+ * that means ANY pointers into the database may be invalid
+ * after you've called this function.
+ */
+int _pack_file_puts(pack_db_t *db, pack_file_t *file,
+		const char *buf, size_t size)
+{
+	size_t i;
+	size_t b_pos = file->pos % BLOCK_SIZE;
+	char *dest = _pack_get_block(db,
+			_pack_inode_add_block(db, file->inode, file->pos / BLOCK_SIZE));
+	int block;
+
+	for (i = 0; i < size; i++, file->pos++, b_pos++)
+	{
+		if (b_pos >= BLOCK_SIZE) {
+			block = _pack_inode_add_block(db, file->inode, file->pos / BLOCK_SIZE);
+			dest = _pack_get_block(db, block);
+			b_pos = 0;
+		}
+		dest[b_pos] = buf[i];
+	}
+
+	inode_t *inode = _pack_get_inode(db, file->inode);
+	inode->size = file->pos;
+
+	return i;
+}
+
+
+int _pack_file_eof(pack_db_t *db, pack_file_t *file)
+{
+	inode_t *inode = _pack_get_inode(db, file->inode);
+	return file->pos >= inode->size;
+}
+
+
+/* Opens a file in the database
+ * flags: if you pass PO_READ, it will return -1
+ * if the file does not exist, if you pass PO_WRITE
+ * the file will be created if it does not exist
+ * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP!
+ * that means ANY pointers into the database may be invalid
+ * after you've called this function.
+ */
+pack_file_t *_pack_open_file(pack_db_t *db, const char *pkgname,
+		const char *filename, int flags)
+{
+	char name[512];
+
+	snprintf(name, 512, "%s/%s", pkgname, filename);
+
+	package_t *pkg = _pack_find_package(db, name);
+	int w = flags & PO_WRITE;
+	int tmp;
+
+	if (w && db->read_only)
+		return NULL;
+
+	if (pkg == NULL && w)
+		pkg = _pack_add_package(db, name);
+	else if (pkg == NULL)
+		return NULL;
+
+	pack_file_t *ret = malloc(sizeof(pack_file_t));
+	ret->pos = 0;
+
+	if (pkg->inode == -1 && w) {
+		tmp = _pack_get_free_inode(db);
+		pkg = _pack_find_package(db, name);
+		ret->inode = pkg->inode = tmp;
+	}
+	else
+		ret->inode = pkg->inode;
+
+	return ret;
+}
+
+
+/* Closes the file given */
+void _pack_close_file(pack_file_t *file)
+{
+	free(file);
+}
+
+
+/* Removes a package along with all it's files */
+int _pack_package_remove(pack_db_t *db, const char *pkgname)
+{
+	inode_t *inode = _pack_get_inode(db, 0);
+	int s_len = strlen(pkgname);
+	package_t *prev, *pkg;
+	int block = 0;
+	int b_pos = 0;
+
+	if (db->read_only)
+		return -1;
+
+	prev = NULL;
+	pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, block));
+
+	while (block < (inode->size / BLOCK_SIZE)) {
+		if (pkg->inode != 0 &&
+				pkg->name_len > s_len &&
+				strncmp(pkgname, pkg->name, s_len) == 0 &&
+				pkg->name[s_len] == '/') {
+			_pack_inode_free(db, pkg->inode);
+
+			if (prev != NULL && prev->inode == 0) {
+				prev->rec_len += pkg->rec_len;
+				pkg = prev;
+			}
+			else
+				pkg->inode = 0;
+		}
+
+		prev = pkg;
+		b_pos += pkg->rec_len;
+		pkg = (package_t*)((char*)pkg + pkg->rec_len);
+		if (b_pos >= BLOCK_SIZE) {
+			pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, ++block));
+			b_pos = 0;
+			prev = NULL;
+		}
+	}
+
+	return 0;
+}
+
+
+/* Gets the next entry in the directory */
+int _pack_dir_next(pack_db_t *db, pack_dir_t *dir, char *name, size_t len)
+{
+	inode_t *inode = _pack_get_inode(db, 0);
+
+	if (dir->pos >= inode->size)
+		return -1;
+
+	int block, b_pos;
+	package_t *pkg;
+
+	do {
+		block = dir->pos / BLOCK_SIZE;
+		b_pos = dir->pos % BLOCK_SIZE;
+		pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, block));
+		if (!pkg)
+			return -1;
+		pkg = (package_t*)((char*)pkg + b_pos);
+		dir->pos += pkg->rec_len;
+	} while (pkg->inode == 0);
+
+	strncpy(name, pkg->name, MIN(len, pkg->name_len));
+	if (len > pkg->name_len)
+		name[pkg->name_len] = '\0';
+
+	return 0;
+}
+
+
+/* Opens the directory for reading */
+pack_dir_t *_pack_open_dir(pack_db_t *db)
+{
+	pack_dir_t *dir = malloc(sizeof(pack_dir_t));
+	dir->pos = 0;
+
+	return dir;
+}
+
+
+/* Closes the given directory */
+void _pack_close_dir(pack_dir_t *dir)
+{
+	free(dir);
+}
+
+
+/* Opens a packed database */
+pack_db_t *_pack_db_open(const char *filename)
+{
+	struct stat st;
+	int is_new = 0;
+
+	pack_db_t *db = malloc(sizeof(pack_db_t));
+	db->fd = open(filename, O_RDWR, 0);
+  db->read_only = 0;
+
+  if (db->fd == -1) {
+		db->fd = open(filename, O_RDONLY, 0);
+		if (db->fd == -1) {
+			db->fd = open(filename, O_RDWR | O_CREAT, 0644);
+
+			if (db->fd == -1) {
+				free(db);
+				printf("Could not open %s\n", filename);
+				return NULL;
+			}
+
+			ftruncate(db->fd, GROUP_SIZE);
+			is_new = 1;
+		}
+	else
+		db->read_only = 1;
+	}
+
+	fstat(db->fd, &st);
+	db->groups = st.st_size / GROUP_SIZE;
+
+  int flags = PROT_READ;
+  if (!db->read_only)
+    flags |= PROT_WRITE;
+
+	db->map = mmap(NULL, GROUP_SIZE * db->groups, flags, MAP_SHARED,
+			db->fd, 0);
+
+	if (db->map == MAP_FAILED) {
+		perror("mmap");
+		free(db);
+		exit(0);
+	}
+
+	if (is_new) {
+		_pack_init_group(db->map);
+		db->map->i_table[0].size = 0;
+	}
+
+	return db;
+}
+
+
+/* Closes a packed database */
+void _pack_db_close(pack_db_t *db)
+{
+	munmap(db->map, db->groups * GROUP_SIZE);
+	close(db->fd);
+	free(db);
+}
+
+#endif /* BE_PACKED_H */
diff --git a/lib/libalpm/db.h b/lib/libalpm/db.h
index 96fac0d..96291fc 100644
--- a/lib/libalpm/db.h
+++ b/lib/libalpm/db.h
@@ -62,6 +62,9 @@ int _alpm_db_populate(pmdb_t *db);
 int _alpm_db_read(pmdb_t *db, pmpkg_t *info, pmdbinfrq_t inforeq);
 int _alpm_db_write(pmdb_t *db, pmpkg_t *info, pmdbinfrq_t inforeq);
 int _alpm_db_remove(pmdb_t *db, pmpkg_t *info);
+void *_alpm_db_changelog_open(pmdb_t *db, pmpkg_t *pkg);
+int _alpm_db_changelog_close(pmdb_t *db, void *stream);
+size_t _alpm_db_changelog_read(pmdb_t *db, const void *pf, void *ptr, size_t size);
 
 #endif /* _ALPM_DB_H */
 
diff --git a/lib/libalpm/package.c b/lib/libalpm/package.c
index ee0ff6f..7ab9710 100644
--- a/lib/libalpm/package.c
+++ b/lib/libalpm/package.c
@@ -447,13 +447,7 @@ void SYMEXPORT *alpm_pkg_changelog_open(pmpkg_t *pkg)
 	ASSERT(pkg != NULL, return(NULL));
 
 	if(pkg->origin == PKG_FROM_CACHE) {
-		char clfile[PATH_MAX];
-		snprintf(clfile, PATH_MAX, "%s/%s/%s-%s/changelog",
-				alpm_option_get_dbpath(),
-				alpm_db_get_name(handle->db_local),
-				alpm_pkg_get_name(pkg),
-				alpm_pkg_get_version(pkg));
-		return fopen(clfile, "r");
+		return(_alpm_db_changelog_open(handle->db_local, pkg));
 	} else if(pkg->origin == PKG_FROM_FILE) {
 		struct archive *archive = NULL;
 		struct archive_entry *entry;
@@ -500,7 +494,7 @@ size_t SYMEXPORT alpm_pkg_changelog_read(void *ptr, size_t size,
 {
 	size_t ret = 0;
 	if(pkg->origin == PKG_FROM_CACHE) {
-		ret = fread(ptr, 1, size, (FILE*)fp);
+		ret = _alpm_db_changelog_read(handle->db_local, fp, ptr, size);
 	} else if(pkg->origin == PKG_FROM_FILE) {
 		ret = archive_read_data((struct archive*)fp, ptr, size);
 	}
@@ -533,7 +527,7 @@ int SYMEXPORT alpm_pkg_changelog_close(const pmpkg_t *pkg, void *fp)
 {
 	int ret = 0;
 	if(pkg->origin == PKG_FROM_CACHE) {
-		ret = fclose((FILE*)fp);
+		ret = _alpm_db_changelog_close(handle->db_local, fp);
 	} else if(pkg->origin == PKG_FROM_FILE) {
 		ret = archive_read_finish((struct archive *)fp);
 	}
diff --git a/src/util/Makefile.am b/src/util/Makefile.am
index 97a0ffa..64e18ad 100644
--- a/src/util/Makefile.am
+++ b/src/util/Makefile.am
@@ -3,7 +3,7 @@ conffile  = ${sysconfdir}/pacman.conf
 dbpath    = ${localstatedir}/lib/pacman/
 cachedir  = ${localstatedir}/cache/pacman/pkg/
 
-bin_PROGRAMS = vercmp testpkg testdb
+bin_PROGRAMS = vercmp testpkg testdb convdb
 
 DEFS = -DLOCALEDIR=\"@localedir@\" \
        -DCONFFILE=\"$(conffile)\" \
@@ -18,6 +18,8 @@ AM_CFLAGS = -pedantic -D_GNU_SOURCE
 vercmp_SOURCES = vercmp.c
 vercmp_LDADD = $(top_builddir)/lib/libalpm/.libs/libalpm.la
 
+convdb_SOURCES = convdb.c
+
 testpkg_SOURCES = testpkg.c
 testpkg_LDADD = $(top_builddir)/lib/libalpm/.libs/libalpm.la
 
diff --git a/src/util/convdb.c b/src/util/convdb.c
new file mode 100644
index 0000000..f1f39bd
--- /dev/null
+++ b/src/util/convdb.c
@@ -0,0 +1,236 @@
+/*
+ * Copyright (c) 2008  Sivert Berg
+ *
+ * Permission is hereby granted, free of charge, to any person
+ * obtaining a copy of this software and associated documentation
+ * files (the "Software"), to deal in the Software without
+ * restriction, including without limitation the rights to use,
+ * copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following
+ * conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
+ * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+ * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+ * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+
+
+/* WARNING: This file is pretty much just a quick hack thrown together
+ * in a hurry. It seems to work, but don't take my word for it.
+ */
+
+#include <be_packed.h>
+#include <dirent.h>
+#include <archive.h>
+#include <archive_entry.h>
+
+
+/* Convert an archieved database into a packed database
+ * Expects a gziped database ending in .db.tar.gz
+ */
+void convert_archive(const char *filename)
+{
+	struct archive_entry *aent = NULL;
+	struct archive *arch = archive_read_new();
+	archive_read_support_compression_gzip(arch);
+	archive_read_support_format_tar(arch);
+	if (archive_read_open_filename(arch, filename, 1) != ARCHIVE_OK) {
+		printf("Could not open archive %s\n", archive_error_string(arch));
+		return;
+	}
+
+	char *db_name = strndup(filename, strlen(filename) - strlen(".tar.gz"));
+	pack_db_t *db = _pack_db_open(db_name);
+	free(db_name);
+	char buffer[4096];
+	size_t size;
+	pack_file_t *pf;
+
+	while (!archive_read_next_header(arch, &aent)) {
+		const char *type = NULL;
+		if (strstr(archive_entry_pathname(aent), "/desc"))
+			type = "desc";
+		else if (strstr(archive_entry_pathname(aent), "/depends"))
+			type = "depends";
+		else if (strstr(archive_entry_pathname(aent), "/install"))
+			type = "install";
+		else if (strstr(archive_entry_pathname(aent), "/files"))
+			type = "files";
+
+		if (type == NULL)
+			continue;
+
+		char *name = strdup(archive_entry_pathname(aent));
+		int i;
+		for (i = 0; name[i] != '/' && name[i]; i++);
+		name[i] = '\0';
+
+		pf = _pack_open_file(db, name, type, PO_WRITE);
+
+		free(name);
+
+		do {
+			size = archive_read_data(arch, buffer, 4096);
+			_pack_file_puts(db, pf, buffer, size);
+		} while (size == 4096);
+
+		_pack_close_file(pf);
+
+	}
+
+	_pack_db_close(db);
+	archive_read_close(arch);
+	archive_read_finish(arch);
+}
+
+
+
+const char *dir_name(const char *full_path)
+{
+	const char *slash = full_path;
+	while (*full_path)
+		if (*full_path++ == '/')
+			slash = full_path;
+	return slash;
+}
+
+
+/* Convert a directory database into a packed database */
+void convert_directory(const char *pathname)
+{
+	char db_name[1024];
+	pack_db_t *db;
+	DIR  *dir = opendir(pathname);
+	struct dirent *dire;
+	char path[1024];
+	char buffer[8196];
+	size_t size;
+	FILE *f;
+	pack_file_t *pf;
+
+	strcpy(db_name, dir_name(pathname));
+	strcat(db_name, ".db");
+	db = _pack_db_open(db_name);
+
+	if (dir == NULL) {
+		perror("opendir");
+		return;
+	}
+
+	dire = readdir(dir);
+	while (dire != NULL) {
+		if (strcmp(dire->d_name, ".") && strcmp(dire->d_name, "..")) {
+
+
+			size_t foo = 0;
+			sprintf(path, "%s/%s/desc", pathname, dire->d_name);
+			f = fopen(path, "r");
+			if (f == NULL)
+				goto depends;
+			pf = _pack_open_file(db, dire->d_name, "desc", PO_WRITE);
+			do {
+				size = fread(buffer, 1, 8196, f);
+				_pack_file_puts(db, pf, buffer, size);
+				foo += size;
+			} while (size == 8196);
+			_pack_close_file(pf);
+			fclose(f);
+depends:
+			sprintf(path, "%s/%s/depends", pathname, dire->d_name);
+			f = fopen(path, "r");
+			if (f == NULL)
+				goto files;
+			pf = _pack_open_file(db, dire->d_name, "depends", PO_WRITE);
+			do {
+				size = fread(buffer, 1, 8196, f);
+				_pack_file_puts(db, pf, buffer, size);
+				foo += size;
+			} while (size == 8196);
+			_pack_close_file(pf);
+			fclose(f);
+files:
+			sprintf(path, "%s/%s/files", pathname, dire->d_name);
+			f = fopen(path, "r");
+			if (f == NULL)
+				goto install;
+			pf = _pack_open_file(db, dire->d_name, "files", PO_WRITE);
+			do {
+				size = fread(buffer, 1, 8196, f);
+				_pack_file_puts(db, pf, buffer, size);
+				foo += size;
+			} while (size == 8196);
+			_pack_close_file(pf);
+			fclose(f);
+install:
+			sprintf(path, "%s/%s/install", pathname, dire->d_name);
+			f = fopen(path, "r");
+			if (f == NULL)
+				goto changelog;
+			pf = _pack_open_file(db, dire->d_name, "install", PO_WRITE);
+			do {
+				size = fread(buffer, 1, 8196, f);
+				_pack_file_puts(db, pf, buffer, size);
+				foo += size;
+			} while (size == 8196);
+			_pack_close_file(pf);
+			fclose(f);
+changelog:
+			sprintf(path, "%s/%s/changelog", pathname, dire->d_name);
+			f = fopen(path, "r");
+			if (f == NULL)
+				goto deltas;
+			pf = _pack_open_file(db, dire->d_name, "changelog", PO_WRITE);
+			do {
+				size = fread(buffer, 1, 8196, f);
+				_pack_file_puts(db, pf, buffer, size);
+				foo += size;
+			} while (size == 8196);
+			_pack_close_file(pf);
+			fclose(f);
+deltas:
+			sprintf(path, "%s/%s/deltas", pathname, dire->d_name);
+			f = fopen(path, "r");
+			if (f == NULL)
+				goto done;
+			pf = _pack_open_file(db, dire->d_name, "deltas", PO_WRITE);
+			do {
+				size = fread(buffer, 1, 8196, f);
+				_pack_file_puts(db, pf, buffer, size);
+				foo += size;
+			} while (size == 8196);
+			_pack_close_file(pf);
+			fclose(f);
+done:
+			pf = NULL;
+		}
+
+		dire = readdir(dir);
+	}
+
+	closedir(dir);
+	_pack_db_close(db);
+}
+
+int main(int argc, char *argv[])
+{
+	if (argc < 2) {
+		printf("Usage: %s <dir/file>\n", argv[0]);
+		exit(0);
+	}
+
+	if (strstr(argv[1], ".tar.gz") != NULL)
+		convert_archive(argv[1]);
+	else
+		convert_directory(argv[1]);
+
+	return 0;
+}
-- 
1.6.0.5


From allan at archlinux.org  Sat Dec 13 21:54:21 2008
From: allan at archlinux.org (Allan McRae)
Date: Sun, 14 Dec 2008 12:54:21 +1000
Subject: [pacman-dev] Order of option parsing and sourcing makepkg.conf
In-Reply-To: <BAY102-DAV3E6C6B3B481D5A1BB321086FA0@phx.gbl>
References: <BAY102-DAV6716983E81A8B8184036F86FA0@phx.gbl>	<BAY102-DAV7449810D3BA002DA9BC1986FA0@phx.gbl>	<449c10960812090640iaf0ce18t806c896a293f3f79@mail.gmail.com>
	<BAY102-DAV3E6C6B3B481D5A1BB321086FA0@phx.gbl>
Message-ID: <BAY102-DAV3289B277E87FF7AB0088F86F70@phx.gbl>

Allan McRae wrote:
> Dan McGee wrote:
>> On Tue, Dec 9, 2008 at 6:55 AM, Allan McRae <allan at archlinux.org> wrote:
>>  
>>> Allan McRae wrote:
>>>    
>>>> Hi,
>>>>
>>>> With the commit which enables us to specify the makepkg config file 
>>>> on the
>>>> command-line
>>>> (http://projects.archlinux.org/?p=pacman.git;a=commitdiff;h=4b183bf9), 
>>>> the
>>>> sourcing of the config file gets moved after the parsing of the 
>>>> options.
>>>>  This creates problems with the --help flag as it needs to know what
>>>> BUILDSCRIPT is defined as.  It also causes problems when specifying a
>>>> different BUILDSCRIPT with -p as this gets set during option 
>>>> parsing then
>>>> gets overwritten with the makepkg config file gets sourced.
>>>>
>>>> I don't see a nice fix for this.  Sourcing the conf file before option
>>>> parsing and then again after if it is changed seems not good to 
>>>> me.  So can
>>>> we back that patch out at least temporarily.
>>>>
>>>> Note that this does not effect maint so the 3.2.2 release will be 
>>>> bug free
>>>> as usual.
>>>>       
>>> Here is a link to the original discussion of this patch:
>>> http://archlinux.org/pipermail/pacman-dev/2008-August/007321.html
>>>     
>>
>> Maybe we should reconsider moving BUILDSCRIPT out of makepkg and into
>> makepkg.conf? This was commit 9c9e18ef32c0cf3, it feels like eons ago.
>> But let's be honest- how often is someone going to change the name of
>> PKGBUILD or should even care about it? If we at least initially define
>> it in makepkg, and allow for overrides in makepkg.conf, it probably
>> isn't the end of the world.
>>   
>
> Well, that would be the easy fix I didn't see...   I have no 
> objections to moving BUILDSCRIPT back into makepkg.  I would be 
> surprised if anyone ever changed it.  And we provide the "-p" flag if 
> someone wants to override it anyway.
>

I was going to fix this issue but I was wanting to know the reasoning 
behind allowing someone to set a different name for the BUILDSCRIPT.  
Was for if another distro using pacman wanted to call it something other 
the PKGBUILD?  I.e. should I autoconf this rather than hard-coding it in 
makepkg?

Allan




From ngaba at bibl.u-szeged.hu  Sun Dec 14 16:41:04 2008
From: ngaba at bibl.u-szeged.hu (Nagy Gabor)
Date: Sun, 14 Dec 2008 22:41:04 +0100
Subject: [pacman-dev] [PATCH] Added a new database backend
In-Reply-To: <1229186134-993-1-git-send-email-sivertb@stud.ntnu.no>
References: <1229186134-993-1-git-send-email-sivertb@stud.ntnu.no>
Message-ID: <1229290864.3213.56.camel@tandk.math.u-szeged.hu>

> This is an attempt to add a one-file-per-database backend. This leads to
> significatly faster loading of the database when the database is not
> cached. As you can see from the patch minimal changes to the rest of
> ALPM was needed. The only change was to make reading of the changelog
> files happen through the backend. The database is basically a scaled
> down ext2-like filesystem. To make corruption detection possible in an
> early phase I have planned to add a checksum, but have not come that far
> yet. I have been using it on my laptop for a week or so now, and
> everything seems to be working as expected. I'm sorry it became such a
> large patch, but about half of it is mostly old code (be_packed.c is
> essentially be_files.c with fopen fprintf etc changed with new
> functions).
> 
> Signed-off-by: Sivert Berg <sivertb at stud.ntnu.no>

First of all, many thanks for your huge work. I will support your
attempt on reworking our db backend stuff. I haven't completely reviewed
your patch yet (it is quite advanced, I guess I will learn a lot from
here), but I have some "pre-reading" comments. (Keep in my mind: I am
just a pacman contributor here.)

1. I like your modification on changelog reading (package.c should not
assume anything about db backend), you may create a separate patch for
this. (We prefer shorter patches.) But we have the same problem with
changelog creation (extraction), which is hardcoded in add.c. And the
same for install scriptlets. The implementation of these fields is quite
hackish, we don't use pmpkg_t and db_write here...

> diff --git a/lib/libalpm/Makefile.am b/lib/libalpm/Makefile.am
> index 871855e..79b9d26 100644
> --- a/lib/libalpm/Makefile.am
> +++ b/lib/libalpm/Makefile.am
> @@ -25,7 +25,7 @@ libalpm_la_SOURCES = \
>  	alpm.h alpm.c \
>  	alpm_list.h alpm_list.c \
>  	backup.h backup.c \
> -	be_files.c \
> +	be_ at BACKEND@.c \
>  	be_package.c \
>  	cache.h cache.c \
>  	conflict.h conflict.c \

2. I think we need a more sophisticated approach here. I would be happy,
if we had database backend plugins (be_files.so, be_packed.so,
be_sqlite.so, ...) for dlopen(), and some way to inform pacman about the
database handler (for example: /var/lib/pacman/local/.backend, which has
one line: packed). I think this is quite straightforward (but needed) to
implement.

3.* Our whole db back-end system needs some redesign (independent from
your work): If we have fast database back-end, I am not sure we need
this ugly pkgcache stuff. Probably pkgcache should be moved to back-end
level (if needed).

4.* When we rework pmpkg_t, we must keep in mind, that at this moment it
is the structure of database entry *and* package file. When we
restructure things, we must find a good place for ".tar.gz".

(* == difficult .-)

Bye



From cmbrannon at cox.net  Sun Dec 14 13:59:39 2008
From: cmbrannon at cox.net (Chris Brannon)
Date: Sun, 14 Dec 2008 12:59:39 -0600
Subject: [pacman-dev] [PATCH] (newgpg) Let pacman specify GnuPG's home
	directory.
Message-ID: <20081214190116.VZHO27943.eastrmmtao104.cox.net@eastrmimpo02.cox.net>

GnuPG looks for configuration files and keyrings in its home directory.
For a user, that is typically ~/.gnupg.
This patch causes pacman to use /etc/pacman.d/gnupg/ as the default
GnuPG home.  One may override the default using --gpgdir on the command-line
or GPGDir in pacman's configuration file.

Signed-off-by: Chris Brannon <cmbrannon at cox.net>
---
 doc/pacman.8.txt       |    7 +++++++
 doc/pacman.conf.5.txt  |    6 ++++++
 src/pacman/Makefile.am |    2 ++
 src/pacman/conf.h      |    1 +
 src/pacman/pacman.c    |   25 +++++++++++++++++++++++++
 5 files changed, 41 insertions(+), 0 deletions(-)

diff --git a/doc/pacman.8.txt b/doc/pacman.8.txt
index 6f071ba..a780627 100644
--- a/doc/pacman.8.txt
+++ b/doc/pacman.8.txt
@@ -136,6 +136,13 @@ Options
 *\--config* <'file'>::
 	Specify an alternate configuration file.
 
+*\--gpgdir* <'dir'::
+	Specify a directory of files used by GnuPG to verify package 
+	signatures.  This directory should contain two files:
+	``pubring.gpg'' and ``trustdb.gpg''.  ``pubring.gpg'' holds the public
+	keys of all packagers.  ``trustdb.gpg'' contains a so-called
+	trust database, which specifies that the keys are authentic and trusted.
+
 *\--logfile* <'file'>::
 	Specify an alternate log file. This is an absolute path, regardless of
 	the installation root setting.
diff --git a/doc/pacman.conf.5.txt b/doc/pacman.conf.5.txt
index 8ef11ec..fa69bfa 100644
--- a/doc/pacman.conf.5.txt
+++ b/doc/pacman.conf.5.txt
@@ -69,6 +69,12 @@ Options
 	path, the root path is not automatically prepended.
 
 
+*GPGDir =* path/to/gpg/dir::
+	Overrides the default location of the directory containing
+	configuration files for GnuPG.
+	A typical default is ``/etc/pacman.d/gnupg''.
+	This is an absolute path, and the root directory is not prepended.
+
 *LogFile =* '/path/to/file'::
 	Overrides the default location of the pacman log file. A typical default
 	is ``/var/log/pacman.log''. This is an absolute path and the root directory
diff --git a/src/pacman/Makefile.am b/src/pacman/Makefile.am
index 220ee9c..4da6ef3 100644
--- a/src/pacman/Makefile.am
+++ b/src/pacman/Makefile.am
@@ -1,6 +1,7 @@
 # paths set at make time
 conffile  = ${sysconfdir}/pacman.conf
 dbpath    = ${localstatedir}/lib/pacman/
+gpgdir    = ${sysconfdir}/pacman.d/gnupg/
 cachedir  = ${localstatedir}/cache/pacman/pkg/
 logfile   = ${localstatedir}/log/pacman.log
 
@@ -10,6 +11,7 @@ DEFS = -DLOCALEDIR=\"@localedir@\" \
        -DCONFFILE=\"$(conffile)\" \
        -DROOTDIR=\"$(ROOTDIR)\" \
        -DDBPATH=\"$(dbpath)\" \
+       -DGPGDIR=\"$(gpgdir)\" \
        -DCACHEDIR=\"$(cachedir)\" \
        -DLOGFILE=\"$(logfile)\" \
        @DEFS@
diff --git a/src/pacman/conf.h b/src/pacman/conf.h
index 8ea6662..f491057 100644
--- a/src/pacman/conf.h
+++ b/src/pacman/conf.h
@@ -37,6 +37,7 @@ typedef struct __config_t {
 	char *rootdir;
 	char *dbpath;
 	char *logfile;
+	char *gpgdir;
 	/* TODO how to handle cachedirs? */
 
 	unsigned short op_q_isfile;
diff --git a/src/pacman/pacman.c b/src/pacman/pacman.c
index 3255cdf..18fd3a8 100644
--- a/src/pacman/pacman.c
+++ b/src/pacman/pacman.c
@@ -138,6 +138,7 @@ static void usage(int op, const char * const myname)
 			printf(_("  -q, --quiet          show less information for query and search\n"));
 		}
 		printf(_("      --config <path>  set an alternate configuration file\n"));
+		printf(_("      --gpgdir <path>  set an alternate home directory for GnuPG\n"));
 		printf(_("      --logfile <path> set an alternate log file\n"));
 		printf(_("      --noconfirm      do not ask for any confirmation\n"));
 		printf(_("      --noprogressbar  do not show a progress bar when downloading files\n"));
@@ -306,6 +307,20 @@ static void setlibpaths(void)
 			}
 		}
 
+		/*
+		* Set GnuPG's home directory.  This is not relative to
+		 * rootdir, even if rootdir is defined.
+		 * Reasoning: gpgdir contains configuration data.
+*/
+		if(config->gpgdir) {
+			ret = alpm_option_set_signaturedir(config->gpgdir);
+			if(ret != 0) {
+				pm_printf(PM_LOG_ERROR, _("problem setting gpgdir '%s' (%s)\n"),
+						config->gpgdir, alpm_strerrorlast());
+				cleanup(ret);
+			}
+		}
+
 		/* add a default cachedir if one wasn't specified */
 		if(alpm_option_get_cachedirs() == NULL) {
 			alpm_option_add_cachedir(CACHEDIR);
@@ -366,6 +381,7 @@ static int parseargs(int argc, char *argv[])
 		{"debug",      optional_argument, 0, 1003},
 		{"noprogressbar", no_argument,    0, 1004},
 		{"noscriptlet", no_argument,      0, 1005},
+		{"gpgdir",     required_argument,   0, 1006},
 		{"cachedir",   required_argument, 0, 1007},
 		{"asdeps",     no_argument,       0, 1008},
 		{"logfile",    required_argument, 0, 1009},
@@ -446,6 +462,9 @@ static int parseargs(int argc, char *argv[])
 			case 1012:
 				config->flags |= PM_TRANS_FLAG_ALLEXPLICIT;
 				break;
+			case 1006:
+				config->gpgdir = strdup(optarg);
+				break;
 			case 'Q': config->op = (config->op != PM_OP_MAIN ? 0 : PM_OP_QUERY); break;
 			case 'R': config->op = (config->op != PM_OP_MAIN ? 0 : PM_OP_REMOVE); break;
 			case 'S': config->op = (config->op != PM_OP_MAIN ? 0 : PM_OP_SYNC); break;
@@ -725,6 +744,11 @@ static int _parseconfig(const char *file, const char *givensection,
 							config->rootdir = strdup(ptr);
 							pm_printf(PM_LOG_DEBUG, "config: rootdir: %s\n", ptr);
 						}
+					} else if (strcmp(key, "GPGDir") == 0) {
+						if(!config->gpgdir) {
+							config->gpgdir = strdup(ptr);
+							pm_printf(PM_LOG_DEBUG, "config: gpgdir: %s\n", ptr);
+						}
 					} else if (strcmp(key, "LogFile") == 0) {
 						if(!config->logfile) {
 							config->logfile = strdup(ptr);
@@ -864,6 +888,7 @@ int main(int argc, char *argv[])
 	/* define paths to reasonable defaults */
 	alpm_option_set_root(ROOTDIR);
 	alpm_option_set_dbpath(DBPATH);
+	alpm_option_set_signaturedir(GPGDIR);
 	alpm_option_set_logfile(LOGFILE);
 
 	/* Priority of options:
-- 
1.6.0.5


From jatheendra at gmail.com  Mon Dec 15 03:37:43 2008
From: jatheendra at gmail.com (Jatheendra)
Date: Mon, 15 Dec 2008 14:07:43 +0530
Subject: [pacman-dev] GPG work
In-Reply-To: <449c10960812130618g1d9625d8y4aba0df77b37e9d5@mail.gmail.com>
References: <449c10960812071318v7a7fb427nb19f4f5726496234@mail.gmail.com>
	<20081208115553.3c2fd3d9@tux1.brauer.lan>
	<449c10960812080434w6ba33471k9fd55ae681b69cf3@mail.gmail.com>
	<20081213132308.5e8c5b4e@tux1.brauer.lan>
	<449c10960812130618g1d9625d8y4aba0df77b37e9d5@mail.gmail.com>
Message-ID: <e94122a10812150037r55835f43t9091640a67c5f281@mail.gmail.com>

There are a few people like myself who are interested in working on
pacman. I started looking at pacman code a few weeks back, had a look
at bug tracker and send a patch for a simple feature(pacman -Qo
which). After that i was stuck because i couldnt find anything to work
on. Most of the feature requests are huge changes and it doesnt seem
that there is consent on those even within current developers. So it
would be better if we can have a roadmap sort of thing for pacman (
not the TODO.dan and TODO.aaron in source) with features which
developers want to include but are unable because of lack of time.
Basically something which beginners can use to get their hand
wet.........

(If such a list already exists, i couldnt find it)

Thanks



On Sat, Dec 13, 2008 at 7:48 PM, Dan McGee <dpmcgee at gmail.com> wrote:
>
> Simple fact- there is no one that is both interested AND can program in C.
>
> If I attached a $500 bounty, we might get results. Until then, we just
> don't have enough people in the community actually interested in
> working on pacman- just those that ask for new features. This code has
> been posted on every single mailing list, every developer has seen it,
> etc. and yet we still have no one else working on it. I can't quit my
> day job.
>
> -Dan
> _______________________________________________
> pacman-dev mailing list
> pacman-dev at archlinux.org
> http://archlinux.org/mailman/listinfo/pacman-dev
>

From sivertb at stud.ntnu.no  Mon Dec 15 04:20:54 2008
From: sivertb at stud.ntnu.no (Sivert Berg)
Date: Mon, 15 Dec 2008 10:20:54 +0100
Subject: [pacman-dev] [PATCH 1/2] Added a new database backend
Message-ID: <1229332855-3569-1-git-send-email-sivertb@stud.ntnu.no>

This is an attempt to add a one-file-per-database backend. The main
reason I wrote this was because load times of the database on my laptop
when it's not yet cached is around 40-50 seconds. When you just want to
do a quick "pacman -Ss" that can be a little annoying. I know you have
discussed the current database vs. one-file earlier, and I think someone
said another backend could be supported too.

This first patch contains the main database code and a program to
convert .db.tar.gz and file databases into the new format.

Signed-off-by: Sivert Berg <sivertb at stud.ntnu.no>
---
 lib/libalpm/be_packed.h |  757 +++++++++++++++++++++++++++++++++++++++++++++++
 src/util/Makefile.am    |    4 +-
 src/util/convdb.c       |  236 +++++++++++++++
 3 files changed, 996 insertions(+), 1 deletions(-)
 create mode 100644 lib/libalpm/be_packed.h
 create mode 100644 src/util/convdb.c

diff --git a/lib/libalpm/be_packed.h b/lib/libalpm/be_packed.h
new file mode 100644
index 0000000..e118a3a
--- /dev/null
+++ b/lib/libalpm/be_packed.h
@@ -0,0 +1,757 @@
+/*
+ * Copyright (c) 2008  Sivert Berg
+ *
+ * Permission is hereby granted, free of charge, to any person
+ * obtaining a copy of this software and associated documentation
+ * files (the "Software"), to deal in the Software without
+ * restriction, including without limitation the rights to use,
+ * copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following
+ * conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
+ * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+ * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+ * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+
+/* _HOW IT WORKS_
+ *
+ * If you've ever looked at the ext2 filesystem you will find alot
+ * of similarities (among others the inode that I didn't have the
+ * imagination to find a new name for).
+ *
+ * _GROUP_
+ * Basicly the database is divided into groups. Each group got the
+ * same number of blocks and inodes.
+ *
+ * _INODE_
+ * Each file is assigned an unique inode. The inode got a list of
+ * blocks where the actual data for the file is stored. Worth noting
+ * is that the last entry in the block list is indirect. That means
+ * it points to a block that points to blocks.
+ */
+
+#include <stdint.h>
+#include <sys/mman.h>
+#include <stdlib.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <stdio.h>
+#include <string.h>
+#include <unistd.h>
+
+
+#ifndef BE_PACKED_H
+#define BE_PACKED_H
+
+#define BLOCK_SIZE 256
+#define GROUP_SIZE (BLOCK_SIZE * 4096)
+#define BLOCKS_PER_GROUP 3936
+#define INODES_PER_GROUP 1024
+
+#define MIN(a, b) (((a) < (b))?(a):(b))
+
+/* _pack_open_file flags */
+#define PO_READ 1
+#define PO_WRITE 2
+
+
+typedef struct __inode_t {
+	int32_t size;
+	int32_t block[7];
+} inode_t;
+
+typedef struct __group_desc_t {
+	uint32_t inodes_free;
+	uint32_t blocks_free;
+	uint32_t checksum;
+	inode_t i_table[INODES_PER_GROUP];
+	uint32_t bitmap[BLOCKS_PER_GROUP / 32];
+	char data[];
+} group_desc_t;
+
+typedef struct __package_t {
+	int32_t inode;
+	uint16_t rec_len;
+	uint16_t name_len;
+	char name[];
+} package_t;
+
+typedef struct __pack_db_t {
+	int fd;
+	int groups;
+  int read_only;
+	group_desc_t *map;
+} pack_db_t;
+
+typedef struct __pack_file_t {
+	int inode;
+	size_t pos;
+} pack_file_t;
+
+typedef struct __pack_dir_t {
+	size_t pos;
+} pack_dir_t;
+
+
+/**
+ ** GROUP FUNCTIONS
+ */
+/* Initilizes a new group */
+void _pack_init_group(group_desc_t *group)
+{
+	int i, j;
+
+	for (i = 0; i < INODES_PER_GROUP; i++) {
+		group->i_table[i].size = -1;
+		for (j = 0; j < 7; j++)
+			group->i_table[i].block[j] = -1;
+	}
+
+	for (i = 0; i < BLOCKS_PER_GROUP / 32; i++) {
+		group->bitmap[i] = 0;
+	}
+
+	group->inodes_free = INODES_PER_GROUP;
+	group->blocks_free = BLOCKS_PER_GROUP;
+	group->checksum = 0;
+}
+
+
+/* Adds a new group to the file */
+void _pack_new_group(pack_db_t *db)
+{
+	db->groups++;
+	ftruncate(db->fd, db->groups * GROUP_SIZE);
+	db->map = mremap(db->map, (db->groups - 1) * GROUP_SIZE,
+			db->groups * GROUP_SIZE, MREMAP_MAYMOVE);
+
+	_pack_init_group((group_desc_t*)((char*)db->map +
+				GROUP_SIZE * (db->groups - 1)));
+}
+
+
+/**
+ ** POINTER FUNCTIONS (return pointers to various structures)
+ */
+group_desc_t *_pack_get_group(pack_db_t *db, int group)
+{
+	return (group_desc_t*)((char*)db->map + group*GROUP_SIZE);
+}
+
+
+inode_t *_pack_get_inode(pack_db_t *db, int inode)
+{
+	int group = inode / INODES_PER_GROUP;
+	group_desc_t *groupd = _pack_get_group(db, group);
+	inode %= INODES_PER_GROUP;
+
+	return groupd->i_table + inode;
+}
+
+void *_pack_get_block(pack_db_t *db, int block)
+{
+	if (block == -1)
+		return NULL;
+
+	group_desc_t *group = _pack_get_group(db, block / BLOCKS_PER_GROUP);
+	return group->data + (block % BLOCKS_PER_GROUP) * BLOCK_SIZE;
+}
+
+
+/* Searches for a free block in the bitmap and marks the bit
+ * it finds as used.
+ */
+int _pack_check_bitmap(group_desc_t *group)
+{
+	int i = 0, j = 0, t = 0;
+
+	for (i = 0; i < BLOCKS_PER_GROUP / 32; i++) {
+		if (group->bitmap[i] != 0xffffffff) {
+			t = group->bitmap[i];
+			for (j = 0; j < 32 && (t & 1); j++, t >>= 1);
+			break;
+		}
+	}
+
+	group->bitmap[i] |= 1 << j;
+	return i * 32 + j;
+}
+
+
+/* Clears the n'th bit */
+int _pack_bitmap_free(group_desc_t *group, int n)
+{
+	int i = n / 32;
+	int j = n % 32;
+
+	group->bitmap[i] &= ~(i << j);
+	return 0;
+}
+
+
+/* Returns the number of a new unused block
+ * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP!
+ * that means ANY pointers into the database may be invalid
+ * after you've called this function.
+ */
+int _pack_get_free_block(pack_db_t *db)
+{
+	int i;
+	group_desc_t *group;
+	int ret;
+
+	for (i = 0; i < db->groups; i++) {
+		group = _pack_get_group(db, i);
+		if (group->blocks_free) {
+			group->blocks_free--;
+			ret = _pack_check_bitmap(group);
+			ret += i * BLOCKS_PER_GROUP;
+			return ret;;
+		}
+	}
+
+	_pack_new_group(db);
+	group = _pack_get_group(db, i);
+	group->blocks_free--;
+	ret = _pack_check_bitmap(group);
+	ret += i * BLOCKS_PER_GROUP;
+
+	return ret;
+}
+
+
+/* Frees the block given */
+int _pack_block_free(pack_db_t *db, int block)
+{
+	group_desc_t *group;
+	int g, b;
+
+	g = block / BLOCKS_PER_GROUP;
+	b = block & BLOCKS_PER_GROUP;
+
+	group = _pack_get_group(db, g);
+  group->blocks_free++;
+
+	return _pack_bitmap_free(group, b);
+}
+
+
+/* Returns the number of a new free inode.
+ * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP!
+ * that means ANY pointers into the database may be invalid
+ * after you've called this function.
+ */
+int _pack_get_free_inode(pack_db_t *db)
+{
+	int i = 0, j = 0;
+	group_desc_t *group;
+
+	for (i = 0; i < db->groups; i++) {
+		group = _pack_get_group(db, i);
+		if (group->inodes_free) {
+			group->inodes_free--;
+			for (j = 0; j < INODES_PER_GROUP; j++) {
+				if (group->i_table[j].size == -1) {
+					group->i_table[j].size = 0;
+					return i * INODES_PER_GROUP + j;
+				}
+			}
+		}
+	}
+
+	_pack_new_group(db);
+	group = _pack_get_group(db, i);
+	group->inodes_free--;
+
+	return i * INODES_PER_GROUP;
+}
+
+
+/* Frees an inode and all of the blocks it might be using */
+int _pack_inode_free(pack_db_t *db, int in)
+{
+	inode_t *inode = _pack_get_inode(db, in);
+  group_desc_t *group = _pack_get_group(db, in / INODES_PER_GROUP);
+	int i;
+	int block;
+	int32_t *blocks;
+	int size = BLOCK_SIZE / sizeof(uint32_t) - 1;
+
+	inode->size = -1;
+
+	for (i = 0; i < 6; i++) {
+		if (inode->block[i] != -1) {
+			_pack_block_free(db, inode->block[i]);
+			inode->block[i] = -1;
+		}
+	}
+
+	block = inode->block[6];
+	blocks = _pack_get_block(db, block);
+
+	while(blocks != NULL) {
+		for (i = 0; i < size; i++) {
+			if (blocks[i] != -1)
+				_pack_block_free(db, blocks[i]);
+		}
+
+		_pack_block_free(db, block);
+		block = blocks[size];
+		blocks = _pack_get_block(db, block);
+	}
+
+  group->inodes_free++;
+
+	return 0;
+}
+
+
+/* Returns the number of the block associated with the inode */
+int _pack_inode_get_block(pack_db_t *db, int i, int block)
+{
+	inode_t *inode = _pack_get_inode(db, i);
+	if (block < 6)
+		return inode->block[block];
+
+	block -= 6;
+	int size = BLOCK_SIZE / sizeof(uint32_t) - 1;
+	uint32_t *blocks = _pack_get_block(db, inode->block[6]);
+
+	for (;blocks != NULL && block >= size; block -= size)
+		blocks = _pack_get_block(db, blocks[size]);
+
+	if (blocks == NULL)
+		return -1;
+
+	return blocks[block];
+}
+
+
+/* Returns the number of the block block associated with the inode.
+ * If there is no such block, it gets allocated.
+ * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP!
+ * that means ANY pointers into the database may be invalid
+ * after you've called this function.
+ */
+int _pack_inode_add_block(pack_db_t *db, int in, int block)
+{
+	inode_t *inode = _pack_get_inode(db, in);
+	int tmp;
+
+	if (block < 6) {
+		if (inode->block[block] != -1)
+			return inode->block[block];
+
+		tmp = _pack_get_free_block(db);
+		inode = _pack_get_inode(db, in);
+		return inode->block[block] = tmp;
+	}
+
+	block -= 6;
+	int size = BLOCK_SIZE / sizeof(int32_t) - 1;
+	int i;
+	int old_block;
+	int32_t *blocks;
+
+	if (inode->block[6] == -1) {
+		tmp = _pack_get_free_block(db);
+		inode = _pack_get_inode(db, in);
+		inode->block[6] = tmp;
+		blocks = _pack_get_block(db, inode->block[6]);
+		for (i = 0; i < size + 1; i++)
+			blocks[i] = -1;
+	}
+	else
+		blocks = _pack_get_block(db, inode->block[6]);
+
+	old_block = inode->block[6];
+
+	for (;block >= size; block -= size) {
+		if (blocks[size] == -1) {
+			tmp = _pack_get_free_block(db);
+			blocks = _pack_get_block(db, old_block);
+			blocks[size] = tmp;
+			old_block = tmp;
+			blocks = _pack_get_block(db, tmp);
+			for (i = 0; i < size + 1; i++)
+				blocks[i] = -1;
+		}
+		else {
+			old_block = blocks[size];
+			blocks = _pack_get_block(db, blocks[size]);
+		}
+	}
+
+	if (blocks[block] == -1) {
+		tmp = _pack_get_free_block(db);
+		blocks = _pack_get_block(db, old_block);
+		blocks[block] = tmp;
+	}
+
+	return blocks[block];
+}
+
+/* Returns a pointer to the package with the name pkgname.
+ * Returns NULL if it's not found
+ */
+package_t *_pack_find_package(pack_db_t *db, const char *pkgname)
+{
+	inode_t *inode = _pack_get_inode(db, 0);
+	int s_len = strlen(pkgname);
+	package_t *pkg;
+	int block = 0;
+	int b_pos = 0;
+
+	pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, block));
+
+	while (block < (inode->size / BLOCK_SIZE)) {
+		if (pkg->inode != 0 &&
+				pkg->name_len == s_len &&
+				strncmp(pkgname, pkg->name, s_len) == 0)
+			break;
+
+		b_pos += pkg->rec_len;
+		pkg = (package_t*)((char*)pkg + pkg->rec_len);
+		if (b_pos >= BLOCK_SIZE) {
+			pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, ++block));
+			b_pos = 0;
+		}
+	}
+
+	return pkg;
+}
+
+/* Creates a new package with the name pkgname.
+ * Returns a pointer to the new package.
+ * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP!
+ * that means ANY pointers into the database may be invalid
+ * after you've called this function.
+ */
+package_t *_pack_add_package(pack_db_t *db, const char *pkgname)
+{
+	inode_t *inode = _pack_get_inode(db, 0);
+	int s_len = strlen(pkgname);
+	int len = s_len + sizeof(package_t);
+	package_t *pkg;
+	int block = 0;
+	int b_pos = 0;
+
+	pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, block));
+
+	while (pkg && block < (inode->size / BLOCK_SIZE)) {
+		if (pkg->rec_len >= len && pkg->inode == 0)
+			break;
+
+		b_pos += pkg->rec_len;
+		pkg = (package_t*)((char*)pkg + pkg->rec_len);
+		if (b_pos >= BLOCK_SIZE) {
+			pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, ++block));
+			b_pos = 0;
+		}
+	}
+
+	if (pkg == NULL) {
+		pkg = _pack_get_block(db, _pack_inode_add_block(db, 0, block));
+		pkg->rec_len = BLOCK_SIZE;
+		pkg->inode = 0;
+		inode = _pack_get_inode(db, 0);
+		inode->size += BLOCK_SIZE;
+	}
+
+	strncpy(pkg->name, pkgname, s_len);
+	if (pkg->rec_len > (len + sizeof(package_t) + 10)) {
+		package_t *foo = (package_t*)((char*)pkg + len);
+		foo->rec_len = pkg->rec_len - len;
+		foo->inode = 0;
+		pkg->rec_len = len;
+	}
+
+	pkg->inode = -1;
+	pkg->name_len = s_len;
+
+	return pkg;
+}
+
+
+/* Reads data from the file until buf is full or we hit a newline */
+int _pack_file_gets(pack_db_t *db, pack_file_t *file, char *buf, size_t size)
+{
+	inode_t *inode = _pack_get_inode(db, file->inode);
+	size_t i;
+	size_t b_pos = file->pos % BLOCK_SIZE;
+	const char *src = _pack_get_block(db,
+			_pack_inode_get_block(db, file->inode, file->pos / BLOCK_SIZE));
+	int newline = 0;
+
+	for (i = 0
+			;i < size && file->pos < inode->size && !newline
+			;i++, b_pos++, file->pos++) {
+		if (b_pos >= BLOCK_SIZE) {
+			src = _pack_get_block(db,
+					_pack_inode_get_block(db, file->inode, file->pos / BLOCK_SIZE));
+			b_pos = 0;
+		}
+
+		buf[i] = src[b_pos];
+
+		if (src[b_pos] == '\n')
+			newline = 1;
+	}
+
+	if (i < size - 1)
+		buf[i] = '\0';
+
+	return i;
+}
+
+
+/* Write the data in buf into the file
+ * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP!
+ * that means ANY pointers into the database may be invalid
+ * after you've called this function.
+ */
+int _pack_file_puts(pack_db_t *db, pack_file_t *file,
+		const char *buf, size_t size)
+{
+	size_t i;
+	size_t b_pos = file->pos % BLOCK_SIZE;
+	char *dest = _pack_get_block(db,
+			_pack_inode_add_block(db, file->inode, file->pos / BLOCK_SIZE));
+	int block;
+
+	for (i = 0; i < size; i++, file->pos++, b_pos++)
+	{
+		if (b_pos >= BLOCK_SIZE) {
+			block = _pack_inode_add_block(db, file->inode, file->pos / BLOCK_SIZE);
+			dest = _pack_get_block(db, block);
+			b_pos = 0;
+		}
+		dest[b_pos] = buf[i];
+	}
+
+	inode_t *inode = _pack_get_inode(db, file->inode);
+	inode->size = file->pos;
+
+	return i;
+}
+
+
+int _pack_file_eof(pack_db_t *db, pack_file_t *file)
+{
+	inode_t *inode = _pack_get_inode(db, file->inode);
+	return file->pos >= inode->size;
+}
+
+
+/* Opens a file in the database
+ * flags: if you pass PO_READ, it will return -1
+ * if the file does not exist, if you pass PO_WRITE
+ * the file will be created if it does not exist
+ * CAUTION: THIS FUNCTION MAY RESULT IN A MMREMAP!
+ * that means ANY pointers into the database may be invalid
+ * after you've called this function.
+ */
+pack_file_t *_pack_open_file(pack_db_t *db, const char *pkgname,
+		const char *filename, int flags)
+{
+	char name[512];
+
+	snprintf(name, 512, "%s/%s", pkgname, filename);
+
+	package_t *pkg = _pack_find_package(db, name);
+	int w = flags & PO_WRITE;
+	int tmp;
+
+	if (w && db->read_only)
+		return NULL;
+
+	if (pkg == NULL && w)
+		pkg = _pack_add_package(db, name);
+	else if (pkg == NULL)
+		return NULL;
+
+	pack_file_t *ret = malloc(sizeof(pack_file_t));
+	ret->pos = 0;
+
+	if (pkg->inode == -1 && w) {
+		tmp = _pack_get_free_inode(db);
+		pkg = _pack_find_package(db, name);
+		ret->inode = pkg->inode = tmp;
+	}
+	else
+		ret->inode = pkg->inode;
+
+	return ret;
+}
+
+
+/* Closes the file given */
+void _pack_close_file(pack_file_t *file)
+{
+	free(file);
+}
+
+
+/* Removes a package along with all it's files */
+int _pack_package_remove(pack_db_t *db, const char *pkgname)
+{
+	inode_t *inode = _pack_get_inode(db, 0);
+	int s_len = strlen(pkgname);
+	package_t *prev, *pkg;
+	int block = 0;
+	int b_pos = 0;
+
+	if (db->read_only)
+		return -1;
+
+	prev = NULL;
+	pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, block));
+
+	while (block < (inode->size / BLOCK_SIZE)) {
+		if (pkg->inode != 0 &&
+				pkg->name_len > s_len &&
+				strncmp(pkgname, pkg->name, s_len) == 0 &&
+				pkg->name[s_len] == '/') {
+			_pack_inode_free(db, pkg->inode);
+
+			if (prev != NULL && prev->inode == 0) {
+				prev->rec_len += pkg->rec_len;
+				pkg = prev;
+			}
+			else
+				pkg->inode = 0;
+		}
+
+		prev = pkg;
+		b_pos += pkg->rec_len;
+		pkg = (package_t*)((char*)pkg + pkg->rec_len);
+		if (b_pos >= BLOCK_SIZE) {
+			pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, ++block));
+			b_pos = 0;
+			prev = NULL;
+		}
+	}
+
+	return 0;
+}
+
+
+/* Gets the next entry in the directory */
+int _pack_dir_next(pack_db_t *db, pack_dir_t *dir, char *name, size_t len)
+{
+	inode_t *inode = _pack_get_inode(db, 0);
+
+	if (dir->pos >= inode->size)
+		return -1;
+
+	int block, b_pos;
+	package_t *pkg;
+
+	do {
+		block = dir->pos / BLOCK_SIZE;
+		b_pos = dir->pos % BLOCK_SIZE;
+		pkg = _pack_get_block(db, _pack_inode_get_block(db, 0, block));
+		if (!pkg)
+			return -1;
+		pkg = (package_t*)((char*)pkg + b_pos);
+		dir->pos += pkg->rec_len;
+	} while (pkg->inode == 0);
+
+	strncpy(name, pkg->name, MIN(len, pkg->name_len));
+	if (len > pkg->name_len)
+		name[pkg->name_len] = '\0';
+
+	return 0;
+}
+
+
+/* Opens the directory for reading */
+pack_dir_t *_pack_open_dir(pack_db_t *db)
+{
+	pack_dir_t *dir = malloc(sizeof(pack_dir_t));
+	dir->pos = 0;
+
+	return dir;
+}
+
+
+/* Closes the given directory */
+void _pack_close_dir(pack_dir_t *dir)
+{
+	free(dir);
+}
+
+
+/* Opens a packed database */
+pack_db_t *_pack_db_open(const char *filename)
+{
+	struct stat st;
+	int is_new = 0;
+
+	pack_db_t *db = malloc(sizeof(pack_db_t));
+	db->fd = open(filename, O_RDWR, 0);
+  db->read_only = 0;
+
+  if (db->fd == -1) {
+		db->fd = open(filename, O_RDONLY, 0);
+		if (db->fd == -1) {
+			db->fd = open(filename, O_RDWR | O_CREAT, 0644);
+
+			if (db->fd == -1) {
+				free(db);
+				printf("Could not open %s\n", filename);
+				return NULL;
+			}
+
+			ftruncate(db->fd, GROUP_SIZE);
+			is_new = 1;
+		}
+	else
+		db->read_only = 1;
+	}
+
+	fstat(db->fd, &st);
+	db->groups = st.st_size / GROUP_SIZE;
+
+  int flags = PROT_READ;
+  if (!db->read_only)
+    flags |= PROT_WRITE;
+
+	db->map = mmap(NULL, GROUP_SIZE * db->groups, flags, MAP_SHARED,
+			db->fd, 0);
+
+	if (db->map == MAP_FAILED) {
+		perror("mmap");
+		free(db);
+		exit(0);
+	}
+
+	if (is_new) {
+		_pack_init_group(db->map);
+		db->map->i_table[0].size = 0;
+	}
+
+	return db;
+}
+
+
+/* Closes a packed database */
+void _pack_db_close(pack_db_t *db)
+{
+	munmap(db->map, db->groups * GROUP_SIZE);
+	close(db->fd);
+	free(db);
+}
+
+#endif /* BE_PACKED_H */
diff --git a/src/util/Makefile.am b/src/util/Makefile.am
index 97a0ffa..64e18ad 100644
--- a/src/util/Makefile.am
+++ b/src/util/Makefile.am
@@ -3,7 +3,7 @@ conffile  = ${sysconfdir}/pacman.conf
 dbpath    = ${localstatedir}/lib/pacman/
 cachedir  = ${localstatedir}/cache/pacman/pkg/
 
-bin_PROGRAMS = vercmp testpkg testdb
+bin_PROGRAMS = vercmp testpkg testdb convdb
 
 DEFS = -DLOCALEDIR=\"@localedir@\" \
        -DCONFFILE=\"$(conffile)\" \
@@ -18,6 +18,8 @@ AM_CFLAGS = -pedantic -D_GNU_SOURCE
 vercmp_SOURCES = vercmp.c
 vercmp_LDADD = $(top_builddir)/lib/libalpm/.libs/libalpm.la
 
+convdb_SOURCES = convdb.c
+
 testpkg_SOURCES = testpkg.c
 testpkg_LDADD = $(top_builddir)/lib/libalpm/.libs/libalpm.la
 
diff --git a/src/util/convdb.c b/src/util/convdb.c
new file mode 100644
index 0000000..f1f39bd
--- /dev/null
+++ b/src/util/convdb.c
@@ -0,0 +1,236 @@
+/*
+ * Copyright (c) 2008  Sivert Berg
+ *
+ * Permission is hereby granted, free of charge, to any person
+ * obtaining a copy of this software and associated documentation
+ * files (the "Software"), to deal in the Software without
+ * restriction, including without limitation the rights to use,
+ * copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following
+ * conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
+ * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+ * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+ * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+
+
+/* WARNING: This file is pretty much just a quick hack thrown together
+ * in a hurry. It seems to work, but don't take my word for it.
+ */
+
+#include <be_packed.h>
+#include <dirent.h>
+#include <archive.h>
+#include <archive_entry.h>
+
+
+/* Convert an archieved database into a packed database
+ * Expects a gziped database ending in .db.tar.gz
+ */
+void convert_archive(const char *filename)
+{
+	struct archive_entry *aent = NULL;
+	struct archive *arch = archive_read_new();
+	archive_read_support_compression_gzip(arch);
+	archive_read_support_format_tar(arch);
+	if (archive_read_open_filename(arch, filename, 1) != ARCHIVE_OK) {
+		printf("Could not open archive %s\n", archive_error_string(arch));
+		return;
+	}
+
+	char *db_name = strndup(filename, strlen(filename) - strlen(".tar.gz"));
+	pack_db_t *db = _pack_db_open(db_name);
+	free(db_name);
+	char buffer[4096];
+	size_t size;
+	pack_file_t *pf;
+
+	while (!archive_read_next_header(arch, &aent)) {
+		const char *type = NULL;
+		if (strstr(archive_entry_pathname(aent), "/desc"))
+			type = "desc";
+		else if (strstr(archive_entry_pathname(aent), "/depends"))
+			type = "depends";
+		else if (strstr(archive_entry_pathname(aent), "/install"))
+			type = "install";
+		else if (strstr(archive_entry_pathname(aent), "/files"))
+			type = "files";
+
+		if (type == NULL)
+			continue;
+
+		char *name = strdup(archive_entry_pathname(aent));
+		int i;
+		for (i = 0; name[i] != '/' && name[i]; i++);
+		name[i] = '\0';
+
+		pf = _pack_open_file(db, name, type, PO_WRITE);
+
+		free(name);
+
+		do {
+			size = archive_read_data(arch, buffer, 4096);
+			_pack_file_puts(db, pf, buffer, size);
+		} while (size == 4096);
+
+		_pack_close_file(pf);
+
+	}
+
+	_pack_db_close(db);
+	archive_read_close(arch);
+	archive_read_finish(arch);
+}
+
+
+
+const char *dir_name(const char *full_path)
+{
+	const char *slash = full_path;
+	while (*full_path)
+		if (*full_path++ == '/')
+			slash = full_path;
+	return slash;
+}
+
+
+/* Convert a directory database into a packed database */
+void convert_directory(const char *pathname)
+{
+	char db_name[1024];
+	pack_db_t *db;
+	DIR  *dir = opendir(pathname);
+	struct dirent *dire;
+	char path[1024];
+	char buffer[8196];
+	size_t size;
+	FILE *f;
+	pack_file_t *pf;
+
+	strcpy(db_name, dir_name(pathname));
+	strcat(db_name, ".db");
+	db = _pack_db_open(db_name);
+
+	if (dir == NULL) {
+		perror("opendir");
+		return;
+	}
+
+	dire = readdir(dir);
+	while (dire != NULL) {
+		if (strcmp(dire->d_name, ".") && strcmp(dire->d_name, "..")) {
+
+
+			size_t foo = 0;
+			sprintf(path, "%s/%s/desc", pathname, dire->d_name);
+			f = fopen(path, "r");
+			if (f == NULL)
+				goto depends;
+			pf = _pack_open_file(db, dire->d_name, "desc", PO_WRITE);
+			do {
+				size = fread(buffer, 1, 8196, f);
+				_pack_file_puts(db, pf, buffer, size);
+				foo += size;
+			} while (size == 8196);
+			_pack_close_file(pf);
+			fclose(f);
+depends:
+			sprintf(path, "%s/%s/depends", pathname, dire->d_name);
+			f = fopen(path, "r");
+			if (f == NULL)
+				goto files;
+			pf = _pack_open_file(db, dire->d_name, "depends", PO_WRITE);
+			do {
+				size = fread(buffer, 1, 8196, f);
+				_pack_file_puts(db, pf, buffer, size);
+				foo += size;
+			} while (size == 8196);
+			_pack_close_file(pf);
+			fclose(f);
+files:
+			sprintf(path, "%s/%s/files", pathname, dire->d_name);
+			f = fopen(path, "r");
+			if (f == NULL)
+				goto install;
+			pf = _pack_open_file(db, dire->d_name, "files", PO_WRITE);
+			do {
+				size = fread(buffer, 1, 8196, f);
+				_pack_file_puts(db, pf, buffer, size);
+				foo += size;
+			} while (size == 8196);
+			_pack_close_file(pf);
+			fclose(f);
+install:
+			sprintf(path, "%s/%s/install", pathname, dire->d_name);
+			f = fopen(path, "r");
+			if (f == NULL)
+				goto changelog;
+			pf = _pack_open_file(db, dire->d_name, "install", PO_WRITE);
+			do {
+				size = fread(buffer, 1, 8196, f);
+				_pack_file_puts(db, pf, buffer, size);
+				foo += size;
+			} while (size == 8196);
+			_pack_close_file(pf);
+			fclose(f);
+changelog:
+			sprintf(path, "%s/%s/changelog", pathname, dire->d_name);
+			f = fopen(path, "r");
+			if (f == NULL)
+				goto deltas;
+			pf = _pack_open_file(db, dire->d_name, "changelog", PO_WRITE);
+			do {
+				size = fread(buffer, 1, 8196, f);
+				_pack_file_puts(db, pf, buffer, size);
+				foo += size;
+			} while (size == 8196);
+			_pack_close_file(pf);
+			fclose(f);
+deltas:
+			sprintf(path, "%s/%s/deltas", pathname, dire->d_name);
+			f = fopen(path, "r");
+			if (f == NULL)
+				goto done;
+			pf = _pack_open_file(db, dire->d_name, "deltas", PO_WRITE);
+			do {
+				size = fread(buffer, 1, 8196, f);
+				_pack_file_puts(db, pf, buffer, size);
+				foo += size;
+			} while (size == 8196);
+			_pack_close_file(pf);
+			fclose(f);
+done:
+			pf = NULL;
+		}
+
+		dire = readdir(dir);
+	}
+
+	closedir(dir);
+	_pack_db_close(db);
+}
+
+int main(int argc, char *argv[])
+{
+	if (argc < 2) {
+		printf("Usage: %s <dir/file>\n", argv[0]);
+		exit(0);
+	}
+
+	if (strstr(argv[1], ".tar.gz") != NULL)
+		convert_archive(argv[1]);
+	else
+		convert_directory(argv[1]);
+
+	return 0;
+}
-- 
1.6.0.5


From sivertb at stud.ntnu.no  Mon Dec 15 04:20:55 2008
From: sivertb at stud.ntnu.no (Sivert Berg)
Date: Mon, 15 Dec 2008 10:20:55 +0100
Subject: [pacman-dev] [PATCH 2/2] Added a new database backend
In-Reply-To: <1229332855-3569-1-git-send-email-sivertb@stud.ntnu.no>
References: <1229332855-3569-1-git-send-email-sivertb@stud.ntnu.no>
Message-ID: <1229332855-3569-2-git-send-email-sivertb@stud.ntnu.no>

This patch contains the changes and additions to libalpm. As you can see
minimal changes to the library was needed. The only change was making
the reading of changelog files happen through the database backend. I
also added an option to the configure script to choose which backend to
use (--with-backend).

Signed-off-by: Sivert Berg <sivertb at stud.ntnu.no>
---
 configure.ac            |    7 +
 lib/libalpm/Makefile.am |    2 +-
 lib/libalpm/be_files.c  |   21 ++
 lib/libalpm/be_packed.c |  845 +++++++++++++++++++++++++++++++++++++++++++++++
 lib/libalpm/db.h        |    3 +
 lib/libalpm/package.c   |   12 +-
 6 files changed, 880 insertions(+), 10 deletions(-)
 create mode 100644 lib/libalpm/be_packed.c

diff --git a/configure.ac b/configure.ac
index c7841b8..6e28ac9 100644
--- a/configure.ac
+++ b/configure.ac
@@ -73,6 +73,12 @@ AC_ARG_WITH(root-dir,
 	AS_HELP_STRING([--with-root-dir=path], [set the location of pacman's root operating directory]),
 	[ROOTDIR=$withval], [ROOTDIR=/])
 
+# Help line for database backend
+AC_ARG_WITH(backend,
+	AS_HELP_STRING([--with-backend=backend], [choose which backend to use. You can choose between 'files' and 'packed', it defaults to files]),
+	[BACKEND=$withval], [BACKEND=files])
+AC_SUBST(BACKEND)
+
 # Help line for package extension
 AC_ARG_WITH(pkg-ext,
 	AS_HELP_STRING([--with-pkg-ext=ext], [set the file extension used by packages]),
@@ -362,6 +368,7 @@ ${PACKAGE_NAME}:
     package extension      : ${PKGEXT}
     source pkg extension   : ${SRCEXT}
     database extension     : ${DBEXT}
+    database backend       : ${BACKEND}
 
   Compilation options:
     Run make in doc/ dir   : ${wantdoc}
diff --git a/lib/libalpm/Makefile.am b/lib/libalpm/Makefile.am
index 871855e..79b9d26 100644
--- a/lib/libalpm/Makefile.am
+++ b/lib/libalpm/Makefile.am
@@ -25,7 +25,7 @@ libalpm_la_SOURCES = \
 	alpm.h alpm.c \
 	alpm_list.h alpm_list.c \
 	backup.h backup.c \
-	be_files.c \
+	be_ at BACKEND@.c \
 	be_package.c \
 	cache.h cache.c \
 	conflict.h conflict.c \
diff --git a/lib/libalpm/be_files.c b/lib/libalpm/be_files.c
index b9ff646..81d7a17 100644
--- a/lib/libalpm/be_files.c
+++ b/lib/libalpm/be_files.c
@@ -874,4 +874,25 @@ int _alpm_db_remove(pmdb_t *db, pmpkg_t *info)
 	return(ret);
 }
 
+void *_alpm_db_changelog_open(pmdb_t *db, pmpkg_t *pkg)
+{
+	char clfile[PATH_MAX];
+	snprintf(clfile, PATH_MAX, "%s/%s/%s-%s/changelog",
+			alpm_option_get_dbpath(),
+			alpm_db_get_name(handle->db_local),
+			alpm_pkg_get_name(pkg),
+			alpm_pkg_get_version(pkg));
+	return fopen(clfile, "r");
+}
+
+int _alpm_db_changelog_close(pmdb_t *db, void *stream)
+{
+	return fclose(stream);
+}
+
+size_t _alpm_db_changelog_read(pmdb_t *db, const void *pf, void *ptr, size_t size)
+{
+	return fread(ptr, 1, size, pf);
+}
+
 /* vim: set ts=2 sw=2 noet: */
diff --git a/lib/libalpm/be_packed.c b/lib/libalpm/be_packed.c
new file mode 100644
index 0000000..3e4ea68
--- /dev/null
+++ b/lib/libalpm/be_packed.c
@@ -0,0 +1,845 @@
+/*
+ *  be_packed.c
+ *
+ *  Copyright (c) 2006 by Christian Hamar <krics at linuxforum.hu>
+ *  Copyright (c) 2006 by Miklos Vajna <vmiklos at frugalware.org>
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation; either version 2 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "config.h"
+
+#include <unistd.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <errno.h>
+#include <string.h>
+#include <stdint.h> /* uintmax_t, intmax_t */
+#include <sys/stat.h>
+#include <dirent.h>
+#include <ctype.h>
+#include <time.h>
+#include <utime.h>
+#include <limits.h> /* PATH_MAX */
+#include <locale.h> /* setlocale */
+
+/* libalpm */
+#include "db.h"
+#include "alpm_list.h"
+#include "cache.h"
+#include "log.h"
+#include "util.h"
+#include "alpm.h"
+#include "handle.h"
+#include "package.h"
+#include "delta.h"
+#include "deps.h"
+#include "dload.h"
+
+#include "be_packed.h"
+
+/** Update a package database
+ * @param force if true, then forces the update, otherwise update only in case
+ * the database isn't up to date
+ * @param db pointer to the package database to update
+ * @return 0 on success, > 0 on error (pm_errno is set accordingly), < 0 if up
+ * to date
+ */
+int SYMEXPORT alpm_db_update(int force, pmdb_t *db)
+{
+	char *dbfile, *dbfilepath;
+	time_t newmtime = 0;
+	char dbpath[PATH_MAX];
+	char dbname[PATH_MAX];
+	char buffer[1024];
+	size_t len;
+	struct stat st;
+	struct utimbuf utb;
+
+	int ret;
+
+	ALPM_LOG_FUNC;
+
+	/* Sanity checks */
+	ASSERT(handle != NULL, RET_ERR(PM_ERR_HANDLE_NULL, -1));
+	ASSERT(db != NULL && db != handle->db_local, RET_ERR(PM_ERR_WRONG_ARGS, -1));
+	/* Verify we are in a transaction.  This is done _mainly_ because we need a DB
+	 * lock - if we update without a db lock, we may kludge some other pacman
+	 * process that _has_ a lock.
+	 */
+	ASSERT(handle->trans != NULL, RET_ERR(PM_ERR_TRANS_NULL, -1));
+	ASSERT(handle->trans->state == STATE_INITIALIZED, RET_ERR(PM_ERR_TRANS_NOT_INITIALIZED, -1));
+	ASSERT(handle->trans->type == PM_TRANS_TYPE_SYNC, RET_ERR(PM_ERR_TRANS_TYPE, -1));
+
+	if(!alpm_list_find_ptr(handle->dbs_sync, db)) {
+		RET_ERR(PM_ERR_DB_NOT_FOUND, -1);
+	}
+
+	strncpy(dbname, db->path, PATH_MAX - 3);
+	/* Strip of the trailing / */
+	dbname[strlen(dbname) - 1] = '\0';
+	strcat(dbname, ".db");
+	stat(dbname, &st);
+
+	if(!force) {
+		/* get the lastupdate time */
+		if(st.st_mtime == 0) {
+			_alpm_log(PM_LOG_DEBUG, "failed to get last write time for %s\n",
+					db->treename);
+		}
+	}
+	else
+		st.st_mtime = 0;
+
+	len = strlen(db->treename) + strlen(DBEXT) + 1;
+	MALLOC(dbfile, len, RET_ERR(PM_ERR_MEMORY, -1));
+	sprintf(dbfile, "%s" DBEXT, db->treename);
+
+	strncpy(dbpath, alpm_option_get_dbpath(), PATH_MAX);
+	strcpy(dbpath + strlen(dbpath), "sync/");
+
+	ret = _alpm_download_single_file(dbfile, db->servers, dbpath,
+			st.st_mtime, &newmtime);
+	free(dbfile);
+
+	if(ret == 1) {
+		/* mtimes match, do nothing */
+		pm_errno = 0;
+		return(1);
+	} else if(ret == -1) {
+		/* pm_errno was set by the download code */
+		_alpm_log(PM_LOG_DEBUG, "failed to sync db: %s\n", alpm_strerrorlast());
+		return(-1);
+	} else {
+		/* remove the old db */
+		_alpm_log(PM_LOG_DEBUG, "removing database %s\n", db->path);
+		unlink(dbname);
+
+		/* cache needs to be rebuilt */
+		_alpm_db_free_pkgcache(db);
+
+		/* form the path to the db location */
+		len = strlen(dbpath) + strlen(db->treename) + strlen(DBEXT) + 1;
+		MALLOC(dbfilepath, len, RET_ERR(PM_ERR_MEMORY, -1));
+		sprintf(dbfilepath, "%s%s" DBEXT, dbpath, db->treename);
+
+		/* convert the database file */
+		snprintf(buffer, 1024, "convdb %s", dbfilepath);
+		system(buffer);
+
+		/* set the correct time on the new database */
+		utb.actime = newmtime;
+		utb.modtime = newmtime;
+		utime(dbname, &utb);
+
+		/* Reopen the database */
+		_pack_db_close(db->handle);
+		db->handle = _pack_db_open(dbname);
+
+		unlink(dbfilepath);
+		free(dbfilepath);
+	}
+
+	return(0);
+}
+
+int _alpm_db_open(pmdb_t *db)
+{
+	ALPM_LOG_FUNC;
+	char path[PATH_MAX];
+
+	if(db == NULL) {
+		RET_ERR(PM_ERR_DB_NULL, -1);
+	}
+
+	strncpy(path, db->path, PATH_MAX - 3);
+	/* Strip of the trailing / */
+	path[strlen(path) - 1] = '\0';
+	strcat(path, ".db");
+
+	_alpm_log(PM_LOG_DEBUG, "opening database from path '%s'\n", db->path);
+	db->handle = _pack_db_open(path);
+
+	if(db->handle == NULL) {
+		RET_ERR(PM_ERR_DB_OPEN, -1);
+	}
+
+	return(0);
+}
+
+void _alpm_db_close(pmdb_t *db)
+{
+	ALPM_LOG_FUNC;
+
+	if(db == NULL) {
+		return;
+	}
+
+	if(db->handle) {
+		_pack_db_close(db->handle);
+		db->handle = NULL;
+	}
+}
+
+static int splitname(const char *target, pmpkg_t *pkg)
+{
+	/* the format of a db entry is as follows:
+	 *    package-version-rel/
+	 * package name can contain hyphens, so parse from the back- go back
+	 * two hyphens and we have split the version from the name.
+	 */
+	char *tmp, *p, *q;
+
+	if(target == NULL || pkg == NULL) {
+		return(-1);
+	}
+	STRDUP(tmp, target, RET_ERR(PM_ERR_MEMORY, -1));
+	p = tmp + strlen(tmp);
+
+	/* do the magic parsing- find the beginning of the version string
+	 * by doing two iterations of same loop to lop off two hyphens */
+	for(q = --p; *q && *q != '-'; q--);
+	for(p = --q; *p && *p != '-'; p--);
+	if(*p != '-' || p == tmp) {
+		return(-1);
+	}
+
+	/* copy into fields and return */
+	if(pkg->version) {
+		FREE(pkg->version);
+	}
+	STRDUP(pkg->version, p+1, RET_ERR(PM_ERR_MEMORY, -1));
+	/* insert a terminator at the end of the name (on hyphen)- then copy it */
+	*p = '\0';
+	if(pkg->name) {
+		FREE(pkg->name);
+	}
+	STRDUP(pkg->name, tmp, RET_ERR(PM_ERR_MEMORY, -1));
+
+	free(tmp);
+	return(0);
+}
+
+int _alpm_db_populate(pmdb_t *db)
+{
+	int count = 0;
+	int i;
+	char name[PATH_MAX];
+	pack_dir_t *dir;
+
+	ALPM_LOG_FUNC;
+
+	ASSERT(db != NULL, RET_ERR(PM_ERR_DB_NULL, -1));
+
+	dir = _pack_open_dir(db->handle);
+
+	while(_pack_dir_next(db->handle, dir, name, PATH_MAX) != -1) {
+		pmpkg_t *pkg;
+
+		pkg = _alpm_pkg_new();
+		if(pkg == NULL) {
+			return(-1);
+		}
+
+		/* Deal only with desc entries, that way we're sure we do
+		 * not add the same package twice
+		 */
+		if(strstr(name, "/desc") == NULL)
+			continue;
+
+		for (i = 0; name[i] && name[i] != '/'; i++);
+		name[i] = '\0';
+
+		/* split the db entry name */
+		if(splitname(name, pkg) != 0) {
+			_alpm_log(PM_LOG_ERROR, _("invalid name for database entry '%s'\n"),
+					name);
+			_alpm_pkg_free(pkg);
+			continue;
+		}
+
+		/* explicitly read with only 'BASE' data, accessors will handle the rest */
+		if(_alpm_db_read(db, pkg, INFRQ_BASE) == -1) {
+			_alpm_log(PM_LOG_ERROR, _("corrupted database entry '%s'\n"), name);
+			_alpm_pkg_free(pkg);
+			continue;
+		}
+		pkg->origin = PKG_FROM_CACHE;
+		pkg->origin_data.db = db;
+		/* add to the collection */
+		_alpm_log(PM_LOG_FUNCTION, "adding '%s' to package cache for db '%s'\n",
+				pkg->name, db->treename);
+		db->pkgcache = alpm_list_add(db->pkgcache, pkg);
+		count++;
+	}
+
+	db->pkgcache = alpm_list_msort(db->pkgcache, count, _alpm_pkg_cmp);
+	return(count);
+}
+
+int _alpm_db_read(pmdb_t *db, pmpkg_t *info, pmdbinfrq_t inforeq)
+{
+	char line[513];
+	char pkgname[PATH_MAX];
+	pack_file_t *pf;
+	ALPM_LOG_FUNC;
+
+	if(db == NULL) {
+		RET_ERR(PM_ERR_DB_NULL, -1);
+	}
+
+	if(info == NULL || info->name == NULL || info->version == NULL) {
+		_alpm_log(PM_LOG_DEBUG, "invalid package entry provided to _alpm_db_read, skipping\n");
+		return(-1);
+	}
+
+	if(info->origin == PKG_FROM_FILE) {
+		_alpm_log(PM_LOG_DEBUG, "request to read database info for a file-based package '%s', skipping...\n", info->name);
+		return(-1);
+	}
+
+	/* bitmask logic here:
+	 * infolevel: 00001111
+	 * inforeq:   00010100
+	 * & result:  00000100
+	 * == to inforeq? nope, we need to load more info. */
+	if((info->infolevel & inforeq) == inforeq) {
+		/* already loaded this info, do nothing */
+		return(0);
+	}
+	_alpm_log(PM_LOG_FUNCTION, "loading package data for %s : level=0x%x\n",
+			info->name, inforeq);
+
+	/* clear out 'line', to be certain - and to make valgrind happy */
+	memset(line, 0, 513);
+	snprintf(pkgname, PATH_MAX, "%s-%s", info->name, info->version);
+
+	/* DESC */
+	if(inforeq & INFRQ_DESC) {
+		if((pf = _pack_open_file(db->handle, pkgname, "desc", PO_READ)) == NULL) {
+			_alpm_log(PM_LOG_ERROR, _("could not open package %s\n"), pkgname);
+			goto error;
+		}
+		while(!_pack_file_eof(db->handle, pf)) {
+			if(_pack_file_gets(db->handle, pf, line, 256) == 0) {
+				break;
+			}
+			_alpm_strtrim(line);
+			if(strcmp(line, "%NAME%") == 0) {
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				if(strcmp(_alpm_strtrim(line), info->name) != 0) {
+					_alpm_log(PM_LOG_ERROR, _("%s database is inconsistent: name "
+								"mismatch on package %s %s\n"), db->treename, info->name, line);
+				}
+			} else if(strcmp(line, "%VERSION%") == 0) {
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				if(strcmp(_alpm_strtrim(line), info->version) != 0) {
+					_alpm_log(PM_LOG_ERROR, _("%s database is inconsistent: version "
+								"mismatch on package %s\n"), db->treename, info->name);
+				}
+			} else if(strcmp(line, "%FILENAME%") == 0) {
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				STRDUP(info->filename, _alpm_strtrim(line), goto error);
+			} else if(strcmp(line, "%DESC%") == 0) {
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				STRDUP(info->desc, _alpm_strtrim(line), goto error);
+			} else if(strcmp(line, "%GROUPS%") == 0) {
+				while(_pack_file_gets(db->handle, pf, line, 512) &&
+						strlen(_alpm_strtrim(line))) {
+					char *linedup;
+					STRDUP(linedup, _alpm_strtrim(line), goto error);
+					info->groups = alpm_list_add(info->groups, linedup);
+				}
+			} else if(strcmp(line, "%URL%") == 0) {
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				STRDUP(info->url, _alpm_strtrim(line), goto error);
+			} else if(strcmp(line, "%LICENSE%") == 0) {
+				while(_pack_file_gets(db->handle, pf, line, 512) &&
+						strlen(_alpm_strtrim(line))) {
+					char *linedup;
+					STRDUP(linedup, _alpm_strtrim(line), goto error);
+					info->licenses = alpm_list_add(info->licenses, linedup);
+				}
+			} else if(strcmp(line, "%ARCH%") == 0) {
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				STRDUP(info->arch, _alpm_strtrim(line), goto error);
+			} else if(strcmp(line, "%BUILDDATE%") == 0) {
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				_alpm_strtrim(line);
+
+				char first = tolower(line[0]);
+				if(first > 'a' && first < 'z') {
+					struct tm tmp_tm = {0}; //initialize to null incase of failure
+					setlocale(LC_TIME, "C");
+					strptime(line, "%a %b %e %H:%M:%S %Y", &tmp_tm);
+					info->builddate = mktime(&tmp_tm);
+					setlocale(LC_TIME, "");
+				} else {
+					info->builddate = atol(line);
+				}
+			} else if(strcmp(line, "%INSTALLDATE%") == 0) {
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				_alpm_strtrim(line);
+
+				char first = tolower(line[0]);
+				if(first > 'a' && first < 'z') {
+					struct tm tmp_tm = {0}; //initialize to null incase of failure
+					setlocale(LC_TIME, "C");
+					strptime(line, "%a %b %e %H:%M:%S %Y", &tmp_tm);
+					info->installdate = mktime(&tmp_tm);
+					setlocale(LC_TIME, "");
+				} else {
+					info->installdate = atol(line);
+				}
+			} else if(strcmp(line, "%PACKAGER%") == 0) {
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				STRDUP(info->packager, _alpm_strtrim(line), goto error);
+			} else if(strcmp(line, "%REASON%") == 0) {
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				info->reason = atol(_alpm_strtrim(line));
+			} else if(strcmp(line, "%SIZE%") == 0 || strcmp(line, "%CSIZE%") == 0) {
+				/* NOTE: the CSIZE and SIZE fields both share the "size" field
+				 *       in the pkginfo_t struct.  This can be done b/c CSIZE
+				 *       is currently only used in sync databases, and SIZE is
+				 *       only used in local databases.
+				 */
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				info->size = atol(_alpm_strtrim(line));
+				/* also store this value to isize if isize is unset */
+				if(info->isize == 0) {
+					info->isize = info->size;
+				}
+			} else if(strcmp(line, "%ISIZE%") == 0) {
+				/* ISIZE (installed size) tag only appears in sync repositories,
+				 * not the local one. */
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				info->isize = atol(_alpm_strtrim(line));
+			} else if(strcmp(line, "%MD5SUM%") == 0) {
+				/* MD5SUM tag only appears in sync repositories,
+				 * not the local one. */
+				if(_pack_file_gets(db->handle, pf, line, 512) == 0) {
+					goto error;
+				}
+				STRDUP(info->md5sum, _alpm_strtrim(line), goto error);
+			} else if(strcmp(line, "%REPLACES%") == 0) {
+				while(_pack_file_gets(db->handle, pf, line, 512) &&
+						strlen(_alpm_strtrim(line))) {
+					char *linedup;
+					STRDUP(linedup, _alpm_strtrim(line), goto error);
+					info->replaces = alpm_list_add(info->replaces, linedup);
+				}
+			} else if(strcmp(line, "%FORCE%") == 0) {
+				info->force = 1;
+			}
+		}
+		_pack_close_file(pf);
+		pf = NULL;
+	}
+
+	/* FILES */
+	if(inforeq & INFRQ_FILES) {
+		if((pf = _pack_open_file(db->handle, pkgname, "files", PO_READ)) == NULL) {
+			_alpm_log(PM_LOG_ERROR, _("could not open package %s\n"), pkgname);
+			goto error;
+		}
+		while(_pack_file_gets(db->handle, pf, line, 256) &&
+			strlen(_alpm_strtrim(line))) {
+			_alpm_strtrim(line);
+			if(strcmp(line, "%FILES%") == 0) {
+				while(_pack_file_gets(db->handle, pf, line, 512) &&
+						strlen(_alpm_strtrim(line))) {
+					char *linedup;
+					STRDUP(linedup, _alpm_strtrim(line), goto error);
+					info->files = alpm_list_add(info->files, linedup);
+				}
+			} else if(strcmp(line, "%BACKUP%") == 0) {
+				while(_pack_file_gets(db->handle, pf, line, 512) &&
+						strlen(_alpm_strtrim(line))) {
+					char *linedup;
+					STRDUP(linedup, _alpm_strtrim(line), goto error);
+					info->backup = alpm_list_add(info->backup, linedup);
+				}
+			}
+		}
+		_pack_close_file(pf);
+		pf = NULL;
+	}
+
+	/* DEPENDS */
+	if(inforeq & INFRQ_DEPENDS) {
+		if((pf = _pack_open_file(db->handle, pkgname, "depends", PO_READ)) == NULL) {
+			_alpm_log(PM_LOG_ERROR, _("could not open package %s\n"), pkgname);
+			goto error;
+		}
+		while(!_pack_file_eof(db->handle, pf)) {
+			_pack_file_gets(db->handle, pf, line, 255);
+			_alpm_strtrim(line);
+			if(strcmp(line, "%DEPENDS%") == 0) {
+				while(_pack_file_gets(db->handle, pf, line, 512) &&
+						strlen(_alpm_strtrim(line))) {
+					pmdepend_t *dep = _alpm_splitdep(_alpm_strtrim(line));
+					info->depends = alpm_list_add(info->depends, dep);
+				}
+			} else if(strcmp(line, "%OPTDEPENDS%") == 0) {
+				while(_pack_file_gets(db->handle, pf, line, 512) &&
+						strlen(_alpm_strtrim(line))) {
+					char *linedup;
+					STRDUP(linedup, _alpm_strtrim(line), goto error);
+					info->optdepends = alpm_list_add(info->optdepends, linedup);
+				}
+			} else if(strcmp(line, "%CONFLICTS%") == 0) {
+				while(_pack_file_gets(db->handle, pf, line, 512) &&
+						strlen(_alpm_strtrim(line))) {
+					char *linedup;
+					STRDUP(linedup, _alpm_strtrim(line), goto error);
+					info->conflicts = alpm_list_add(info->conflicts, linedup);
+				}
+			} else if(strcmp(line, "%PROVIDES%") == 0) {
+				while(_pack_file_gets(db->handle, pf, line, 512) &&
+						strlen(_alpm_strtrim(line))) {
+					char *linedup;
+					STRDUP(linedup, _alpm_strtrim(line), goto error);
+					info->provides = alpm_list_add(info->provides, linedup);
+				}
+			}
+		}
+		_pack_close_file(pf);
+		pf = NULL;
+	}
+
+	/* DELTAS */
+	if(inforeq & INFRQ_DELTAS) {
+		if((pf = _pack_open_file(db->handle, pkgname, "deltas", PO_READ)) != NULL) {
+			while(!_pack_file_eof(db->handle, pf)) {
+				_pack_file_gets(db->handle, pf, line, 255);
+				_alpm_strtrim(line);
+				if(strcmp(line, "%DELTAS%") == 0) {
+					while(_pack_file_gets(db->handle, pf, line, 512) &&
+							strlen(_alpm_strtrim(line))) {
+						pmdelta_t *delta = _alpm_delta_parse(line);
+						if(delta) {
+							info->deltas = alpm_list_add(info->deltas, delta);
+						}
+					}
+				}
+			}
+			_pack_close_file(pf);
+			pf = NULL;
+		}
+	}
+	/* INSTALL */
+	if(inforeq & INFRQ_SCRIPTLET) {
+		if((pf = _pack_open_file(db->handle, pkgname, "depends", PO_READ)) != NULL) {
+			info->scriptlet = 1;
+			_pack_close_file(pf);
+		}
+	}
+
+	/* internal */
+	info->infolevel |= inforeq;
+
+	return(0);
+
+error:
+	if(pf) {
+		_pack_close_file(pf);
+	}
+	return(-1);
+}
+
+int _alpm_db_write(pmdb_t *db, pmpkg_t *info, pmdbinfrq_t inforeq)
+{
+	char pkgname[PATH_MAX];
+	alpm_list_t *lp = NULL;
+	int retval = 0;
+	int local = 0;
+	pack_file_t *pf;
+	char buffer[1024];
+	size_t len;
+
+	ALPM_LOG_FUNC;
+
+	if(db == NULL || info == NULL) {
+		return(-1);
+	}
+
+	if(strcmp(db->treename, "local") == 0) {
+		local = 1;
+	}
+
+	snprintf(pkgname, PATH_MAX, "%s-%s", info->name, info->version);
+
+	/* DESC */
+	if(inforeq & INFRQ_DESC) {
+		_alpm_log(PM_LOG_DEBUG, "writing %s-%s DESC information back to db\n",
+				info->name, info->version);
+		if((pf = _pack_open_file(db->handle, pkgname, "desc", PO_WRITE)) == NULL) {
+			_alpm_log(PM_LOG_ERROR, _("could not open package %s\n"), pkgname);
+			retval = -1;
+			goto cleanup;
+		}
+
+		len = snprintf(buffer, 1024, "%%NAME%%\n%s\n\n"
+						"%%VERSION%%\n%s\n\n", info->name, info->version);
+		_pack_file_puts(db->handle, pf, buffer, len);
+		if(info->desc) {
+			len = snprintf(buffer, 1024, "%%DESC%%\n"
+							"%s\n\n", info->desc);
+			_pack_file_puts(db->handle, pf, buffer, len);
+		}
+		if(info->groups) {
+			_pack_file_puts(db->handle, pf, "%GROUPS%\n", 9);
+			for(lp = info->groups; lp; lp = lp->next) {
+				len = snprintf(buffer, 1024, "%s\n", (char *)lp->data);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			_pack_file_puts(db->handle, pf, "\n", 1);
+		}
+		if(info->replaces) {
+			_pack_file_puts(db->handle, pf, "%REPLACES%\n", 11);
+			for(lp = info->replaces; lp; lp = lp->next) {
+				len = snprintf(buffer, 1024, "%s\n", (char *)lp->data);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			len = snprintf(buffer, 1024, "\n");
+			_pack_file_puts(db->handle, pf, buffer, len);
+		}
+		if(info->force) {
+			len = snprintf(buffer, 1024, "%%FORCE%%\n\n");
+			_pack_file_puts(db->handle, pf, buffer, len);
+		}
+		if(local) {
+			if(info->url) {
+				len = snprintf(buffer, 1024, "%%URL%%\n"
+								"%s\n\n", info->url);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			if(info->licenses) {
+				_pack_file_puts(db->handle, pf, "%LICENSE%\n", 10);
+				for(lp = info->licenses; lp; lp = lp->next) {
+					len = snprintf(buffer, 1024, "%s\n", (char *)lp->data);
+					_pack_file_puts(db->handle, pf, buffer, len);
+				}
+				len = snprintf(buffer, 1024, "\n");
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			if(info->arch) {
+				len = snprintf(buffer, 1024, "%%ARCH%%\n"
+								"%s\n\n", info->arch);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			if(info->builddate) {
+				len = snprintf(buffer, 1024, "%%BUILDDATE%%\n"
+								"%ju\n\n", (uintmax_t)info->builddate);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			if(info->installdate) {
+				len = snprintf(buffer, 1024, "%%INSTALLDATE%%\n"
+								"%ju\n\n", (uintmax_t)info->installdate);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			if(info->packager) {
+				len = snprintf(buffer, 1024, "%%PACKAGER%%\n"
+								"%s\n\n", info->packager);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			if(info->isize) {
+				/* only write installed size, csize is irrelevant once installed */
+				len = snprintf(buffer, 1024, "%%SIZE%%\n"
+								"%ju\n\n", (intmax_t)info->isize);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			if(info->reason) {
+				len = snprintf(buffer, 1024, "%%REASON%%\n"
+								"%u\n\n", info->reason);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+		} else {
+			if(info->size) {
+				len = snprintf(buffer, 1024, "%%CSIZE%%\n"
+								"%ju\n\n", (intmax_t)info->size);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			if(info->isize) {
+				len = snprintf(buffer, 1024, "%%ISIZE%%\n"
+								"%ju\n\n", (intmax_t)info->isize);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			if(info->md5sum) {
+				len = snprintf(buffer, 1024, "%%MD5SUM%%\n"
+								"%s\n\n", info->md5sum);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+		}
+		_pack_close_file(pf);
+		pf = NULL;
+	}
+
+	/* FILES */
+	if(local && (inforeq & INFRQ_FILES)) {
+		_alpm_log(PM_LOG_DEBUG, "writing %s-%s FILES information back to db\n",
+				info->name, info->version);
+		if((pf = _pack_open_file(db->handle, pkgname, "files", PO_WRITE)) == NULL) {
+			_alpm_log(PM_LOG_ERROR, _("could not open package %s\n"), pkgname);
+			retval = -1;
+			goto cleanup;
+		}
+
+		if(info->files) {
+			len = snprintf(buffer, 1024, "%%FILES%%\n");
+			_pack_file_puts(db->handle, pf, buffer, len);
+			for(lp = info->files; lp; lp = lp->next) {
+				len = snprintf(buffer, 1024, "%s\n", (char *)lp->data);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			len = snprintf(buffer, 1024, "\n");
+			_pack_file_puts(db->handle, pf, buffer, len);
+		}
+		if(info->backup) {
+			len = snprintf(buffer, 1024, "%%BACKUP%%\n");
+			_pack_file_puts(db->handle, pf, buffer, len);
+			for(lp = info->backup; lp; lp = lp->next) {
+				len = snprintf(buffer, 1024, "%s\n", (char *)lp->data);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			len = snprintf(buffer, 1024, "\n");
+			_pack_file_puts(db->handle, pf, buffer, len);
+		}
+		_pack_close_file(pf);
+		pf = NULL;
+	}
+
+	/* DEPENDS */
+	if(inforeq & INFRQ_DEPENDS) {
+		_alpm_log(PM_LOG_DEBUG, "writing %s-%s DEPENDS information back to db\n",
+				info->name, info->version);
+		if((pf = _pack_open_file(db->handle, pkgname, "depends", PO_WRITE)) == NULL) {
+			_alpm_log(PM_LOG_ERROR, _("could not open package %s\n"), pkgname);
+			retval = -1;
+			goto cleanup;
+		}
+
+		if(info->depends) {
+			_pack_file_puts(db->handle, pf, "%DEPENDS%\n", 10);
+			for(lp = info->depends; lp; lp = lp->next) {
+				char *depstring = alpm_dep_get_string(lp->data);
+				len = snprintf(buffer, 1024, "%s\n", depstring);
+				_pack_file_puts(db->handle, pf, buffer, len);
+				free(depstring);
+			}
+			len = snprintf(buffer, 1024, "\n");
+			_pack_file_puts(db->handle, pf, buffer, len);
+		}
+		if(info->optdepends) {
+			_pack_file_puts(db->handle, pf, "%OPTDEPENDS%\n", 13);
+			for(lp = info->optdepends; lp; lp = lp->next) {
+				len = snprintf(buffer, 1024, "%s\n", (char *)lp->data);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			len = snprintf(buffer, 1024, "\n");
+			_pack_file_puts(db->handle, pf, buffer, len);
+		}
+		if(info->conflicts) {
+			_pack_file_puts(db->handle, pf, "%CONFLICTS%\n", 12);
+			for(lp = info->conflicts; lp; lp = lp->next) {
+				len = snprintf(buffer, 1024, "%s\n", (char *)lp->data);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			len = snprintf(buffer, 1024, "\n");
+			_pack_file_puts(db->handle, pf, buffer, len);
+		}
+		if(info->provides) {
+			_pack_file_puts(db->handle, pf, "%PROVIDES%\n", 11);
+			for(lp = info->provides; lp; lp = lp->next) {
+				len = snprintf(buffer, 1024, "%s\n", (char *)lp->data);
+				_pack_file_puts(db->handle, pf, buffer, len);
+			}
+			len = snprintf(buffer, 1024, "\n");
+			_pack_file_puts(db->handle, pf, buffer, len);
+		}
+		_pack_close_file(pf);
+		pf = NULL;
+	}
+
+	/* INSTALL */
+	/* nothing needed here (script is automatically extracted) */
+
+cleanup:
+	if(pf) {
+		_pack_close_file(pf);
+	}
+
+	return(retval);
+}
+
+int _alpm_db_remove(pmdb_t *db, pmpkg_t *info)
+{
+	int ret = 0;
+	char pkgname[512];
+
+	ALPM_LOG_FUNC;
+
+	snprintf(pkgname, 512, "%s-%s", info->name, info->version);
+	_pack_package_remove(db->handle, pkgname);
+
+	if(db == NULL || info == NULL) {
+		RET_ERR(PM_ERR_DB_NULL, -1);
+	}
+
+	return(ret);
+}
+
+void *_alpm_db_changelog_open(pmdb_t *db, pmpkg_t *pkg)
+{
+	char name[512];
+	snprintf(name, 512, "%s-%s", pkg->name, pkg->version);
+	return _pack_open_file(db->handle, name, "changelog", PO_READ);
+}
+
+int _alpm_db_changelog_close(pmdb_t *db, void *stream)
+{
+	_pack_close_file(stream);
+	return 0;
+}
+
+size_t _alpm_db_changelog_read(pmdb_t *db, const void *pf, void *ptr, size_t size)
+{
+	return _pack_file_gets(db->handle, pf, ptr, size);
+}
+
+/* vim: set ts=2 sw=2 noet: */
diff --git a/lib/libalpm/db.h b/lib/libalpm/db.h
index 96fac0d..96291fc 100644
--- a/lib/libalpm/db.h
+++ b/lib/libalpm/db.h
@@ -62,6 +62,9 @@ int _alpm_db_populate(pmdb_t *db);
 int _alpm_db_read(pmdb_t *db, pmpkg_t *info, pmdbinfrq_t inforeq);
 int _alpm_db_write(pmdb_t *db, pmpkg_t *info, pmdbinfrq_t inforeq);
 int _alpm_db_remove(pmdb_t *db, pmpkg_t *info);
+void *_alpm_db_changelog_open(pmdb_t *db, pmpkg_t *pkg);
+int _alpm_db_changelog_close(pmdb_t *db, void *stream);
+size_t _alpm_db_changelog_read(pmdb_t *db, const void *pf, void *ptr, size_t size);
 
 #endif /* _ALPM_DB_H */
 
diff --git a/lib/libalpm/package.c b/lib/libalpm/package.c
index ee0ff6f..7ab9710 100644
--- a/lib/libalpm/package.c
+++ b/lib/libalpm/package.c
@@ -447,13 +447,7 @@ void SYMEXPORT *alpm_pkg_changelog_open(pmpkg_t *pkg)
 	ASSERT(pkg != NULL, return(NULL));
 
 	if(pkg->origin == PKG_FROM_CACHE) {
-		char clfile[PATH_MAX];
-		snprintf(clfile, PATH_MAX, "%s/%s/%s-%s/changelog",
-				alpm_option_get_dbpath(),
-				alpm_db_get_name(handle->db_local),
-				alpm_pkg_get_name(pkg),
-				alpm_pkg_get_version(pkg));
-		return fopen(clfile, "r");
+		return(_alpm_db_changelog_open(handle->db_local, pkg));
 	} else if(pkg->origin == PKG_FROM_FILE) {
 		struct archive *archive = NULL;
 		struct archive_entry *entry;
@@ -500,7 +494,7 @@ size_t SYMEXPORT alpm_pkg_changelog_read(void *ptr, size_t size,
 {
 	size_t ret = 0;
 	if(pkg->origin == PKG_FROM_CACHE) {
-		ret = fread(ptr, 1, size, (FILE*)fp);
+		ret = _alpm_db_changelog_read(handle->db_local, fp, ptr, size);
 	} else if(pkg->origin == PKG_FROM_FILE) {
 		ret = archive_read_data((struct archive*)fp, ptr, size);
 	}
@@ -533,7 +527,7 @@ int SYMEXPORT alpm_pkg_changelog_close(const pmpkg_t *pkg, void *fp)
 {
 	int ret = 0;
 	if(pkg->origin == PKG_FROM_CACHE) {
-		ret = fclose((FILE*)fp);
+		ret = _alpm_db_changelog_close(handle->db_local, fp);
 	} else if(pkg->origin == PKG_FROM_FILE) {
 		ret = archive_read_finish((struct archive *)fp);
 	}
-- 
1.6.0.5


From mad at wol.de  Mon Dec 15 04:28:27 2008
From: mad at wol.de (Marc - A. Dahlhaus [ Administration | Westermann GmbH ])
Date: Mon, 15 Dec 2008 10:28:27 +0100
Subject: [pacman-dev] makepkg: package splitting
In-Reply-To: <493E655D.8030501@archlinux.org>
References: <493C87FD.8080501@archlinux.org>
	<1228822256.16706.12.camel@marc> <493E5D35.5050702@archlinux.org>
	<1228825387.16706.20.camel@marc>  <493E655D.8030501@archlinux.org>
Message-ID: <1229333307.8183.13.camel@marc>

Am Dienstag, den 09.12.2008, 22:32 +1000 schrieb Allan McRae:
> Marc - A. Dahlhaus [ Administration | Westermann GmbH ] wrote:
> >
> > I'll tested that again and i forgot to change two of the DESTDIR
> > locations to the new home of the pkgdirs... So the missing cleanup was
> > actually a stupid user error...
> >   
> 
> You should only be doing something like "make DESTDIR=$pkgdir install".  
> Makepkg points $pkgdir in the right place for each split package.  (I 
> will get around to documenting all this at some stage)
> 
> Allan

Hello Allan,

i think i found another bug. Every package that is a split-build and not
in the first position in the pkgname build-order shows an empty
"Optional dependencies" string during installation. I think there is
something wrong in restore_package_variables during variable resetting
to initial values or in backup_package_variables during the backup
declaration. It looks like there are to much quotes around the
declaration so that it declares an empty string. Maybe a "-z" check
before the actual variable declaration could fix that particular
problem.


Marc


From allan at archlinux.org  Mon Dec 15 06:04:18 2008
From: allan at archlinux.org (Allan McRae)
Date: Mon, 15 Dec 2008 21:04:18 +1000
Subject: [pacman-dev] makepkg: package splitting
In-Reply-To: <1229333307.8183.13.camel@marc>
References: <493C87FD.8080501@archlinux.org>	<1228822256.16706.12.camel@marc>
	<493E5D35.5050702@archlinux.org>	<1228825387.16706.20.camel@marc>
	<493E655D.8030501@archlinux.org> <1229333307.8183.13.camel@marc>
Message-ID: <BAY102-DAV5A8BCFCC5998522518EB286F40@phx.gbl>

Marc - A. Dahlhaus [ Administration | Westermann GmbH ] wrote:
> i think i found another bug. Every package that is a split-build and not
> in the first position in the pkgname build-order shows an empty
> "Optional dependencies" string during installation. I think there is
> something wrong in restore_package_variables during variable resetting
> to initial values or in backup_package_variables during the backup
> declaration. It looks like there are to much quotes around the
> declaration so that it declares an empty string. Maybe a "-z" check
> before the actual variable declaration could fix that particular
> problem.
>   

Fix now on my working branch:
http://dev.archlinux.org/~allan/gitweb/gitweb.cgi?p=pacman.git;a=commitdiff;h=91014b8b

Thanks for your testing!
Allan





From allan at archlinux.org  Mon Dec 15 06:43:28 2008
From: allan at archlinux.org (Allan McRae)
Date: Mon, 15 Dec 2008 21:43:28 +1000
Subject: [pacman-dev] makepkg: package splitting
In-Reply-To: <BAY102-DAV5A8BCFCC5998522518EB286F40@phx.gbl>
References: <493C87FD.8080501@archlinux.org>	<1228822256.16706.12.camel@marc>	<493E5D35.5050702@archlinux.org>	<1228825387.16706.20.camel@marc>	<493E655D.8030501@archlinux.org>
	<1229333307.8183.13.camel@marc>
	<BAY102-DAV5A8BCFCC5998522518EB286F40@phx.gbl>
Message-ID: <BAY102-DAV11B69E9B35B290208D2CFC86F40@phx.gbl>

Allan McRae wrote:
> Marc - A. Dahlhaus [ Administration | Westermann GmbH ] wrote:
>> i think i found another bug. Every package that is a split-build and not
>> in the first position in the pkgname build-order shows an empty
>> "Optional dependencies" string during installation. I think there is
>> something wrong in restore_package_variables during variable resetting
>> to initial values or in backup_package_variables during the backup
>> declaration. It looks like there are to much quotes around the
>> declaration so that it declares an empty string. Maybe a "-z" check
>> before the actual variable declaration could fix that particular
>> problem.
>>   
>
> Fix now on my working branch:
> http://dev.archlinux.org/~allan/gitweb/gitweb.cgi?p=pacman.git;a=commitdiff;h=91014b8b 
>
>

And when I say my working branch, I mean my splitpkg branch...




From mad at wol.de  Mon Dec 15 06:53:55 2008
From: mad at wol.de (Marc - A. Dahlhaus [ Administration | Westermann GmbH ])
Date: Mon, 15 Dec 2008 12:53:55 +0100
Subject: [pacman-dev] makepkg: package splitting
In-Reply-To: <494642E0.6030805@archlinux.org>
References: <493C87FD.8080501@archlinux.org>
	<1228822256.16706.12.camel@marc>	<493E5D35.5050702@archlinux.org>
	<1228825387.16706.20.camel@marc>	<493E655D.8030501@archlinux.org>
	<1229333307.8183.13.camel@marc>
	<BAY102-DAV5A8BCFCC5998522518EB286F40@phx.gbl>
	<494642E0.6030805@archlinux.org>
Message-ID: <1229342035.10057.3.camel@marc>

Am Montag, den 15.12.2008, 21:43 +1000 schrieb Allan McRae:
> Allan McRae wrote:
> > Marc - A. Dahlhaus [ Administration | Westermann GmbH ] wrote:
> >> i think i found another bug. Every package that is a split-build and not
> >> in the first position in the pkgname build-order shows an empty
> >> "Optional dependencies" string during installation. I think there is
> >> something wrong in restore_package_variables during variable resetting
> >> to initial values or in backup_package_variables during the backup
> >> declaration. It looks like there are to much quotes around the
> >> declaration so that it declares an empty string. Maybe a "-z" check
> >> before the actual variable declaration could fix that particular
> >> problem.
> >>   
> >
> > Fix now on my working branch:
> > http://dev.archlinux.org/~allan/gitweb/gitweb.cgi?p=pacman.git;a=commitdiff;h=91014b8b 
> >
> >
> 
> And when I say my working branch, I mean my splitpkg branch...

Nice, i'll test and report back...
Is there any schedule on when it will be merged to mainline?

Marc


From sivertb at stud.ntnu.no  Mon Dec 15 06:57:26 2008
From: sivertb at stud.ntnu.no (Sivert Berg)
Date: Mon, 15 Dec 2008 12:57:26 +0100
Subject: [pacman-dev] [PATCH 2/2] Added a new database backend
In-Reply-To: <1229332855-3569-2-git-send-email-sivertb@stud.ntnu.no>
References: <1229332855-3569-1-git-send-email-sivertb@stud.ntnu.no>
	<1229332855-3569-2-git-send-email-sivertb@stud.ntnu.no>
Message-ID: <20081215125726.ba95293b.sivertb@stud.ntnu.no>

Sorry for the double post, I thought the earlier post was ignored because it was too long.

From sivertb at stud.ntnu.no  Mon Dec 15 07:24:36 2008
From: sivertb at stud.ntnu.no (Sivert Berg)
Date: Mon, 15 Dec 2008 13:24:36 +0100
Subject: [pacman-dev] [PATCH] Added a new database backend
In-Reply-To: <1229290864.3213.56.camel@tandk.math.u-szeged.hu>
References: <1229186134-993-1-git-send-email-sivertb@stud.ntnu.no>
	<1229290864.3213.56.camel@tandk.math.u-szeged.hu>
Message-ID: <20081215132436.9dec40b5.sivertb@stud.ntnu.no>

On Sun, 14 Dec 2008 22:41:04 +0100
Nagy Gabor <ngaba at bibl.u-szeged.hu> wrote:

> First of all, many thanks for your huge work. I will support your
> attempt on reworking our db backend stuff. I haven't completely reviewed
> your patch yet (it is quite advanced, I guess I will learn a lot from
> here), but I have some "pre-reading" comments. (Keep in my mind: I am
> just a pacman contributor here.)
> 

Thanks a lot for taking the time to look at the patch, it's much appreciated!

> > -	be_files.c \
> > +	be_ at BACKEND@.c \
> 2. I think we need a more sophisticated approach here. I would be happy,
> if we had database backend plugins (be_files.so, be_packed.so,
> be_sqlite.so, ...) for dlopen(), and some way to inform pacman about the
> database handler (for example: /var/lib/pacman/local/.backend, which has
> one line: packed). I think this is quite straightforward (but needed) to
> implement.

I agree with that. The @BACKEND@ was just a quick hack to prevent things from breaking.

> 3.* Our whole db back-end system needs some redesign (independent from
> your work): If we have fast database back-end, I am not sure we need
> this ugly pkgcache stuff. Probably pkgcache should be moved to back-end
> level (if needed).

Yes, if you look at the packed backend the pkgcache basicly duplicates the data that's already mmaped, kind of a waste of memory.

> 4.* When we rework pmpkg_t, we must keep in mind, that at this moment it
> is the structure of database entry *and* package file. When we
> restructure things, we must find a good place for ".tar.gz".

Maybe we should start a thread where we could just throw around some ideas on possible ways to rework the current package handling in a sane way.

Cheers
-- 
Sivert Berg <sivertb at stud.ntnu.no>

From allan at archlinux.org  Mon Dec 15 07:24:21 2008
From: allan at archlinux.org (Allan McRae)
Date: Mon, 15 Dec 2008 22:24:21 +1000
Subject: [pacman-dev] makepkg: package splitting
In-Reply-To: <1229342035.10057.3.camel@marc>
References: <493C87FD.8080501@archlinux.org>	<1228822256.16706.12.camel@marc>	<493E5D35.5050702@archlinux.org>	<1228825387.16706.20.camel@marc>	<493E655D.8030501@archlinux.org>	<1229333307.8183.13.camel@marc>	<BAY102-DAV5A8BCFCC5998522518EB286F40@phx.gbl>	<494642E0.6030805@archlinux.org>
	<1229342035.10057.3.camel@marc>
Message-ID: <BAY102-DAV1260E02BFFA76D976584F086F40@phx.gbl>

Marc - A. Dahlhaus [ Administration | Westermann GmbH ] wrote:
> Am Montag, den 15.12.2008, 21:43 +1000 schrieb Allan McRae:
>   
> Nice, i'll test and report back...
> Is there any schedule on when it will be merged to mainline?
>
>   

The current schedule is when I tell Dan it ready...  I want to add 
documentation and clean up some of the output with the package splitting 
but otherwise I am happy with it now.  So, the answer is "soonish".

Allan




From mad at wol.de  Mon Dec 15 07:32:09 2008
From: mad at wol.de (Marc - A. Dahlhaus [ Administration | Westermann GmbH ])
Date: Mon, 15 Dec 2008 13:32:09 +0100
Subject: [pacman-dev] makepkg: package splitting
In-Reply-To: <49464C75.3060805@archlinux.org>
References: <493C87FD.8080501@archlinux.org>
	<1228822256.16706.12.camel@marc>	<493E5D35.5050702@archlinux.org>
	<1228825387.16706.20.camel@marc>	<493E655D.8030501@archlinux.org>
	<1229333307.8183.13.camel@marc>
	<BAY102-DAV5A8BCFCC5998522518EB286F40@phx.gbl>
	<494642E0.6030805@archlinux.org> <1229342035.10057.3.camel@marc>
	<49464C75.3060805@archlinux.org>
Message-ID: <1229344329.10057.11.camel@marc>

Am Montag, den 15.12.2008, 22:24 +1000 schrieb Allan McRae:
> Marc - A. Dahlhaus [ Administration | Westermann GmbH ] wrote:
> > Am Montag, den 15.12.2008, 21:43 +1000 schrieb Allan McRae:
> >   
> > Nice, i'll test and report back...
> > Is there any schedule on when it will be merged to mainline?
> >
> >   
> 
> The current schedule is when I tell Dan it ready...  I want to add 
> documentation and clean up some of the output with the package splitting 
> but otherwise I am happy with it now.  So, the answer is "soonish".
> 
> Allan

If you need any help, be it implementation of missing functionality or
testing of something, let me know.

I'll also try to come up with a option to build only a subset of the
split-packages (if it makes sense) but haven't looked deeper into it
yet.

Marc


From allan at archlinux.org  Mon Dec 15 08:11:44 2008
From: allan at archlinux.org (Allan McRae)
Date: Mon, 15 Dec 2008 23:11:44 +1000
Subject: [pacman-dev] Places to start with pacman code - Was: GPG work
In-Reply-To: <e94122a10812150037r55835f43t9091640a67c5f281@mail.gmail.com>
References: <449c10960812071318v7a7fb427nb19f4f5726496234@mail.gmail.com>	<20081208115553.3c2fd3d9@tux1.brauer.lan>	<449c10960812080434w6ba33471k9fd55ae681b69cf3@mail.gmail.com>	<20081213132308.5e8c5b4e@tux1.brauer.lan>	<449c10960812130618g1d9625d8y4aba0df77b37e9d5@mail.gmail.com>
	<e94122a10812150037r55835f43t9091640a67c5f281@mail.gmail.com>
Message-ID: <BAY102-DAV75154A0A27EF9C6E920FF86F40@phx.gbl>

Jatheendra wrote:
> There are a few people like myself who are interested in working on
> pacman. I started looking at pacman code a few weeks back, had a look
> at bug tracker and send a patch for a simple feature(pacman -Qo
> which). After that i was stuck because i couldnt find anything to work
> on. Most of the feature requests are huge changes and it doesnt seem
> that there is consent on those even within current developers. So it
> would be better if we can have a roadmap sort of thing for pacman (
> not the TODO.dan and TODO.aaron in source) with features which
> developers want to include but are unable because of lack of time.
> Basically something which beginners can use to get their hand
> wet.........
>
> (If such a list already exists, i couldnt find it)
>
> Thanks
>   

Hi,

This is something I have not got around to finishing which should be 
fairly simple to do:

FS#9424 - db.lck storing pid
I have a patch which stores the pid in the lock file here:
http://dev.archlinux.org/~allan/gitweb/gitweb.cgi?p=pacman.git;a=commitdiff;h=060c73be
The idea is that when a lock file is present, pacman reads in this 
number, checks if the process exists and if not automatically removes 
the lock file.

Other than that, look at bugs in the low or very low priority 
categories.  Most ideas we have for pacman improvement are posted there 
so we don't lose them.  The lower priority level normally mean that it 
is a nice idea but the devs have other things they want to do.  I found 
these to be a goldmine of fairly simple fixes when I was learning the 
pacman code base.  If you have queries about a specific idea, just post 
here asking for opinions or some direction on how to fix it.

Allan




From dpmcgee at gmail.com  Mon Dec 15 08:15:01 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Mon, 15 Dec 2008 07:15:01 -0600
Subject: [pacman-dev] [PATCH] Added a new database backend
In-Reply-To: <20081215132436.9dec40b5.sivertb@stud.ntnu.no>
References: <1229186134-993-1-git-send-email-sivertb@stud.ntnu.no>
	<1229290864.3213.56.camel@tandk.math.u-szeged.hu>
	<20081215132436.9dec40b5.sivertb@stud.ntnu.no>
Message-ID: <449c10960812150515j18c72276ya70bf8b82862b65b@mail.gmail.com>

On Mon, Dec 15, 2008 at 6:24 AM, Sivert Berg <sivertb at stud.ntnu.no> wrote:
> On Sun, 14 Dec 2008 22:41:04 +0100
> Nagy Gabor <ngaba at bibl.u-szeged.hu> wrote:
>
>> First of all, many thanks for your huge work. I will support your
>> attempt on reworking our db backend stuff. I haven't completely reviewed
>> your patch yet (it is quite advanced, I guess I will learn a lot from
>> here), but I have some "pre-reading" comments. (Keep in my mind: I am
>> just a pacman contributor here.)
>>
>
> Thanks a lot for taking the time to look at the patch, it's much appreciated!

Just as an FYI, I haven't forgotten this patch but I want to have time
to sit down and take it all in. I would really love to go somewhere
with all of this backend stuff as we know it can be improved, but I
have struggled each time trying to do it within the framework of this
be_* junk. I'm glad to see someone has had mild success.

I will try to look at this patch early this week.

-Dan

From dpmcgee at gmail.com  Mon Dec 15 08:28:18 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Mon, 15 Dec 2008 07:28:18 -0600
Subject: [pacman-dev] Places to start with pacman code - Was: GPG work
In-Reply-To: <BAY102-DAV75154A0A27EF9C6E920FF86F40@phx.gbl>
References: <449c10960812071318v7a7fb427nb19f4f5726496234@mail.gmail.com>
	<20081208115553.3c2fd3d9@tux1.brauer.lan>
	<449c10960812080434w6ba33471k9fd55ae681b69cf3@mail.gmail.com>
	<20081213132308.5e8c5b4e@tux1.brauer.lan>
	<449c10960812130618g1d9625d8y4aba0df77b37e9d5@mail.gmail.com>
	<e94122a10812150037r55835f43t9091640a67c5f281@mail.gmail.com>
	<BAY102-DAV75154A0A27EF9C6E920FF86F40@phx.gbl>
Message-ID: <449c10960812150528h1f38930ej61bc29ed17973a24@mail.gmail.com>

On Mon, Dec 15, 2008 at 7:11 AM, Allan McRae <allan at archlinux.org> wrote:
> Jatheendra wrote:
>>
>> There are a few people like myself who are interested in working on
>> pacman. I started looking at pacman code a few weeks back, had a look
>> at bug tracker and send a patch for a simple feature(pacman -Qo
>> which). After that i was stuck because i couldnt find anything to work
>> on. Most of the feature requests are huge changes and it doesnt seem
>> that there is consent on those even within current developers. So it
>> would be better if we can have a roadmap sort of thing for pacman (
>> not the TODO.dan and TODO.aaron in source) with features which
>> developers want to include but are unable because of lack of time.
>> Basically something which beginners can use to get their hand
>> wet.........
>>
>> (If such a list already exists, i couldnt find it)
>>
>> Thanks
>>
>
> Hi,
>
> This is something I have not got around to finishing which should be fairly
> simple to do:
>
> FS#9424 - db.lck storing pid
> I have a patch which stores the pid in the lock file here:
> http://dev.archlinux.org/~allan/gitweb/gitweb.cgi?p=pacman.git;a=commitdiff;h=060c73be
> The idea is that when a lock file is present, pacman reads in this number,
> checks if the process exists and if not automatically removes the lock file.
>
> Other than that, look at bugs in the low or very low priority categories.
>  Most ideas we have for pacman improvement are posted there so we don't lose
> them.  The lower priority level normally mean that it is a nice idea but the
> devs have other things they want to do.  I found these to be a goldmine of
> fairly simple fixes when I was learning the pacman code base.  If you have
> queries about a specific idea, just post here asking for opinions or some
> direction on how to fix it.

A few ideas:
* globbing for sync operations: http://bugs.archlinux.org/task/1561
* multiple pacman instances on same cache: http://bugs.archlinux.org/task/8226
* Allow -Qo to perform functional which (started, looks like it still
needs some follow-up): http://bugs.archlinux.org/task/8798

From aaronmgriffin at gmail.com  Mon Dec 15 12:45:05 2008
From: aaronmgriffin at gmail.com (Aaron Griffin)
Date: Mon, 15 Dec 2008 11:45:05 -0600
Subject: [pacman-dev] makepkg: package splitting
In-Reply-To: <BAY102-DAV1260E02BFFA76D976584F086F40@phx.gbl>
References: <493C87FD.8080501@archlinux.org> <1228822256.16706.12.camel@marc>
	<493E5D35.5050702@archlinux.org> <1228825387.16706.20.camel@marc>
	<493E655D.8030501@archlinux.org> <1229333307.8183.13.camel@marc>
	<BAY102-DAV5A8BCFCC5998522518EB286F40@phx.gbl>
	<494642E0.6030805@archlinux.org> <1229342035.10057.3.camel@marc>
	<BAY102-DAV1260E02BFFA76D976584F086F40@phx.gbl>
Message-ID: <d64a48980812150945t77191026p1efc02f0bb28c15d@mail.gmail.com>

On Mon, Dec 15, 2008 at 6:24 AM, Allan McRae <allan at archlinux.org> wrote:
> Marc - A. Dahlhaus [ Administration | Westermann GmbH ] wrote:
>>
>> Am Montag, den 15.12.2008, 21:43 +1000 schrieb Allan McRae:
>>  Nice, i'll test and report back...
>> Is there any schedule on when it will be merged to mainline?
>>
>>
>
> The current schedule is when I tell Dan it ready...  I want to add
> documentation and clean up some of the output with the package splitting but
> otherwise I am happy with it now.  So, the answer is "soonish".

Yay. I'm hoping for a Christmas present 8)

From gerbra at archlinux.de  Mon Dec 15 13:47:44 2008
From: gerbra at archlinux.de (Gerhard Brauer)
Date: Mon, 15 Dec 2008 19:47:44 +0100
Subject: [pacman-dev] [PATCH] (newgpg) Let pacman specify GnuPG's home
 directory.
In-Reply-To: <20081214190116.VZHO27943.eastrmmtao104.cox.net@eastrmimpo02.cox.net>
References: <20081214190116.VZHO27943.eastrmmtao104.cox.net@eastrmimpo02.cox.net>
Message-ID: <20081215194744.34be654e@tux1.brauer.lan>

Am Sun, 14 Dec 2008 12:59:39 -0600
schrieb Chris Brannon <cmbrannon at cox.net>:

> GnuPG looks for configuration files and keyrings in its home
> directory. For a user, that is typically ~/.gnupg.
> This patch causes pacman to use /etc/pacman.d/gnupg/ as the default
> GnuPG home.  One may override the default using --gpgdir on the
> command-line or GPGDir in pacman's configuration file.

Seems to work here in test environment.
I copied root's pubrig and trustdb to /etc/pacman.d/gnupg/
The package itself isn't checked (.sig file or signature), but that was
not the reason of your patch.

What i mentioned in another posting (that libalpm don't find the
keyring dir) is gone with your patch.


Regards
	Gerhard

From aaronmgriffin at gmail.com  Mon Dec 15 13:55:50 2008
From: aaronmgriffin at gmail.com (Aaron Griffin)
Date: Mon, 15 Dec 2008 12:55:50 -0600
Subject: [pacman-dev] [PATCH 2/2] Added a new database backend
In-Reply-To: <20081215125726.ba95293b.sivertb@stud.ntnu.no>
References: <1229332855-3569-1-git-send-email-sivertb@stud.ntnu.no>
	<1229332855-3569-2-git-send-email-sivertb@stud.ntnu.no>
	<20081215125726.ba95293b.sivertb@stud.ntnu.no>
Message-ID: <d64a48980812151055v29daaae0k6668b10ea555a795@mail.gmail.com>

On Mon, Dec 15, 2008 at 5:57 AM, Sivert Berg <sivertb at stud.ntnu.no> wrote:
> Sorry for the double post, I thought the earlier post was ignored because it was too long.

Naw, Dan is just all sorts of busy. Don't get scared, we're all
salivating over this. The prospect of a new backend that actually
works is fun

From aaronmgriffin at gmail.com  Mon Dec 15 13:58:53 2008
From: aaronmgriffin at gmail.com (Aaron Griffin)
Date: Mon, 15 Dec 2008 12:58:53 -0600
Subject: [pacman-dev] [PATCH] (newgpg) Let pacman specify GnuPG's home
	directory.
In-Reply-To: <20081215194744.34be654e@tux1.brauer.lan>
References: <20081214190116.VZHO27943.eastrmmtao104.cox.net@eastrmimpo02.cox.net>
	<20081215194744.34be654e@tux1.brauer.lan>
Message-ID: <d64a48980812151058x2b132fa3tde4400c872226f18@mail.gmail.com>

On Mon, Dec 15, 2008 at 12:47 PM, Gerhard Brauer <gerbra at archlinux.de> wrote:
> Am Sun, 14 Dec 2008 12:59:39 -0600
> schrieb Chris Brannon <cmbrannon at cox.net>:
>
>> GnuPG looks for configuration files and keyrings in its home
>> directory. For a user, that is typically ~/.gnupg.
>> This patch causes pacman to use /etc/pacman.d/gnupg/ as the default
>> GnuPG home.  One may override the default using --gpgdir on the
>> command-line or GPGDir in pacman's configuration file.
>
> Seems to work here in test environment.
> I copied root's pubrig and trustdb to /etc/pacman.d/gnupg/
> The package itself isn't checked (.sig file or signature), but that was
> not the reason of your patch.
>
> What i mentioned in another posting (that libalpm don't find the
> keyring dir) is gone with your patch.

I like /etc/pacman.d/gnupg/ as the default dir. +1 from me 8)

From cmbrannon at cox.net  Mon Dec 15 14:50:49 2008
From: cmbrannon at cox.net (Chris Brannon)
Date: Mon, 15 Dec 2008 13:50:49 -0600
Subject: [pacman-dev] [PATCH] (newgpg) Let pacman specify GnuPG's home
	directory.
In-Reply-To: <20081215194744.34be654e@tux1.brauer.lan> (Gerhard Brauer's
	message of "Mon\, 15 Dec 2008 19\:47\:44 +0100")
References: <20081214190116.VZHO27943.eastrmmtao104.cox.net@eastrmimpo02.cox.net>
	<20081215194744.34be654e@tux1.brauer.lan>
Message-ID: <87fxkpb4km.fsf@cox.net>

Gerhard Brauer <gerbra at archlinux.de> writes:

> Seems to work here in test environment.
> I copied root's pubrig and trustdb to /etc/pacman.d/gnupg/
> The package itself isn't checked (.sig file or signature), but that was
> not the reason of your patch.

If the signing key is not found in your public keyring, then pacman
will install the package without checking the signature.  OTOH, if the
signing key is available but not trusted and valid, pacman will refuse
to install the signed package.  Try removing trustdb from the gpg
directory, while leaving pubring intact.  You'll see what I mean.
To summarize, it checks the signature if the key is found in pubring.

I think pacman should at least complain if the signing key is not found
in the public keyring.  Thoughts?

-- Chris

From gerbra at archlinux.de  Mon Dec 15 15:11:21 2008
From: gerbra at archlinux.de (Gerhard Brauer)
Date: Mon, 15 Dec 2008 21:11:21 +0100
Subject: [pacman-dev] [PATCH] (newgpg) Let pacman specify GnuPG's home
 directory.
In-Reply-To: <87fxkpb4km.fsf@cox.net>
References: <20081214190116.VZHO27943.eastrmmtao104.cox.net@eastrmimpo02.cox.net>
	<20081215194744.34be654e@tux1.brauer.lan> <87fxkpb4km.fsf@cox.net>
Message-ID: <20081215211121.0d1d72ff@tux1.brauer.lan>

Am Mon, 15 Dec 2008 13:50:49 -0600
schrieb Chris Brannon <cmbrannon at cox.net>:

> Try removing trustdb from the gpg
> directory, while leaving pubring intact.  You'll see what I mean.
> To summarize, it checks the signature if the key is found in pubring.

Yes, you're right. Got this in debug when the key is not trusted (no
trustdb):
summary=0
fpr=0403BBB7C3907CDA95FBB3E61221825A96A08062
status=0
timestamp=1228738371
wrong_key_usage=0
pka_trust=0
chain_model=0
validity=0
validity_reason=0
key=17
hash=2
error: Package /var/cache/pacman/pkg/abook-0.5.6-4-i686.pkg.tar.gz has
an invalid signature. abook-0.5.6-4-i686.pkg.tar.gz is invalid or corrupted

And this when the signing pubkey is trusted:
summary=3
fpr=0403BBB7C3907CDA95FBB3E61221825A96A08062
status=0
timestamp=1228738371
wrong_key_usage=0
pka_trust=0
chain_model=0
validity=4
validity_reason=0
key=17
hash=2
debug: installing packages
debug: found cached pkg: /var/cache/pacman/pkg/abook-0.5.6-4-i686.pkg.tar.gz
debug: loading target '/var/cache/pacman/pkg/abook-0.5.6-4-i686.pkg.tar.gz'
debug: no package signature file found

The last line confused me...

> I think pacman should at least complain if the signing key is not
> found in the public keyring.  Thoughts?

IMHO pacman should refuse to install anything from core and extra if
the signature is not found or corrupted.
I don't know what to to with community (maybe a second keyring with
TU signatures?)

My thoughts were to make a option to each repo section in pacman.conf.
With this option: Keyring = /foo/bar we have an indicator that pacman
should check for correct signatures and users could have their
unsigned or self-signed repos additionally.

> -- Chris

Regards
	Gerhard


From dpmcgee at gmail.com  Mon Dec 15 17:19:59 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Mon, 15 Dec 2008 16:19:59 -0600
Subject: [pacman-dev] [PATCH] (newgpg) Let pacman specify GnuPG's home
	directory.
In-Reply-To: <20081215211121.0d1d72ff@tux1.brauer.lan>
References: <20081214190116.VZHO27943.eastrmmtao104.cox.net@eastrmimpo02.cox.net>
	<20081215194744.34be654e@tux1.brauer.lan> <87fxkpb4km.fsf@cox.net>
	<20081215211121.0d1d72ff@tux1.brauer.lan>
Message-ID: <449c10960812151419x4e1ce960g6781bd97b76aba75@mail.gmail.com>

On Mon, Dec 15, 2008 at 2:11 PM, Gerhard Brauer <gerbra at archlinux.de> wrote:
> Am Mon, 15 Dec 2008 13:50:49 -0600
> schrieb Chris Brannon <cmbrannon at cox.net>:
>> I think pacman should at least complain if the signing key is not
>> found in the public keyring.  Thoughts?
>
> IMHO pacman should refuse to install anything from core and extra if
> the signature is not found or corrupted.
> I don't know what to to with community (maybe a second keyring with
> TU signatures?)

Pacman knows nothing about [core], [extra], and [community], so this
will not be possible. However, I had considered a few possibilities
for this type of stuff and this was the best I could think of:
One shared keyring for all repos. Under each repository section, we
would have a VerifySignatures option or something similar, which would
take values of "Always", "Optional", or "Never", with one of these as
a sane default. We would fail when set to "Always" if packages had no
signature, we didn't have the signature on the package, or if the
signature was invalid. For optional, we would verify the signature if
it was there and we had it in our keychain; spit a warning otherwise
but continue on. Never seems self explanatory

> My thoughts were to make a option to each repo section in pacman.conf.
> With this option: Keyring = /foo/bar we have an indicator that pacman
> should check for correct signatures and users could have their
> unsigned or self-signed repos additionally.

Ha! We think alike. I actually typed the above before I read this.

-Dan

From jatheendra at gmail.com  Tue Dec 16 12:51:54 2008
From: jatheendra at gmail.com (Jatheendra)
Date: Tue, 16 Dec 2008 23:21:54 +0530
Subject: [pacman-dev] [PATCH] (newgpg) Let pacman specify GnuPG's home
	directory.
In-Reply-To: <449c10960812151419x4e1ce960g6781bd97b76aba75@mail.gmail.com>
References: <20081214190116.VZHO27943.eastrmmtao104.cox.net@eastrmimpo02.cox.net>
	<20081215194744.34be654e@tux1.brauer.lan> <87fxkpb4km.fsf@cox.net>
	<20081215211121.0d1d72ff@tux1.brauer.lan>
	<449c10960812151419x4e1ce960g6781bd97b76aba75@mail.gmail.com>
Message-ID: <e94122a10812160951l44c73763s1861ff34895bf6c8@mail.gmail.com>

A quick question......

I saw this in  be_package.c  pkg_load()

    /* look around for a PGP signature file; load if available */
>         MALLOC(pgpfile, strlen(pkgfile) + 5, RET_ERR(PM_ERR_MEMORY, NULL));
>         sprintf(pgpfile, "%s.sig", pkgfile);
>         if(access(pgpfile, R_OK) == 0) {
>             FILE *f;
>             long bytes;
>             f = fopen(pgpfile, "rb");
>             fseek(f, 0L, SEEK_END);
>             bytes = ftell(f);
>             fseek(f, 0L, SEEK_SET);
>             /* don't read the file in if it is obviously not the size of a
> sig */
>             if(bytes == 72) {
>                 CALLOC(newpkg->pgpsig.rawdata, bytes, sizeof(char),
>                         RET_ERR(PM_ERR_MEMORY, NULL));
>                 fread(newpkg->pgpsig.rawdata, sizeof(char), bytes, f);
>                 newpkg->pgpsig.rawlen = bytes;
>                 _alpm_log(PM_LOG_DEBUG,
>                         "loaded package .sig file, location %s\n",
> pgpfile);
>             } else {
>                 _alpm_log(PM_LOG_WARNING, _("PGP signature file for %s was
> abnormal"
>                             " (had length %ld), skipping\n"), pkgfile,
> bytes);
>             }
>             fclose(f);
>         } else {
>             _alpm_log(PM_LOG_DEBUG, "no package signature file found\n");
>         }
>         FREE(pgpfile);
>


So do we download the signature file along with the package? Or use
%PGPSIG% in the db?



On Tue, Dec 16, 2008 at 3:49 AM, Dan McGee <dpmcgee at gmail.com> wrote:
> On Mon, Dec 15, 2008 at 2:11 PM, Gerhard Brauer <gerbra at archlinux.de>
wrote:
>> Am Mon, 15 Dec 2008 13:50:49 -0600
>> schrieb Chris Brannon <cmbrannon at cox.net>:
>>> I think pacman should at least complain if the signing key is not
>>> found in the public keyring.  Thoughts?
>>
>> IMHO pacman should refuse to install anything from core and extra if
>> the signature is not found or corrupted.
>> I don't know what to to with community (maybe a second keyring with
>> TU signatures?)
>
> Pacman knows nothing about [core], [extra], and [community], so this
> will not be possible. However, I had considered a few possibilities
> for this type of stuff and this was the best I could think of:
> One shared keyring for all repos. Under each repository section, we
> would have a VerifySignatures option or something similar, which would
> take values of "Always", "Optional", or "Never", with one of these as
> a sane default. We would fail when set to "Always" if packages had no
> signature, we didn't have the signature on the package, or if the
> signature was invalid. For optional, we would verify the signature if
> it was there and we had it in our keychain; spit a warning otherwise
> but continue on. Never seems self explanatory
>
>> My thoughts were to make a option to each repo section in pacman.conf.
>> With this option: Keyring = /foo/bar we have an indicator that pacman
>> should check for correct signatures and users could have their
>> unsigned or self-signed repos additionally.
>
> Ha! We think alike. I actually typed the above before I read this.
>
> -Dan
> _______________________________________________
> pacman-dev mailing list
> pacman-dev at archlinux.org
> http://archlinux.org/mailman/listinfo/pacman-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://archlinux.org/pipermail/pacman-dev/attachments/20081216/18ce12bc/attachment-0001.htm>

From shiningxc at gmail.com  Tue Dec 16 13:16:51 2008
From: shiningxc at gmail.com (Xavier)
Date: Tue, 16 Dec 2008 19:16:51 +0100
Subject: [pacman-dev] [PATCH] makepkg: ensure PKGBUILD does not contain
	CRLF characters
In-Reply-To: <449c10960812070635q797cdae3j58ec07eab970d9bc@mail.gmail.com>
References: <1228628001-21945-1-git-send-email-dan@archlinux.org>
	<BAY102-DAV31572D0BFC210DDA5713086FC0@phx.gbl>
	<449c10960812070635q797cdae3j58ec07eab970d9bc@mail.gmail.com>
Message-ID: <91752840812161016v6a5a2ben7e805624762365f9@mail.gmail.com>

from http://linuxreviews.org/beginner/abs-guide/en/x7610.html

#!/bin/bash
# Du.sh: DOS to UNIX text file converter.

E_WRONGARGS=65

if [ -z "$1" ]
then
  echo "Usage: `basename $0` filename-to-convert"
  exit $E_WRONGARGS
fi

NEWFILENAME=$1.unx

CR='\015'  # Carriage return.
           # 015 is octal ASCII code for CR.
           # Lines in a DOS text file end in CR-LF.
           # Lines in a UNIX text file end in LF only.

tr -d $CR < $1 > $NEWFILENAME
# Delete CR's and write to new file.

echo "Original DOS text file is \"$1\"."
echo "Converted UNIX text file is \"$NEWFILENAME\"."

exit 0

based on this, I have two comments :
1) just grepping for \015 seems enough for the detection
2) if we want automatic conversion, tr seems enough. Not sure what
dos2unix adds though. At least simplicity, no need to remember and
type \015. Is there anything else? Otherwise tr looks fine for a
script.

On Sun, Dec 7, 2008 at 3:35 PM, Dan McGee <dpmcgee at gmail.com> wrote:
> On Sun, Dec 7, 2008 at 4:33 AM, Allan McRae <allan at archlinux.org> wrote:
>> Dan McGee wrote:
>>>
>>> Do a simple check before sourcing the file to ensure we are a valid bash
>>> script.
>>>
>>> Signed-off-by: Dan McGee <dan at archlinux.org>
>>> ---
>>>  scripts/makepkg.sh.in |    6 ++++++
>>>  1 files changed, 6 insertions(+), 0 deletions(-)
>>>
>>> diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in
>>> index b889625..179746d 100644
>>> --- a/scripts/makepkg.sh.in
>>> +++ b/scripts/makepkg.sh.in
>>> @@ -1354,6 +1354,12 @@ if [ ! -f "$BUILDSCRIPT" ]; then
>>>                # PKGBUILD passed through a pipe
>>>                BUILDSCRIPT=/dev/stdin
>>>        fi
>>> +else
>>> +       crlftest=$(file $BUILDSCRIPT | grep -F 'CRLF' || true)
>>> +       if [ "$crlftest" != "" ]; then
>>> +               error "$(gettext "%s contains CRLF characters and cannot
>>> be sourced.")" "$BUILDSCRIPT"
>>> +               exit 1
>>> +       fi
>>>  fi
>>>   source "$BUILDSCRIPT"
>>>
>>
>> Do you want to attempt a dos2unix there (with the error message downgraded
>> to a warning) before failing?
>
> I'm not sure everyone would have dos2unix installed, so that makes me
> shy away from this a bit. In addition, I feel like actually modifying
> the PKGBUILD should be a rare case. We do it in the case of VCS
> PKGBUILDs, but that is it, and I'd rather not destroy someone's work.
>
> -Dan
> _______________________________________________
> pacman-dev mailing list
> pacman-dev at archlinux.org
> http://archlinux.org/mailman/listinfo/pacman-dev
>

From sivertb at stud.ntnu.no  Tue Dec 16 14:04:32 2008
From: sivertb at stud.ntnu.no (Sivert Berg)
Date: Tue, 16 Dec 2008 20:04:32 +0100
Subject: [pacman-dev] [PATCH] Fixed some small but critical bugs
In-Reply-To: <1229186134-993-1-git-send-email-sivertb@stud.ntnu.no>
References: <1229186134-993-1-git-send-email-sivertb@stud.ntnu.no>
Message-ID: <1229454272-29776-1-git-send-email-sivertb@stud.ntnu.no>

The not so well tested freeing code turned out to have a few subtle yet
very dangerous bugs. They should now be sorted out, preventing the
database from going corrupt everytime you reuse an old inode.

Signed-off-by: Sivert Berg <sivertb at stud.ntnu.no>
---
 lib/libalpm/be_packed.h |   12 ++++++------
 1 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/lib/libalpm/be_packed.h b/lib/libalpm/be_packed.h
index e118a3a..b2654c7 100644
--- a/lib/libalpm/be_packed.h
+++ b/lib/libalpm/be_packed.h
@@ -195,7 +195,7 @@ int _pack_bitmap_free(group_desc_t *group, int n)
 	int i = n / 32;
 	int j = n % 32;
 
-	group->bitmap[i] &= ~(i << j);
+	group->bitmap[i] &= ~(1 << j);
 	return 0;
 }
 
@@ -217,7 +217,7 @@ int _pack_get_free_block(pack_db_t *db)
 			group->blocks_free--;
 			ret = _pack_check_bitmap(group);
 			ret += i * BLOCKS_PER_GROUP;
-			return ret;;
+			return ret;
 		}
 	}
 
@@ -238,7 +238,7 @@ int _pack_block_free(pack_db_t *db, int block)
 	int g, b;
 
 	g = block / BLOCKS_PER_GROUP;
-	b = block & BLOCKS_PER_GROUP;
+	b = block % BLOCKS_PER_GROUP;
 
 	group = _pack_get_group(db, g);
   group->blocks_free++;
@@ -299,6 +299,7 @@ int _pack_inode_free(pack_db_t *db, int in)
 
 	block = inode->block[6];
 	blocks = _pack_get_block(db, block);
+	inode->block[6] = -1;
 
 	while(blocks != NULL) {
 		for (i = 0; i < size; i++) {
@@ -526,9 +527,8 @@ int _pack_file_puts(pack_db_t *db, pack_file_t *file,
 {
 	size_t i;
 	size_t b_pos = file->pos % BLOCK_SIZE;
-	char *dest = _pack_get_block(db,
-			_pack_inode_add_block(db, file->inode, file->pos / BLOCK_SIZE));
-	int block;
+	int block = _pack_inode_add_block(db, file->inode, file->pos / BLOCK_SIZE);
+	char *dest = _pack_get_block(db, block);
 
 	for (i = 0; i < size; i++, file->pos++, b_pos++)
 	{
-- 
1.6.0.5


From shiningxc at gmail.com  Tue Dec 16 14:18:19 2008
From: shiningxc at gmail.com (Xavier)
Date: Tue, 16 Dec 2008 20:18:19 +0100
Subject: [pacman-dev] [PATCH 1/2] Added a new database backend
In-Reply-To: <1229332855-3569-1-git-send-email-sivertb@stud.ntnu.no>
References: <1229332855-3569-1-git-send-email-sivertb@stud.ntnu.no>
Message-ID: <91752840812161118u340aa632lc657478640b58854@mail.gmail.com>

On Mon, Dec 15, 2008 at 10:20 AM, Sivert Berg <sivertb at stud.ntnu.no> wrote:
> This is an attempt to add a one-file-per-database backend. The main
> reason I wrote this was because load times of the database on my laptop
> when it's not yet cached is around 40-50 seconds. When you just want to
> do a quick "pacman -Ss" that can be a little annoying. I know you have
> discussed the current database vs. one-file earlier, and I think someone
> said another backend could be supported too.
>
> This first patch contains the main database code and a program to
> convert .db.tar.gz and file databases into the new format.
>
> Signed-off-by: Sivert Berg <sivertb at stud.ntnu.no>
> ---

What about the disk space usage, compared to a simple tar file?
Did you ever consider the idea of reading directly tar.gz or
reading/writing directly tar, knowing we already have the libarchive
dependency.

From shiningxc at gmail.com  Tue Dec 16 14:23:14 2008
From: shiningxc at gmail.com (Xavier)
Date: Tue, 16 Dec 2008 20:23:14 +0100
Subject: [pacman-dev] PKGBUILD.proto - Take3
In-Reply-To: <20081208172032.6845854a@judfilm.net>
References: <20081208172032.6845854a@judfilm.net>
Message-ID: <91752840812161123g296c3b46wa5e2b23eed93e12d@mail.gmail.com>

2008/12/8 Jud <jud at judfilm.net>:
> Hi,
>
> Edit: Take3, suggested changes - thanks everyone!
>
> Dan suggested I send this to the pacman-dev list.
>
> After completing some research and asking alot of questions I present
> some minor changes to PKGBUILD.proto supplied as a .diff to be merged
> after your approval. I believe it helps the intended audience create a
> better PKGBUILD in less time according to the latest Arch Packaging
> Standards.
>
> Cheers
> Jud
>
>
> Inline:
> --- PKGBUILD.proto      2008-12-05 23:32:33.000005000 +1000
> +++ PKGBUILD.proto.new  2008-12-08 17:17:15.536302100 +1000
> @@ -4,12 +4,13 @@
>  # then please put 'unknown'.
>
>  # Contributor: Your Name <youremail at domain.com>
> +
>  pkgname=NAME
> -pkgver=VERSION
> +pkgver=VERSION # Note: Cannot contain hyphens

I like the previously suggested idea of avoiding comments in the prototype.
Instead, this belongs to the doc for sure, and if it is already there,
it should be as clear as possible and difficult to miss.

>  pkgrel=1
>  pkgdesc=""
> +url="http://ADDRESS/"
>  arch=()
> -url=""
>  license=('GPL')
>  groups=()
>  depends=()
> @@ -20,17 +21,14 @@
>  replaces=()
>  backup=()
>  options=()
> -install=
> -source=($pkgname-$pkgver.tar.gz)
> +install=$pkgname.install
> +source=(http://ADDRESS/TO/FILE/$pkgname-$pkgver.tar.gz)
>  noextract=()
> -md5sums=() #generate with 'makepkg -g'
> +md5sums=() # Generate with 'makepkg -g'
>
>  build() {
>   cd "$srcdir/$pkgname-$pkgver"
> -
>   ./configure --prefix=/usr
>   make || return 1
> -  make DESTDIR="$pkgdir/" install
> +  make DESTDIR="$pkgdir/" install || return 1
>  }
> -
> -# vim:set ts=2 sw=2 et:
>

The rest looks fine to me now that all other comments have been addressed :)

From louipc.ist at gmail.com  Tue Dec 16 15:52:23 2008
From: louipc.ist at gmail.com (Loui Chang)
Date: Tue, 16 Dec 2008 15:52:23 -0500
Subject: [pacman-dev] PKGBUILD.proto - Take3
In-Reply-To: <91752840812161123g296c3b46wa5e2b23eed93e12d@mail.gmail.com>
References: <20081208172032.6845854a@judfilm.net>
	<91752840812161123g296c3b46wa5e2b23eed93e12d@mail.gmail.com>
Message-ID: <20081216205223.GA23066@lynn>

> 2008/12/8 Jud <jud at judfilm.net>:
> >  pkgrel=1
> >  pkgdesc=""
> > +url="http://ADDRESS/"

What's the point of 'ADDRESS' if you're going to remove/replace that
100% of the time? 'url' should be indicative enough of what should
appear in that field. That's the beauty of PKGBUILDs. They almost need
no documentation.


From jatheendra at gmail.com  Wed Dec 17 07:52:36 2008
From: jatheendra at gmail.com (Jatheendra)
Date: Wed, 17 Dec 2008 18:22:36 +0530
Subject: [pacman-dev] [PATCH] (newgpg) Let pacman specify GnuPG's home
	directory.
In-Reply-To: <449c10960812151419x4e1ce960g6781bd97b76aba75@mail.gmail.com>
References: <20081214190116.VZHO27943.eastrmmtao104.cox.net@eastrmimpo02.cox.net>
	<20081215194744.34be654e@tux1.brauer.lan> <87fxkpb4km.fsf@cox.net>
	<20081215211121.0d1d72ff@tux1.brauer.lan>
	<449c10960812151419x4e1ce960g6781bd97b76aba75@mail.gmail.com>
Message-ID: <e94122a10812170452r38140cc3k879af17f02391eab@mail.gmail.com>

A patch for adding VerifySignature options in pacman.conf


>From cbe0f2ccf64509f6182136bbfa35ec934dd18d2d Mon Sep 17 00:00:00 2001
From: shankar <jatheendra at gmail.com>
Date: Wed, 17 Dec 2008 16:25:07 +0530
Subject: [PATCH] Added gpg verification options per repo to the config file

---
 lib/libalpm/alpm.h  |    9 +++++++++
 lib/libalpm/db.c    |   31 +++++++++++++++++++++++++++++++
 lib/libalpm/db.h    |    2 ++
 src/pacman/pacman.c |   18 ++++++++++++++++++
 4 files changed, 60 insertions(+), 0 deletions(-)

diff --git a/lib/libalpm/alpm.h b/lib/libalpm/alpm.h
index c26b8bb..fedfc12 100644
--- a/lib/libalpm/alpm.h
+++ b/lib/libalpm/alpm.h
@@ -249,6 +249,15 @@ typedef enum _pgpcheck_t {

 pgpcheck_t alpm_pkg_check_pgp_signature(pmpkg_t *pkg);

+/* GPG signature verification option */
+typedef enum _pmdb_verify_gpg {
+	PM_GPG_VERIFY_ALWAYS,
+	PM_GPG_VERIFY_OPTIONAL,
+	PM_GPG_VERIFY_NEVER
+} pmdb_verify_gpg;
+
+
+int alpm_db_set_gpg_opt(pmdb_t *db, pmdb_verify_gpg verify);
 /*
  * Deltas
  */
diff --git a/lib/libalpm/db.c b/lib/libalpm/db.c
index 9b91ce4..2bf03fb 100644
--- a/lib/libalpm/db.c
+++ b/lib/libalpm/db.c
@@ -206,6 +206,37 @@ int SYMEXPORT alpm_db_setserver(pmdb_t *db, const
char *url)

 	return(0);
 }
+/** Set the verify gpg signature option for a database.
+ * @param db database pointer
+ * @param verify enum pmdb_verify_gpg
+ * @return 0 on success, -1 on error (pm_errno is set accordingly)
+ */
+int SYMEXPORT alpm_db_set_gpg_opt(pmdb_t *db, pmdb_verify_gpg verify)
+{
+	alpm_list_t *i;
+	int found = 0;
+
+	ALPM_LOG_FUNC;
+
+	/* Sanity checks */
+	ASSERT(db != NULL, RET_ERR(PM_ERR_DB_NULL, -1));
+
+	for(i = handle->dbs_sync; i && !found; i = i->next) {
+		pmdb_t *sdb = i->data;
+		if(strcmp(db->treename, sdb->treename) == 0) {
+			found = 1;
+		}
+	}
+	if(!found) {
+		RET_ERR(PM_ERR_DB_NOT_FOUND, -1);
+	}
+
+	db->verify_gpg = verify;
+	_alpm_log(PM_LOG_DEBUG, "adding VerifySig option to database '%s': %d\n",
+			db->treename, verify);
+
+	return(0);
+}

 /** Get the name of a package database
  * @param db pointer to the package database
diff --git a/lib/libalpm/db.h b/lib/libalpm/db.h
index 96fac0d..b94ef01 100644
--- a/lib/libalpm/db.h
+++ b/lib/libalpm/db.h
@@ -37,6 +37,7 @@ typedef enum _pmdbinfrq_t {
 	INFRQ_ALL = 0x3F
 } pmdbinfrq_t;

+
 /* Database */
 struct __pmdb_t {
 	char *path;
@@ -45,6 +46,7 @@ struct __pmdb_t {
 	alpm_list_t *pkgcache;
 	alpm_list_t *grpcache;
 	alpm_list_t *servers;
+	pmdb_verify_gpg verify_gpg;
 };

 /* db.c, database general calls */
diff --git a/src/pacman/pacman.c b/src/pacman/pacman.c
index 18fd3a8..0292cfa 100644
--- a/src/pacman/pacman.c
+++ b/src/pacman/pacman.c
@@ -788,6 +788,24 @@ static int _parseconfig(const char *file, const
char *givensection,
 					}

 					free(server);
+				} else if(strcmp(key, "VerifySig") == 0) {
+					if (strcmp(ptr, "Always") == 0) {
+						ret = alpm_db_set_gpg_opt(db,PM_GPG_VERIFY_ALWAYS);
+					} else if (strcmp(ptr, "Optional") == 0) {
+						ret = alpm_db_set_gpg_opt(db,PM_GPG_VERIFY_OPTIONAL);
+					} else if (strcmp(ptr, "Never") == 0) {
+						ret = alpm_db_set_gpg_opt(db,PM_GPG_VERIFY_NEVER);
+					} else {
+						pm_printf(PM_LOG_ERROR, _("invalid value for 'VerifySig' :
'%s'\n"), ptr);
+						ret = 1;
+						goto cleanup;
+					}
+					if ( ret != 0) {
+						pm_printf(PM_LOG_ERROR, _("could not add gpg verify option to
database '%s': %s (%s)\n"),
+								alpm_db_get_name(db), ptr, alpm_strerrorlast());
+						goto cleanup;
+					}
+					pm_printf(PM_LOG_DEBUG, "Verify GPG signature for %s:
%s\n",alpm_db_get_name(db), ptr);
 				} else {
 					pm_printf(PM_LOG_ERROR, _("config file %s, line %d: directive
'%s' not recognized.\n"),
 							file, linenum, key);
-- 
1.6.0.4







On Tue, Dec 16, 2008 at 3:49 AM, Dan McGee <dpmcgee at gmail.com> wrote:
>
> On Mon, Dec 15, 2008 at 2:11 PM, Gerhard Brauer <gerbra at archlinux.de> wrote:
> > Am Mon, 15 Dec 2008 13:50:49 -0600
> > schrieb Chris Brannon <cmbrannon at cox.net>:
> >> I think pacman should at least complain if the signing key is not
> >> found in the public keyring.  Thoughts?
> >
> > IMHO pacman should refuse to install anything from core and extra if
> > the signature is not found or corrupted.
> > I don't know what to to with community (maybe a second keyring with
> > TU signatures?)
>
> Pacman knows nothing about [core], [extra], and [community], so this
> will not be possible. However, I had considered a few possibilities
> for this type of stuff and this was the best I could think of:
> One shared keyring for all repos. Under each repository section, we
> would have a VerifySignatures option or something similar, which would
> take values of "Always", "Optional", or "Never", with one of these as
> a sane default. We would fail when set to "Always" if packages had no
> signature, we didn't have the signature on the package, or if the
> signature was invalid. For optional, we would verify the signature if
> it was there and we had it in our keychain; spit a warning otherwise
> but continue on. Never seems self explanatory
>
> > My thoughts were to make a option to each repo section in pacman.conf.
> > With this option: Keyring = /foo/bar we have an indicator that pacman
> > should check for correct signatures and users could have their
> > unsigned or self-signed repos additionally.
>
> Ha! We think alike. I actually typed the above before I read this.
>
> -Dan
> _______________________________________________
> pacman-dev mailing list
> pacman-dev at archlinux.org
> http://archlinux.org/mailman/listinfo/pacman-dev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0014-Added-gpg-verification-options-per-repo-to-the-confi.patch
Type: application/octet-stream
Size: 3680 bytes
Desc: not available
URL: <http://archlinux.org/pipermail/pacman-dev/attachments/20081217/ce386dbc/attachment-0001.obj>

From jatheendra at gmail.com  Wed Dec 17 10:40:45 2008
From: jatheendra at gmail.com (Jatheendra)
Date: Wed, 17 Dec 2008 21:10:45 +0530
Subject: [pacman-dev] [patch]GPG verification
Message-ID: <e94122a10812170740p2a1f18b8m2a05659bd7b9fc2a@mail.gmail.com>

These patches will add VerifySig option to pacman.conf. VerifySig
takes options Always, Optional or Never

[repo-name]
      Server = ServerName
      VerifySig = Always
      Include = IncludePath





>From 77be2c5cbfa3c7a750fe46d115c23096d2cf51e5 Mon Sep 17 00:00:00 2001
From: shankar <jatheendra at gmail.com>
Date: Wed, 17 Dec 2008 20:52:21 +0530
Subject: [PATCH] changed gpg verification logic

Signed-off-by: shankar <jatheendra at gmail.com>
---
 lib/libalpm/signing.c |    3 +++
 lib/libalpm/sync.c    |   26 ++++++++++++++++++++++----
 2 files changed, 25 insertions(+), 4 deletions(-)

diff --git a/lib/libalpm/signing.c b/lib/libalpm/signing.c
index ddb89bc..0835b5e 100644
--- a/lib/libalpm/signing.c
+++ b/lib/libalpm/signing.c
@@ -166,6 +166,9 @@ pgpcheck_t _alpm_gpgme_checksig(const char
*pkgpath, const pmpgpsig_t *sig)

 	if(gpgsig->summary & GPGME_SIGSUM_VALID) {
 		/* good signature, continue */
+		ret = PM_PGP_SIG_VALID;
+		_alpm_log(PM_LOG_DEBUG, _("Package %s has a valid signature.\n"),
+				pkgpath);
 	} else if(gpgsig->summary & GPGME_SIGSUM_GREEN) {
 		/* 'green' signature, not sure what to do here */
 		_alpm_log(PM_LOG_WARNING, _("Package %s has a green signature.\n"),
diff --git a/lib/libalpm/sync.c b/lib/libalpm/sync.c
index 24f2b98..f658ae2 100644
--- a/lib/libalpm/sync.c
+++ b/lib/libalpm/sync.c
@@ -901,12 +901,30 @@ int _alpm_sync_commit(pmtrans_t *trans, pmdb_t
*db_local, alpm_list_t **data)
 			*data = alpm_list_add(*data, strdup(filename));
 		}
 		/* check PGP signature next */
-		if(_alpm_gpgme_checksig(filepath, pgpsig) == PM_PGP_SIG_INVALID) {
-			errors++;
-			*data = alpm_list_add(*data, strdup(filename));
+		pmdb_t *sdb = alpm_pkg_get_db(spkg);
+
+		if(sdb->verify_gpg == PM_GPG_VERIFY_ALWAYS) {
+			if(_alpm_gpgme_checksig(filepath, pgpsig) != PM_PGP_SIG_VALID) {
+				errors++;
+				*data = alpm_list_add(*data, strdup(filename));
+				_alpm_log(PM_LOG_ERROR, _("Invalid GPG signature on package:
%s\n"),alpm_pkg_get_name(spkg));
+			}
+			FREE(filepath);
+		} else if (sdb->verify_gpg == PM_GPG_VERIFY_OPTIONAL) {
+			pgpcheck_t ret1 = _alpm_gpgme_checksig(filepath, pgpsig);
+
+			if(ret1  == PM_PGP_SIG_MISSING) {
+				/*no problems here*/
+			} else if (ret1 != PM_PGP_SIG_VALID) {
+				errors++;
+				*data = alpm_list_add(*data, strdup(filename));
+				_alpm_log(PM_LOG_ERROR, _("Invalid GPG signature on package:
%s\n"),alpm_pkg_get_name(spkg));
+			}
+			FREE(filepath);
 		}
-		FREE(filepath);
 	}
+
+
 	if(errors) {
 		pm_errno = PM_ERR_PKG_INVALID;
 		goto error;
-- 
1.6.0.4
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Added-gpg-verification-options-per-repo-to-the-confi.patch
Type: application/octet-stream
Size: 3680 bytes
Desc: not available
URL: <http://archlinux.org/pipermail/pacman-dev/attachments/20081217/e8ec6c2c/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-changed-gpg-verification-logic.patch
Type: application/octet-stream
Size: 2341 bytes
Desc: not available
URL: <http://archlinux.org/pipermail/pacman-dev/attachments/20081217/e8ec6c2c/attachment-0001.obj>

From sivertb at stud.ntnu.no  Thu Dec 18 06:15:42 2008
From: sivertb at stud.ntnu.no (Sivert Berg)
Date: Thu, 18 Dec 2008 12:15:42 +0100
Subject: [pacman-dev] [PATCH 1/2] Added a new database backend
In-Reply-To: <91752840812161118u340aa632lc657478640b58854@mail.gmail.com>
References: <1229332855-3569-1-git-send-email-sivertb@stud.ntnu.no>
	<91752840812161118u340aa632lc657478640b58854@mail.gmail.com>
Message-ID: <20081218121542.dbd25e9f.sivertb@stud.ntnu.no>

On Tue, 16 Dec 2008 20:18:19 +0100
Xavier <shiningxc at gmail.com> wrote:

> What about the disk space usage, compared to a simple tar file?
> Did you ever consider the idea of reading directly tar.gz or
> reading/writing directly tar, knowing we already have the libarchive
> dependency.

Well, if I understand tar files correctly, tar files require files to be
stored in one continous chunk. If you want to update a file, and it
gets bigger than it already is, you would have to move the whole file
to the end of the archive. So while tar archives would work great for
the sync db's that doesn't get modified, it would be hard to do for the
local db, unless you rewrote the complete database every time you
modified it. Now keeping two separate database formats for the sync
and the local database seems a bit weird to me. As for the size:
extra.db.tar:      5.3 mb
extra.db (packed): 5.0 mb
This is probably because the packed format uses 256 byte blocks, while
tar uses 512 byte blocks, leading to less waste on small files in the
packed format.

-- 
Sivert Berg <sivertb at stud.ntnu.no>

From shiningxc at gmail.com  Thu Dec 18 07:10:07 2008
From: shiningxc at gmail.com (Xavier)
Date: Thu, 18 Dec 2008 13:10:07 +0100
Subject: [pacman-dev] [PATCH 1/2] Added a new database backend
In-Reply-To: <20081218121542.dbd25e9f.sivertb@stud.ntnu.no>
References: <1229332855-3569-1-git-send-email-sivertb@stud.ntnu.no>
	<91752840812161118u340aa632lc657478640b58854@mail.gmail.com>
	<20081218121542.dbd25e9f.sivertb@stud.ntnu.no>
Message-ID: <91752840812180410l30551cdat5e926b843e9cafe6@mail.gmail.com>

On Thu, Dec 18, 2008 at 12:15 PM, Sivert Berg <sivertb at stud.ntnu.no> wrote:
> On Tue, 16 Dec 2008 20:18:19 +0100
> Xavier <shiningxc at gmail.com> wrote:
>
>> What about the disk space usage, compared to a simple tar file?
>> Did you ever consider the idea of reading directly tar.gz or
>> reading/writing directly tar, knowing we already have the libarchive
>> dependency.
>
> Well, if I understand tar files correctly, tar files require files to be
> stored in one continous chunk. If you want to update a file, and it
> gets bigger than it already is, you would have to move the whole file
> to the end of the archive. So while tar archives would work great for
> the sync db's that doesn't get modified, it would be hard to do for the
> local db, unless you rewrote the complete database every time you
> modified it. Now keeping two separate database formats for the sync
> and the local database seems a bit weird to me. As for the size:
> extra.db.tar:      5.3 mb
> extra.db (packed): 5.0 mb
> This is probably because the packed format uses 256 byte blocks, while
> tar uses 512 byte blocks, leading to less waste on small files in the
> packed format.
>

Hm ok, I don't know much about the subject, but I would like to get
idea of the "code complexity" / "performance gain" ratio compared to
the other ideas or existing solutions.
Another one I have in mind is the ext2 on loopback fs way :
http://bbs.archlinux.org/viewtopic.php?id=20385
Isn't the result a bit similar, but instead of having to implement it
ourself in pacman, we can just re-use stable and working kernel
filesystems?
Doing it on pacman level reduces the layers so could increase the
performance, and also probably allows a better control on flexibility,
but I still wonder if it is worth it.
Again I don't know much, so correct me if I am wrong and enlighten me :)

From sivertb at stud.ntnu.no  Thu Dec 18 08:02:27 2008
From: sivertb at stud.ntnu.no (Sivert Berg)
Date: Thu, 18 Dec 2008 14:02:27 +0100
Subject: [pacman-dev] [PATCH 1/2] Added a new database backend
In-Reply-To: <91752840812180410l30551cdat5e926b843e9cafe6@mail.gmail.com>
References: <1229332855-3569-1-git-send-email-sivertb@stud.ntnu.no>
	<91752840812161118u340aa632lc657478640b58854@mail.gmail.com>
	<20081218121542.dbd25e9f.sivertb@stud.ntnu.no>
	<91752840812180410l30551cdat5e926b843e9cafe6@mail.gmail.com>
Message-ID: <20081218140227.d2f5f864.sivertb@stud.ntnu.no>

On Thu, 18 Dec 2008 13:10:07 +0100
Xavier <shiningxc at gmail.com> wrote:
> Hm ok, I don't know much about the subject, but I would like to get
> idea of the "code complexity" / "performance gain" ratio compared to
> the other ideas or existing solutions.
> Another one I have in mind is the ext2 on loopback fs way :
> http://bbs.archlinux.org/viewtopic.php?id=20385
> Isn't the result a bit similar, but instead of having to implement it
> ourself in pacman, we can just re-use stable and working kernel
> filesystems?

Well, I guess that's a possibility. However you can't resize loopback
devices just like that. So either you have to make it so big you'll
never need to resize it, or when you need to resize, make a
new loopback device, make a filesystem on it, copy over the old database
and then start using the new loopback device. Is that really so much
easier than having the "fs" code in pacman? If you look at
be_packed.h (which is where all the database code is) you see it's only
about 750 lines and it shouldn't be that hard to make that into good,
stable code.

> Doing it on pacman level reduces the layers so could increase the
> performance, and also probably allows a better control on flexibility,
> but I still wonder if it is worth it.
> Again I don't know much, so correct me if I am wrong and enlighten me :)

In my opinion anything that is able to cut down the runtime of "pacman
-Ss" from 50 to 2 seconds on a database that isn't cached yet is
worth it. Might just be me being impatient though :)

-- 
Sivert Berg <sivertb at stud.ntnu.no>

From gerbra at archlinux.de  Thu Dec 18 08:02:48 2008
From: gerbra at archlinux.de (Gerhard Brauer)
Date: Thu, 18 Dec 2008 14:02:48 +0100
Subject: [pacman-dev] [PATCH] (newgpg) Let pacman specify GnuPG's home
 directory.
In-Reply-To: <e94122a10812170452r38140cc3k879af17f02391eab@mail.gmail.com>
References: <20081214190116.VZHO27943.eastrmmtao104.cox.net@eastrmimpo02.cox.net>
	<20081215194744.34be654e@tux1.brauer.lan> <87fxkpb4km.fsf@cox.net>
	<20081215211121.0d1d72ff@tux1.brauer.lan>
	<449c10960812151419x4e1ce960g6781bd97b76aba75@mail.gmail.com>
	<e94122a10812170452r38140cc3k879af17f02391eab@mail.gmail.com>
Message-ID: <20081218140248.0d6a6886@tux1.brauer.lan>

Am Wed, 17 Dec 2008 18:22:36 +0530
schrieb Jatheendra <jatheendra at gmail.com>:

> A patch for adding VerifySignature options in pacman.conf

From your other mail:
------------
These patches will add VerifySig option to pacman.conf. VerifySig
takes options Always, Optional or Never

[repo-name]
      Server = ServerName
      VerifySig = Always
      Include = IncludePath
------------

I've not tested your patch (today evening maybe), but i am not very
happy with this triple state. If i choose to use a repo which offers
signed packages then i want the "full program", so if something wrong
with one package i don't want it get installed/upgraded.
And if i have a repo without signing then i don't put the option in the
repo section of pacman.conf.

I found my suggestion has mor advantages, ex.:
------------
[core]
Keyring = /etc/pacman.d/gpg/devel.gpg
...

[community]
Keyring = /etc/pacman.d/gpg/community.gpg
...

[myrepo]
...

[yaourt]
Keyring = /etc/pacman.d/gpg/fr-repo.gpg
...
------------

a) The Keyring= Option indicates pacman if the signing framework should
be used

b) This var signals pacman where to find the public keyring for this
repo. AND we could have different keyrings for repos.
Ex.: the TU (if community packages get signed) fluctuation is IMHO
bigger than on the Developers side. So keyring updates are more often
necassary on community/TU side. And myself find it better to have the
TUs signatures/trustlevel not in the same keyring like developers
(core,extra) keyring for package signing.

c) With this var a extern repo (ex. the france yaourt repo) could
offers also signed packages - and a properly public keyring.

I think a little further (currently in the phase were we not finished
with programming to maybe avoid false paradigms):
- How get the users the keyrings, how get they updated?
- How is the trustdb(Trustlevel) handled? Must/should each user first
  sign each public key in the keyrings (Urghhh!)? Or is it better to
  deliver both signing framework files for a repo (ex. core): devel.gpg
  (=the keyring with public keys) and develdb-gpg (=the trustdb file).
  This trustdb file could could be the arch-web-of-trust where the arch
  leader for .ex. signes all devel keys, so they are trusted for this
  keyring pair.

From gerbra at archlinux.de  Thu Dec 18 08:09:52 2008
From: gerbra at archlinux.de (Gerhard Brauer)
Date: Thu, 18 Dec 2008 14:09:52 +0100
Subject: [pacman-dev] [PATCH] (newgpg) Let pacman specify GnuPG's home
 directory.
In-Reply-To: <20081218140248.0d6a6886@tux1.brauer.lan>
References: <20081214190116.VZHO27943.eastrmmtao104.cox.net@eastrmimpo02.cox.net>
	<20081215194744.34be654e@tux1.brauer.lan> <87fxkpb4km.fsf@cox.net>
	<20081215211121.0d1d72ff@tux1.brauer.lan>
	<449c10960812151419x4e1ce960g6781bd97b76aba75@mail.gmail.com>
	<e94122a10812170452r38140cc3k879af17f02391eab@mail.gmail.com>
	<20081218140248.0d6a6886@tux1.brauer.lan>
Message-ID: <20081218140952.1160abc6@tux1.brauer.lan>

Am Thu, 18 Dec 2008 14:02:48 +0100
schrieb Gerhard Brauer <gerbra at archlinux.de>:

Sorry, bad key stroke send this mail before finished...

> trusted for this keyring pair.

Last sentence:
If we - you developers ;-) - discuss this early enough then the
necessary code changes could be smarter...

Regards
	Gerhard

From cmbrannon at cox.net  Thu Dec 18 08:36:06 2008
From: cmbrannon at cox.net (Chris Brannon)
Date: Thu, 18 Dec 2008 07:36:06 -0600
Subject: [pacman-dev] [PATCH] (newgpg) Let pacman specify GnuPG's home
	directory.
In-Reply-To: <20081218140248.0d6a6886@tux1.brauer.lan> (Gerhard Brauer's
	message of "Thu\, 18 Dec 2008 14\:02\:48 +0100")
References: <20081214190116.VZHO27943.eastrmmtao104.cox.net@eastrmimpo02.cox.net>
	<20081215194744.34be654e@tux1.brauer.lan> <87fxkpb4km.fsf@cox.net>
	<20081215211121.0d1d72ff@tux1.brauer.lan>
	<449c10960812151419x4e1ce960g6781bd97b76aba75@mail.gmail.com>
	<e94122a10812170452r38140cc3k879af17f02391eab@mail.gmail.com>
	<20081218140248.0d6a6886@tux1.brauer.lan>
Message-ID: <87bpv9ip15.fsf@cox.net>

Gerhard Brauer <gerbra at archlinux.de> writes:

> a) The Keyring= Option indicates pacman if the signing framework should
> be used
>
> b) This var signals pacman where to find the public keyring for this
> repo. AND we could have different keyrings for repos.
> Ex.: the TU (if community packages get signed) fluctuation is IMHO
> bigger than on the Developers side. So keyring updates are more often
> necassary on community/TU side. And myself find it better to have the
> TUs signatures/trustlevel not in the same keyring like developers
> (core,extra) keyring for package signing.
>
> c) With this var a extern repo (ex. the france yaourt repo) could
> offers also signed packages - and a properly public keyring.

If I understand gpgme correctly, you can't just tell it to use a public
keyring from a given file.  This applies to the gpg binary as well.
GnuPG's paradigm is one of home directories.  You specify a GnuPG home
directory, such as ~/.gnupg or /etc/pacman.d/gnupg, and it looks for
pubring.gpg and other necessary files in that place.

One possibility is to allow overriding of GPGDir on a per-repo basis.

Regards,
-- Chris

From gerbra at archlinux.de  Thu Dec 18 09:16:35 2008
From: gerbra at archlinux.de (Gerhard Brauer)
Date: Thu, 18 Dec 2008 15:16:35 +0100
Subject: [pacman-dev] [PATCH] (newgpg) Let pacman specify GnuPG's home
 directory.
In-Reply-To: <87bpv9ip15.fsf@cox.net>
References: <20081214190116.VZHO27943.eastrmmtao104.cox.net@eastrmimpo02.cox.net>
	<20081215194744.34be654e@tux1.brauer.lan> <87fxkpb4km.fsf@cox.net>
	<20081215211121.0d1d72ff@tux1.brauer.lan>
	<449c10960812151419x4e1ce960g6781bd97b76aba75@mail.gmail.com>
	<e94122a10812170452r38140cc3k879af17f02391eab@mail.gmail.com>
	<20081218140248.0d6a6886@tux1.brauer.lan> <87bpv9ip15.fsf@cox.net>
Message-ID: <20081218151635.6259ac14@tux1.brauer.lan>

Am Thu, 18 Dec 2008 07:36:06 -0600
schrieb Chris Brannon <cmbrannon at cox.net>:

> Gerhard Brauer <gerbra at archlinux.de> writes:
> 
> > c) With this var a extern repo (ex. the france yaourt repo) could
> > offers also signed packages - and a properly public keyring.
> 
> If I understand gpgme correctly, you can't just tell it to use a
> public keyring from a given file.  This applies to the gpg binary as
> well. 

Hmm, gpg manpage list the options:
--trustdb-name file
and
--keyring file
and fore general place the  --homedir dir option. gpgme have not a
similar function or any possibility to use/set these?

> One possibility is to allow overriding of GPGDir on a per-repo basis.

> Regards,
> -- Chris

Gerhard

From aaronmgriffin at gmail.com  Thu Dec 18 11:22:25 2008
From: aaronmgriffin at gmail.com (Aaron Griffin)
Date: Thu, 18 Dec 2008 10:22:25 -0600
Subject: [pacman-dev] [PATCH] (newgpg) Let pacman specify GnuPG's home
	directory.
In-Reply-To: <20081218140248.0d6a6886@tux1.brauer.lan>
References: <20081214190116.VZHO27943.eastrmmtao104.cox.net@eastrmimpo02.cox.net>
	<20081215194744.34be654e@tux1.brauer.lan> <87fxkpb4km.fsf@cox.net>
	<20081215211121.0d1d72ff@tux1.brauer.lan>
	<449c10960812151419x4e1ce960g6781bd97b76aba75@mail.gmail.com>
	<e94122a10812170452r38140cc3k879af17f02391eab@mail.gmail.com>
	<20081218140248.0d6a6886@tux1.brauer.lan>
Message-ID: <d64a48980812180822g6a51000euf34fb7373e63af31@mail.gmail.com>

On Thu, Dec 18, 2008 at 7:02 AM, Gerhard Brauer <gerbra at archlinux.de> wrote:
> Am Wed, 17 Dec 2008 18:22:36 +0530
> schrieb Jatheendra <jatheendra at gmail.com>:
>
>> A patch for adding VerifySignature options in pacman.conf
>
> >From your other mail:
> ------------
> These patches will add VerifySig option to pacman.conf. VerifySig
> takes options Always, Optional or Never
>
> [repo-name]
>      Server = ServerName
>      VerifySig = Always
>      Include = IncludePath
> ------------
>
> I've not tested your patch (today evening maybe), but i am not very
> happy with this triple state. If i choose to use a repo which offers
> signed packages then i want the "full program", so if something wrong
> with one package i don't want it get installed/upgraded.
> And if i have a repo without signing then i don't put the option in the
> repo section of pacman.conf.

I think "Optional" makes sense in some cases. Let's take the community
repo, where things tend to be a hodge-podge of ideas and attitudes. I
can imagine half the packages being signed, some being unsigned, and
some being signed by keys not in the keyring.

That is an edge case though...

From pierre at archlinux.de  Thu Dec 18 11:42:13 2008
From: pierre at archlinux.de (Pierre Schmitz)
Date: Thu, 18 Dec 2008 17:42:13 +0100
Subject: [pacman-dev] [PATCH] (newgpg) Let pacman specify GnuPG's home
	directory.
In-Reply-To: <d64a48980812180822g6a51000euf34fb7373e63af31@mail.gmail.com>
References: <20081214190116.VZHO27943.eastrmmtao104.cox.net@eastrmimpo02.cox.net>
	<20081218140248.0d6a6886@tux1.brauer.lan>
	<d64a48980812180822g6a51000euf34fb7373e63af31@mail.gmail.com>
Message-ID: <200812181742.13635.pierre@archlinux.de>

Am Donnerstag 18 Dezember 2008 17:22:25 schrieb Aaron Griffin:
> I think "Optional" makes sense in some cases. Let's take the community
> repo, where things tend to be a hodge-podge of ideas and attitudes. I
> can imagine half the packages being signed, some being unsigned, and
> some being signed by keys not in the keyring.

Well, if that will be the case we can forget about the whole signing stuff. 
One "unprotected" package is enough to inject your custom code.

-- 

Pierre Schmitz


Clemens-August-Stra?e 76
53115 Bonn

Telefon		0228 9716608
Mobil		0160 95269831
Jabber		pierre at jabber.archlinux.de
WWW		http://www.archlinux.de


From aaronmgriffin at gmail.com  Thu Dec 18 12:01:52 2008
From: aaronmgriffin at gmail.com (Aaron Griffin)
Date: Thu, 18 Dec 2008 11:01:52 -0600
Subject: [pacman-dev] [PATCH] (newgpg) Let pacman specify GnuPG's home
	directory.
In-Reply-To: <200812181742.13635.pierre@archlinux.de>
References: <20081214190116.VZHO27943.eastrmmtao104.cox.net@eastrmimpo02.cox.net>
	<20081218140248.0d6a6886@tux1.brauer.lan>
	<d64a48980812180822g6a51000euf34fb7373e63af31@mail.gmail.com>
	<200812181742.13635.pierre@archlinux.de>
Message-ID: <d64a48980812180901p7430fb0ft2b52060304d4b16@mail.gmail.com>

On Thu, Dec 18, 2008 at 10:42 AM, Pierre Schmitz <pierre at archlinux.de> wrote:
> Am Donnerstag 18 Dezember 2008 17:22:25 schrieb Aaron Griffin:
>> I think "Optional" makes sense in some cases. Let's take the community
>> repo, where things tend to be a hodge-podge of ideas and attitudes. I
>> can imagine half the packages being signed, some being unsigned, and
>> some being signed by keys not in the keyring.
>
> Well, if that will be the case we can forget about the whole signing stuff.
> One "unprotected" package is enough to inject your custom code.

Right, but that's not what I'm saying. As a user, I might not care.
Actually, I don't. Here's our cases:

People who care about super-secure packages: Set things to "Always"
and then your system will only install signed packages

Middle of the road people: Set core and extra to "Always" and other
repos to either "Never" or "Optional".

People who don't care: Everything is set to "Never".

See, I fall in the middle case. I'd love to have everything signed,
but I know it won't happen for everything all the time. So, if I set
community to "Always", I'm going to run into a case where I want to
install a package from community that is unsigned. We need a "fuck it,
install it anyway" case.

Now, instead of the "Optional" setting, if there was a
--skip-signature flag that I could use, I would also be sated. Either
way, I'd just like to see a case where I can force it to skip the
signature check.

From gerbra at archlinux.de  Thu Dec 18 12:05:13 2008
From: gerbra at archlinux.de (Gerhard Brauer)
Date: Thu, 18 Dec 2008 18:05:13 +0100
Subject: [pacman-dev] [PATCH] (newgpg) Let pacman specify GnuPG's home
 directory.
In-Reply-To: <d64a48980812180822g6a51000euf34fb7373e63af31@mail.gmail.com>
References: <20081214190116.VZHO27943.eastrmmtao104.cox.net@eastrmimpo02.cox.net>
	<20081215194744.34be654e@tux1.brauer.lan> <87fxkpb4km.fsf@cox.net>
	<20081215211121.0d1d72ff@tux1.brauer.lan>
	<449c10960812151419x4e1ce960g6781bd97b76aba75@mail.gmail.com>
	<e94122a10812170452r38140cc3k879af17f02391eab@mail.gmail.com>
	<20081218140248.0d6a6886@tux1.brauer.lan>
	<d64a48980812180822g6a51000euf34fb7373e63af31@mail.gmail.com>
Message-ID: <20081218180513.6f2427e3@tux1.brauer.lan>

Am Thu, 18 Dec 2008 10:22:25 -0600
schrieb "Aaron Griffin" <aaronmgriffin at gmail.com>:

> On Thu, Dec 18, 2008 at 7:02 AM, Gerhard Brauer <gerbra at archlinux.de>
> wrote:
> > I've not tested your patch (today evening maybe), but i am not very
> > happy with this triple state. If i choose to use a repo which offers
> > signed packages then i want the "full program", so if something
> > wrong with one package i don't want it get installed/upgraded.
> > And if i have a repo without signing then i don't put the option in
> > the repo section of pacman.conf.
> 
> I think "Optional" makes sense in some cases. Let's take the community
> repo, where things tend to be a hodge-podge of ideas and attitudes. I
> can imagine half the packages being signed, some being unsigned, and
> some being signed by keys not in the keyring.

If we wan't this (or if the [community] situation lead us to MUST wan't
this) then such a "Optional" is ok IMHO (with a pacman warning...)

But the question (also the TU's must discuss this) is: why not also
sign community packages per policy?
- The build tools (where signing is integrated) are both the same for
  developers and TUs.
- Also the TU's have the same interest like the devs that their
  packages are on good integrity when they leaves their machines and
  that this could be checked before the users installed these. (we make
  this signing thing not for funny ;-)
- Only TU's(and Devs) putting packages in community, so the count of
  needed keys is not soo much. But TUs come and go more often than
  Devs, so a separate keyring makes sense IMHO cause it changes often.

So if we find a solution to avoid this "Optional" setting, that would be
better IMHO.

(To clear: I'm neither a Dev nor a TU (and badly also not a C
programmer...), my suggestions comes most from the "user side" of this
theme... But i like always to say: "We" <g>)

Regards
	Gerhard

From pierre at archlinux.de  Thu Dec 18 12:15:04 2008
From: pierre at archlinux.de (Pierre Schmitz)
Date: Thu, 18 Dec 2008 18:15:04 +0100
Subject: [pacman-dev] [PATCH] (newgpg) Let pacman specify GnuPG's home
	directory.
In-Reply-To: <d64a48980812180901p7430fb0ft2b52060304d4b16@mail.gmail.com>
References: <20081214190116.VZHO27943.eastrmmtao104.cox.net@eastrmimpo02.cox.net>
	<200812181742.13635.pierre@archlinux.de>
	<d64a48980812180901p7430fb0ft2b52060304d4b16@mail.gmail.com>
Message-ID: <200812181815.04788.pierre@archlinux.de>

Am Donnerstag 18 Dezember 2008 18:01:52 schrieb Aaron Griffin:
> Right, but that's not what I'm saying. As a user, I might not care.
> Actually, I don't. Here's our cases:

OK, I got the point. I think we should forget about what I have said. From 
pacman's point of view those options might make sense. 

When we really start using this we should discuss about our sigingn policy 
(meaning if we accept unsigned packages or not)

-- 

Pierre Schmitz


Clemens-August-Stra?e 76
53115 Bonn

Telefon		0228 9716608
Mobil		0160 95269831
Jabber		pierre at jabber.archlinux.de
WWW		http://www.archlinux.de


From belanger at ASTRO.UMontreal.CA  Fri Dec 19 00:49:49 2008
From: belanger at ASTRO.UMontreal.CA (=?ISO-8859-1?Q?Eric_B=E9langer?=)
Date: Fri, 19 Dec 2008 00:49:49 -0500 (EST)
Subject: [pacman-dev] PKGBUILD.proto - Take3
In-Reply-To: <20081216205223.GA23066@lynn>
References: <20081208172032.6845854a@judfilm.net>
	<91752840812161123g296c3b46wa5e2b23eed93e12d@mail.gmail.com>
	<20081216205223.GA23066@lynn>
Message-ID: <Pine.LNX.4.58.0812190049260.6963@mars>

On Tue, 16 Dec 2008, Loui Chang wrote:

> > 2008/12/8 Jud <jud at judfilm.net>:
> > >  pkgrel=1
> > >  pkgdesc=""
> > > +url="http://ADDRESS/"
>
> What's the point of 'ADDRESS' if you're going to remove/replace that
> 100% of the time? 'url' should be indicative enough of what should
> appear in that field. That's the beauty of PKGBUILDs. They almost need
> no documentation.
>

Agree. Keep the url field empty like it is currently so we can just paste
the url address in .




-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From allan at archlinux.org  Sun Dec 21 02:13:44 2008
From: allan at archlinux.org (Allan McRae)
Date: Sun, 21 Dec 2008 17:13:44 +1000
Subject: [pacman-dev] [PATCH] makepkg: Add docmunetation for separate
	pacakge function
Message-ID: <1229843624-26306-1-git-send-email-allan@archlinux.org>

Signed-off-by: Allan McRae <allan at archlinux.org>
---
 doc/PKGBUILD.5.txt |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)

diff --git a/doc/PKGBUILD.5.txt b/doc/PKGBUILD.5.txt
index e42a8b8..5171798 100644
--- a/doc/PKGBUILD.5.txt
+++ b/doc/PKGBUILD.5.txt
@@ -254,6 +254,15 @@ If you create any variables of your own in the build function, it is
 recommended to use the bash `local` keyword to scope the variable to inside
 the build function.
 
+package() Function
+------------------
+An optional package() function can be specified in addtion to the build() function.
+This function is directly sourced by makepkg and run immediately after the build()
+function. When specified in combination with the fakeroot BUILDENV option in
+linkman:makepkg.conf[5], fakeroot usage will be limited to running the package()
+function and the subsequent packing of the package. The build() function will be
+run as the user calling makepkg.
+
 Install/Upgrade/Remove Scripting
 --------------------------------
 Pacman has the ability to store and execute a package-specific script when it
-- 
1.6.0.5


From allan at archlinux.org  Sun Dec 21 02:15:39 2008
From: allan at archlinux.org (Allan McRae)
Date: Sun, 21 Dec 2008 17:15:39 +1000
Subject: [pacman-dev] [PATCH] makepkg: Add docmunetation for separate
 pacakge function
In-Reply-To: <1229843624-26306-1-git-send-email-allan@archlinux.org>
References: <1229843624-26306-1-git-send-email-allan@archlinux.org>
Message-ID: <BAY102-DAV1162EB8F218E893CD93BD686EE0@phx.gbl>

I'd like some comments here as I find it hard writing documentation when 
I know exactly what the code is doing.  Any suggestions for wording or 
parts which need clarification are appreciated.

Allan

Allan McRae wrote:
> Signed-off-by: Allan McRae <allan at archlinux.org>
> ---
>  doc/PKGBUILD.5.txt |    9 +++++++++
>  1 files changed, 9 insertions(+), 0 deletions(-)
>
> diff --git a/doc/PKGBUILD.5.txt b/doc/PKGBUILD.5.txt
> index e42a8b8..5171798 100644
> --- a/doc/PKGBUILD.5.txt
> +++ b/doc/PKGBUILD.5.txt
> @@ -254,6 +254,15 @@ If you create any variables of your own in the build function, it is
>  recommended to use the bash `local` keyword to scope the variable to inside
>  the build function.
>  
> +package() Function
> +------------------
> +An optional package() function can be specified in addtion to the build() function.
> +This function is directly sourced by makepkg and run immediately after the build()
> +function. When specified in combination with the fakeroot BUILDENV option in
> +linkman:makepkg.conf[5], fakeroot usage will be limited to running the package()
> +function and the subsequent packing of the package. The build() function will be
> +run as the user calling makepkg.
> +
>  Install/Upgrade/Remove Scripting
>  --------------------------------
>  Pacman has the ability to store and execute a package-specific script when it
>   




From shiningxc at gmail.com  Sun Dec 21 03:59:26 2008
From: shiningxc at gmail.com (Xavier)
Date: Sun, 21 Dec 2008 09:59:26 +0100
Subject: [pacman-dev] [PATCH] makepkg: Add docmunetation for separate
	pacakge function
In-Reply-To: <1229843624-26306-1-git-send-email-allan@archlinux.org>
References: <1229843624-26306-1-git-send-email-allan@archlinux.org>
Message-ID: <91752840812210059t52e611a6rdeb87396cf06ba7c@mail.gmail.com>

makepkg: Add docmunetation for separate pacakge function
->
makepkg: Add documentation for the package function

On Sun, Dec 21, 2008 at 8:13 AM, Allan McRae <allan at archlinux.org> wrote:
> Signed-off-by: Allan McRae <allan at archlinux.org>
> ---
>  doc/PKGBUILD.5.txt |    9 +++++++++
>  1 files changed, 9 insertions(+), 0 deletions(-)
>
> diff --git a/doc/PKGBUILD.5.txt b/doc/PKGBUILD.5.txt
> index e42a8b8..5171798 100644
> --- a/doc/PKGBUILD.5.txt
> +++ b/doc/PKGBUILD.5.txt
> @@ -254,6 +254,15 @@ If you create any variables of your own in the build function, it is
>  recommended to use the bash `local` keyword to scope the variable to inside
>  the build function.
>
> +package() Function
> +------------------
> +An optional package() function can be specified in addtion to the build() function.

addition

> +This function is directly sourced by makepkg and run immediately after the build()

Maybe it could be simplified a bit to just say :
This function is run immediately after the build() function.

> +function. When specified in combination with the fakeroot BUILDENV option in
> +linkman:makepkg.conf[5], fakeroot usage will be limited to running the package()
> +function and the subsequent packing of the package. The build() function will be

Maybe it could be simplified a bit to just say :
fakeroot usage will be limited to running the package() function.

Otherwise it looks fine to me :)

From allan at archlinux.org  Sun Dec 21 04:47:50 2008
From: allan at archlinux.org (Allan McRae)
Date: Sun, 21 Dec 2008 19:47:50 +1000
Subject: [pacman-dev] [PATCH] makepkg: Add docmunetation for separate
 pacakge function
In-Reply-To: <91752840812210059t52e611a6rdeb87396cf06ba7c@mail.gmail.com>
References: <1229843624-26306-1-git-send-email-allan@archlinux.org>
	<91752840812210059t52e611a6rdeb87396cf06ba7c@mail.gmail.com>
Message-ID: <BAY102-DAV231EB0F0A27012469DD2E86EE0@phx.gbl>

Xavier wrote:
>   
>> +function. When specified in combination with the fakeroot BUILDENV option in
>> +linkman:makepkg.conf[5], fakeroot usage will be limited to running the package()
>> +function and the subsequent packing of the package. The build() function will be
>>     
>
> Maybe it could be simplified a bit to just say :
> fakeroot usage will be limited to running the package() function.
>
>   

But it is the package() and internal create_package functions that are 
run in the fakeroot.  What if I just get rid of the bit about the 
build() function being run as the user? Perhaps "fakeroot usage will be 
limited to running the package() function and the actual creation of the 
package."   Still sounds a bit awful...

Allan



From shiningxc at gmail.com  Sun Dec 21 06:55:20 2008
From: shiningxc at gmail.com (Xavier)
Date: Sun, 21 Dec 2008 12:55:20 +0100
Subject: [pacman-dev] [PATCH] makepkg: Add docmunetation for separate
	pacakge function
In-Reply-To: <BAY102-DAV231EB0F0A27012469DD2E86EE0@phx.gbl>
References: <1229843624-26306-1-git-send-email-allan@archlinux.org>
	<91752840812210059t52e611a6rdeb87396cf06ba7c@mail.gmail.com>
	<BAY102-DAV231EB0F0A27012469DD2E86EE0@phx.gbl>
Message-ID: <91752840812210355q592071fdsa943659546d2e5b7@mail.gmail.com>

On Sun, Dec 21, 2008 at 10:47 AM, Allan McRae <allan at archlinux.org> wrote:
> Xavier wrote:
>>
>>
>>>
>>> +function. When specified in combination with the fakeroot BUILDENV
>>> option in
>>> +linkman:makepkg.conf[5], fakeroot usage will be limited to running the
>>> package()
>>> +function and the subsequent packing of the package. The build() function
>>> will be
>>>
>>
>> Maybe it could be simplified a bit to just say :
>> fakeroot usage will be limited to running the package() function.
>>
>>
>
> But it is the package() and internal create_package functions that are run
> in the fakeroot.

I know, I was trying to think like an user simply looking at a PKGBUILD.
I always see the build() function, now I know I can add an optional
package() one which will be run under fakeroot so that build() can be
run as an user. I might not care or think about the rest.

Anyway, it is not a big deal. Your original documentation is still
clear and simple enough, and it is more complete, so you can keep it
as is if you prefer :)

From allan at archlinux.org  Sun Dec 21 09:20:25 2008
From: allan at archlinux.org (Allan McRae)
Date: Mon, 22 Dec 2008 00:20:25 +1000
Subject: [pacman-dev] [PATCH] Added the function parse_options
 to	replace getopt
In-Reply-To: <BAY102-DAV1270BB22AB145720612A8886FD0@phx.gbl>
References: <1227138210-32437-1-git-send-email-yunzheng.hu@gmail.com>	<BAY102-DAV720DCFA5BD7A81DDA252B86040@phx.gbl>	<df84014e0812080622v47d6bd88w79908aa5231ea1a1@mail.gmail.com>	<449c10960812080631w41945303v6d44d366b819040e@mail.gmail.com>
	<BAY102-DAV1270BB22AB145720612A8886FD0@phx.gbl>
Message-ID: <BAY102-DAV1463D93DFDD42697A159286EE0@phx.gbl>

Allan McRae wrote:
> Dan McGee wrote:
>> On Mon, Dec 8, 2008 at 8:22 AM, Yun Zheng Hu <yunzheng.hu at gmail.com> 
>> wrote:
>>  
>>>> Just letting the submitter know that this patch is being looked 
>>>> at.  I have
>>>> a getopt branch on my git repo where this will receive more testing 
>>>> before
>>>> it gets pulled in.  From my current testing, it looks good and we 
>>>> seem to
>>>> have lost no functionality.
>>>>
>>>>       
>>> Do you think this can make it to the pacman 3.2.3 release?
>>>     
>>
>> We (Allan and I) talked about this last night a bit. I would like to
>> get it in, but with a maint release especially, we don't want to have
>> regressions.
>>   
>
> I will try and find time to give it a thorough review in the next 
> couple of days and see if we can get it in.

I've given this a fairly thorough testing now.  It even managed to deal 
properly with passing an argument with a space in it (which is where I 
thought it may fail...). It does not put unused stuff at the end, but 
makepkg actually does nothing about those so this does not really matter...

e.g.  outputting $OPT_TEMP

Not patched
./makepkg --log -p "BUILD SCRIPT" foobar
==>  --log -p 'BUILD SCRIPT' -- 'foobar'

Patched
./makepkg2 --log -p "BUILD SCRIPT" foobar
==>  --log -p 'BUILD SCRIPT' --

It would be nice to do something about adding the unused values at the 
end in case we ever do something with them but that should be a simple 
fix.  I can probably add this tomorrow and pull the patch onto my 
working branch.

Allan



From allan at archlinux.org  Mon Dec 22 06:23:47 2008
From: allan at archlinux.org (Allan McRae)
Date: Mon, 22 Dec 2008 21:23:47 +1000
Subject: [pacman-dev] [PATCH] makepkg: Add docmunetation for separate
 pacakge function
In-Reply-To: <91752840812210355q592071fdsa943659546d2e5b7@mail.gmail.com>
References: <1229843624-26306-1-git-send-email-allan@archlinux.org>	<91752840812210059t52e611a6rdeb87396cf06ba7c@mail.gmail.com>	<BAY102-DAV231EB0F0A27012469DD2E86EE0@phx.gbl>
	<91752840812210355q592071fdsa943659546d2e5b7@mail.gmail.com>
Message-ID: <BAY102-DAV3E46593100F7EC60014A986EF0@phx.gbl>

Incorporating Xavier's comments, how about:

package() Function
------------------
An optional package() function can be specified in addition to the 
build() function.
This function is run immediately after the build() function. When 
specified in
combination with the fakeroot BUILDENV option in 
linkman:makepkg.conf[5], fakeroot
usage will be limited to running the packaging stages. The build() 
function will be
run as the user calling makepkg.





From allan at archlinux.org  Mon Dec 22 06:39:16 2008
From: allan at archlinux.org (Allan McRae)
Date: Mon, 22 Dec 2008 21:39:16 +1000
Subject: [pacman-dev] [PATCH] Added the function parse_options
 to	replace getopt
In-Reply-To: <BAY102-DAV1463D93DFDD42697A159286EE0@phx.gbl>
References: <1227138210-32437-1-git-send-email-yunzheng.hu@gmail.com>	<BAY102-DAV720DCFA5BD7A81DDA252B86040@phx.gbl>	<df84014e0812080622v47d6bd88w79908aa5231ea1a1@mail.gmail.com>	<449c10960812080631w41945303v6d44d366b819040e@mail.gmail.com>	<BAY102-DAV1270BB22AB145720612A8886FD0@phx.gbl>
	<BAY102-DAV1463D93DFDD42697A159286EE0@phx.gbl>
Message-ID: <BAY102-DAV1ED0C5AB58D899F338F3886EF0@phx.gbl>

Allan McRae wrote:
> Allan McRae wrote:
>> Dan McGee wrote:
>>> On Mon, Dec 8, 2008 at 8:22 AM, Yun Zheng Hu <yunzheng.hu at gmail.com> 
>>> wrote:
>>>  
>>>>> Just letting the submitter know that this patch is being looked 
>>>>> at.  I have
>>>>> a getopt branch on my git repo where this will receive more 
>>>>> testing before
>>>>> it gets pulled in.  From my current testing, it looks good and we 
>>>>> seem to
>>>>> have lost no functionality.
>>>>>
>>>>>       
>>>> Do you think this can make it to the pacman 3.2.3 release?
>>>>     
>>>
>>> We (Allan and I) talked about this last night a bit. I would like to
>>> get it in, but with a maint release especially, we don't want to have
>>> regressions.
>>>   
>>
>> I will try and find time to give it a thorough review in the next 
>> couple of days and see if we can get it in.
>
> I've given this a fairly thorough testing now.  It even managed to 
> deal properly with passing an argument with a space in it (which is 
> where I thought it may fail...). It does not put unused stuff at the 
> end, but makepkg actually does nothing about those so this does not 
> really matter...
>
> e.g.  outputting $OPT_TEMP
>
> Not patched
> ./makepkg --log -p "BUILD SCRIPT" foobar
> ==>  --log -p 'BUILD SCRIPT' -- 'foobar'
>
> Patched
> ./makepkg2 --log -p "BUILD SCRIPT" foobar
> ==>  --log -p 'BUILD SCRIPT' --
>
> It would be nice to do something about adding the unused values at the 
> end in case we ever do something with them but that should be a simple 
> fix.  I can probably add this tomorrow and pull the patch onto my 
> working branch.

I found some other differences too.  makepkg = original, makepkg2 = this 
patch

 > ./makepkg -psr
==>  -p 'sr' --

 > ./makepkg2 -psr
==>  -p '' --


 > ./makepkg -sp
makepkg: option requires an argument -- 'p'
==>  -s --
GETOPT GO BANG!

 > ./makepkg2 -sp
makepkg: option requires an argument -- 'sp'
==>  -s -p --
PARSE_OPTIONS FAILED


And just for completeness....

 > ./makepkg fakeroot -- -s -r
==>  -- 'fakeroot' '-s' '-r'

 > ./makepkg2 fakeroot -- -s -r
==>  -- '-s' '-r'

I have fixed/rewritten the patch and will send it here soon.

Allan




From allan at archlinux.org  Mon Dec 22 06:48:57 2008
From: allan at archlinux.org (Allan McRae)
Date: Mon, 22 Dec 2008 21:48:57 +1000
Subject: [pacman-dev] [PATCH] makepkg: Replace getopt with internal function
Message-ID: <1229946537-19820-1-git-send-email-allan@archlinux.org>

This will allow makepkg to work on systems like Mac OS X where the
default getopt is too old to properly handle long options.

The new parse_options function should replicate getopt's behaviour
completely.

Original work: Yun Zheng Hu <yunzheng.hu at gmail.com>
[Allan: Rewrite and bug fixes]
Signed-off-by: Allan McRae <allan at archlinux.org>
---
 scripts/makepkg.sh.in |   91 +++++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 89 insertions(+), 2 deletions(-)

diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in
index ea034ad..8cf301c 100644
--- a/scripts/makepkg.sh.in
+++ b/scripts/makepkg.sh.in
@@ -1186,6 +1186,93 @@ devel_update() {
 	fi
 }
 
+# getopt like parser
+parse_options() {
+	local short_options=$1; shift;
+	local long_options=$1; shift;
+	local ret=0;
+	local unused_options=""
+
+	while [ -n "$1" ]; do
+		if [ ${1:0:2} = '--' ]; then
+			if [ -n "${1:2}" ]; then
+				local match=""
+				for i in ${long_options//,/ }; do
+					if [ ${1:2} = ${i//:} ]; then
+						match=$i
+						break
+					fi
+				done
+				if [ -n "$match" ]; then
+					if [ ${1:2} = $match ]; then
+						printf ' %s' "$1"
+					else
+						if [ -n "$2" ]; then
+							printf ' %s' "$1"
+							shift
+							printf " '%s'" "$1"
+						else
+							echo "makepkg: option '$1' $(gettext "requires an argument")" >&2
+							ret=1
+						fi
+					fi
+				else
+					echo "makepkg: $(gettext "unrecognized option") '$1'" >&2
+					ret=1
+				fi
+			else
+				shift
+				break
+			fi
+		elif [ ${1:0:1} = '-' ]; then
+			for ((i=1; i<${#1}; i++)); do
+				if [[ "$short_options" =~ "${1:i:1}" ]]; then
+					if [[ "$short_options" =~ "${1:i:1}:" ]]; then
+						if [ -n "${1:$i+1}" ]; then
+							printf ' -%s' "${1:i:1}"
+							printf " '%s'" "${1:$i+1}"
+						else
+							if [ -n "$2" ]; then
+								printf ' -%s' "${1:i:1}"
+								shift
+								printf " '%s'" "${1}"
+							else
+								echo "makepkg: option $(gettext "requires an argument") -- '${1:i:1}'" >&2
+								ret=1
+							fi
+						fi
+						break
+					else
+						printf ' -%s' "${1:i:1}"
+					fi
+				else
+					echo "makepkg: $(gettext "invalid option") -- '${1:i:1}'" >&2
+					ret=1
+				fi
+			done
+		else
+			unused_options="${unused_options} '$1'"
+		fi
+		shift
+	done
+
+	printf " --"
+	if [ -n "$unused_options" ]; then
+		for i in ${unused_options[@]}; do
+			printf ' %s' "$i"
+		done
+	fi
+	if [ -n "$1" ]; then
+		while [ -n "$1" ]; do
+			printf " '%s'" "${1}"
+			shift
+		done
+	fi
+	printf "\n"
+
+	return $ret
+}
+
 usage() {
 	printf "makepkg (pacman) %s\n" "$myver"
 	echo
@@ -1251,8 +1338,8 @@ OPT_LONG="$OPT_LONG,install,log,nocolor,nobuild,rmdeps,repackage,source"
 OPT_LONG="$OPT_LONG,syncdeps,version,config:"
 # Pacman Options
 OPT_LONG="$OPT_LONG,noconfirm,noprogressbar"
-OPT_TEMP="$(getopt -o "$OPT_SHORT" -l "$OPT_LONG" -n "$(basename "$0")" -- "$@" || echo 'GETOPT GO BANG!')"
-if echo "$OPT_TEMP" | grep -q 'GETOPT GO BANG!'; then
+OPT_TEMP="$(parse_options $OPT_SHORT $OPT_LONG "$@" || echo 'PARSE_OPTIONS FAILED')"
+if echo "$OPT_TEMP" | grep -q 'PARSE_OPTIONS FAILED'; then
 	# This is a small hack to stop the script bailing with 'set -e'
 	echo; usage; exit 1 # E_INVALID_OPTION;
 fi
-- 
1.6.0.5


From dpmcgee at gmail.com  Mon Dec 22 13:34:50 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Mon, 22 Dec 2008 12:34:50 -0600
Subject: [pacman-dev] [PATCH] makepkg: Add docmunetation for separate
	pacakge function
In-Reply-To: <BAY102-DAV3E46593100F7EC60014A986EF0@phx.gbl>
References: <1229843624-26306-1-git-send-email-allan@archlinux.org>
	<91752840812210059t52e611a6rdeb87396cf06ba7c@mail.gmail.com>
	<BAY102-DAV231EB0F0A27012469DD2E86EE0@phx.gbl>
	<91752840812210355q592071fdsa943659546d2e5b7@mail.gmail.com>
	<BAY102-DAV3E46593100F7EC60014A986EF0@phx.gbl>
Message-ID: <449c10960812221034p2b598a05w4a95f73211dcfaa9@mail.gmail.com>

On Mon, Dec 22, 2008 at 5:23 AM, Allan McRae <allan at archlinux.org> wrote:
> Incorporating Xavier's comments, how about:
>
> package() Function
> ------------------
> An optional package() function can be specified in addition to the build()
> function.
> This function is run immediately after the build() function. When specified
> in
> combination with the fakeroot BUILDENV option in linkman:makepkg.conf[5],
> fakeroot
> usage will be limited to running the packaging stages. The build() function
How about just "stage"? Or maybe it just sounds weird to me because
I'm not used to makepkg being able to make multiple packages.
> will be
> run as the user calling makepkg.

From allan at archlinux.org  Tue Dec 23 00:44:24 2008
From: allan at archlinux.org (Allan McRae)
Date: Tue, 23 Dec 2008 15:44:24 +1000
Subject: [pacman-dev] makepkg: package splitting
In-Reply-To: <d64a48980812150945t77191026p1efc02f0bb28c15d@mail.gmail.com>
References: <493C87FD.8080501@archlinux.org>
	<1228822256.16706.12.camel@marc>	<493E5D35.5050702@archlinux.org>
	<1228825387.16706.20.camel@marc>	<493E655D.8030501@archlinux.org>
	<1229333307.8183.13.camel@marc>	<BAY102-DAV5A8BCFCC5998522518EB286F40@phx.gbl>	<494642E0.6030805@archlinux.org>
	<1229342035.10057.3.camel@marc>	<BAY102-DAV1260E02BFFA76D976584F086F40@phx.gbl>
	<d64a48980812150945t77191026p1efc02f0bb28c15d@mail.gmail.com>
Message-ID: <BAY102-DAV2FE5D212BF7AE9BE9676086EC0@phx.gbl>

Aaron Griffin wrote:
> On Mon, Dec 15, 2008 at 6:24 AM, Allan McRae <allan at archlinux.org> wrote:
>   
>> Marc - A. Dahlhaus [ Administration | Westermann GmbH ] wrote:
>>     
>>> Am Montag, den 15.12.2008, 21:43 +1000 schrieb Allan McRae:
>>>  Nice, i'll test and report back...
>>> Is there any schedule on when it will be merged to mainline?
>>>
>>>
>>>       
>> The current schedule is when I tell Dan it ready...  I want to add
>> documentation and clean up some of the output with the package splitting but
>> otherwise I am happy with it now.  So, the answer is "soonish".
>>     
>
> Yay. I'm hoping for a Christmas present 8)

Well, I have decided the implementation is stable enough now and any 
further issues/polishing can be done on the master branch.  I have 
pulled all the package splitting work on to my working branch ready for 
Dan to pull to mainline.

I have made a makepkg-git package which can be installed alongside 
makepkg from pacman.  I will post it later so people can test out 
package splitting and the other new features.

Allan




From allan at archlinux.org  Tue Dec 23 06:45:55 2008
From: allan at archlinux.org (Allan McRae)
Date: Tue, 23 Dec 2008 21:45:55 +1000
Subject: [pacman-dev] makepkg developmental version
Message-ID: <BAY102-DAV1B36A570F173A80998A4286EC0@phx.gbl>

Hi all,

I have just packaged a version of makepkg from my git working branch for 
people to test out.  This installs alongside the makepkg from pacman so 
won't affect your regular package building.

i686: http://dev.archlinux.org/~allan/makepkg-git-20081223-1-i686.pkg.tar.gz
x86_64: 
http://dev.archlinux.org/~allan/makepkg-git-20081223-1-x86_64.pkg.tar.gz


New features of interest:

Package splitting support!
  - multiple packages are specified in the pkgname variable
  - each package gets its own packaging function where package variables 
can be overridden
  - see PKGBUILD-split.proto (in /usr/share/pacman) for "documentation"

Added optional package() function in PKGBUILDs
  - limits fakeroot usage to packaging stages
  - see "man PKGBUILD-git.conf" for more info

Treat info pages like man pages:
  - automatically compress info pages
  - info pages are not removed with the !docs options

Added purge option:
  - automatically removes common conflicting or removed files
  - specify files in PURGE_TARGETS in makepkg.conf
  - default PURGE_TARGETS=(usr/{,share}/info/dir .packlist *.pod)

Replace getopt usage with internal function:
  - allows makepkg to run on systems with old getopt version (e.g. Mac OS X)

Other smaller changes:
  - ability to pass PKGBUILD from pipe
  - package compression type auto-detection
  - add configuration option to specify man/info page directories
  - specify alternative configuration file using the --config option
  - all integrity sums provided are checked
  - some fixes from the pacman 3.2 maintenance branch:
      - detect CRLF line endings
      - when automatically updating an SCM package pkgver, pkgrel is 
reset to 1
      - speed up mercurial package revision checking
      - improved compatibility with older bash versions

Known issues:
  - The -p flag to specify an alternative name for a PKGBUILD is broken
  - Repackaging with split-packages definitely does not work
  - Repackaging when using package() function probably does not work...
  - Some minor output changes are needed for package splitting


Bug reports either to the pacman-dev list or the pacman/makepkg section 
of the bug tracker.

Enjoy,
Allan




From yunzheng.hu at gmail.com  Tue Dec 23 17:10:13 2008
From: yunzheng.hu at gmail.com (Yun Zheng Hu)
Date: Tue, 23 Dec 2008 23:10:13 +0100
Subject: [pacman-dev] makepkg developmental version
In-Reply-To: <BAY102-DAV1B36A570F173A80998A4286EC0@phx.gbl>
References: <BAY102-DAV1B36A570F173A80998A4286EC0@phx.gbl>
Message-ID: <df84014e0812231410h4b35d2c2ue4724ad271b7b1a9@mail.gmail.com>

On Tue, Dec 23, 2008 at 12:45 PM, Allan McRae <allan at archlinux.org> wrote:
>
> Hi all,
>
> I have just packaged a version of makepkg from my git working branch for people to test out.  This installs alongside the makepkg from pacman so won't affect your regular package building.
>
> i686: http://dev.archlinux.org/~allan/makepkg-git-20081223-1-i686.pkg.tar.gz
> x86_64: http://dev.archlinux.org/~allan/makepkg-git-20081223-1-x86_64.pkg.tar.gz
>

Can you also supply a source tarball? (preferably already autogen'ed).

From allan at archlinux.org  Tue Dec 23 19:35:28 2008
From: allan at archlinux.org (Allan McRae)
Date: Wed, 24 Dec 2008 10:35:28 +1000
Subject: [pacman-dev] makepkg developmental version
In-Reply-To: <df84014e0812231410h4b35d2c2ue4724ad271b7b1a9@mail.gmail.com>
References: <BAY102-DAV1B36A570F173A80998A4286EC0@phx.gbl>
	<df84014e0812231410h4b35d2c2ue4724ad271b7b1a9@mail.gmail.com>
Message-ID: <BAY102-DAV7B9731B2960FAE9EA35F986ED0@phx.gbl>

Yun Zheng Hu wrote:
> On Tue, Dec 23, 2008 at 12:45 PM, Allan McRae <allan at archlinux.org> wrote:
>   
>> Hi all,
>>
>> I have just packaged a version of makepkg from my git working branch for people to test out.  This installs alongside the makepkg from pacman so won't affect your regular package building.
>>
>> i686: http://dev.archlinux.org/~allan/makepkg-git-20081223-1-i686.pkg.tar.gz
>> x86_64: http://dev.archlinux.org/~allan/makepkg-git-20081223-1-x86_64.pkg.tar.gz
>>
>>     
>
> Can you also supply a source tarball? (preferably already autogen'ed).
>   

I suppose... but in the meantime, here is a really bad PKGBUILD that 
will pull it from git: http://dev.archlinux.org/~allan/PKGBUILD





From yunzheng.hu at gmail.com  Wed Dec 24 11:32:21 2008
From: yunzheng.hu at gmail.com (Yun Zheng Hu)
Date: Wed, 24 Dec 2008 17:32:21 +0100
Subject: [pacman-dev] makepkg developmental version
In-Reply-To: <BAY102-DAV7B9731B2960FAE9EA35F986ED0@phx.gbl>
References: <BAY102-DAV1B36A570F173A80998A4286EC0@phx.gbl>
	<df84014e0812231410h4b35d2c2ue4724ad271b7b1a9@mail.gmail.com>
	<BAY102-DAV7B9731B2960FAE9EA35F986ED0@phx.gbl>
Message-ID: <df84014e0812240832q2d6a261eo378e9ca6e5aaf87a@mail.gmail.com>

On Wed, Dec 24, 2008 at 1:35 AM, Allan McRae <allan at archlinux.org> wrote:
>
> I suppose... but in the meantime, here is a really bad PKGBUILD that will
> pull it from git: http://dev.archlinux.org/~allan/PKGBUILD
>

Thanks, got the PKGBUILD working on Mac OS X with some small modifications.

Found a bug with the -p <buildscript> flag, if you set this option it
will get overriden in makepkg when sourcing the config file, on line
1443:

# Source the config file; fail if it is not found
if [ -r "$MAKEPKG_CONF" ]; then
	source "$MAKEPKG_CONF"

Another small bug, on line 1500, if the BUILDSCRIPT filename contains
whitespace it will fail:
if [ -z $BUILDSCRIPT ]; then

should be quoted like this:
if [ -z "$BUILDSCRIPT" ]; then

From yunzheng.hu at gmail.com  Wed Dec 24 11:55:57 2008
From: yunzheng.hu at gmail.com (Yun Zheng Hu)
Date: Wed, 24 Dec 2008 17:55:57 +0100
Subject: [pacman-dev] makepkg developmental version
In-Reply-To: <df84014e0812240832q2d6a261eo378e9ca6e5aaf87a@mail.gmail.com>
References: <BAY102-DAV1B36A570F173A80998A4286EC0@phx.gbl>
	<df84014e0812231410h4b35d2c2ue4724ad271b7b1a9@mail.gmail.com>
	<BAY102-DAV7B9731B2960FAE9EA35F986ED0@phx.gbl>
	<df84014e0812240832q2d6a261eo378e9ca6e5aaf87a@mail.gmail.com>
Message-ID: <df84014e0812240855h57b24211s4fdb4697a200132e@mail.gmail.com>

On Wed, Dec 24, 2008 at 5:32 PM, Yun Zheng Hu <yunzheng.hu at gmail.com> wrote:
> Found a bug with the -p <buildscript> flag, if you set this option it
> will get overriden in makepkg when sourcing the config file, on line
> 1443:
>
> # Source the config file; fail if it is not found
> if [ -r "$MAKEPKG_CONF" ]; then
>        source "$MAKEPKG_CONF"
>

Oops, I just saw this mentioned in the known issues. So ignore this one.

From shiningxc at gmail.com  Wed Dec 24 12:55:47 2008
From: shiningxc at gmail.com (Xavier)
Date: Wed, 24 Dec 2008 18:55:47 +0100
Subject: [pacman-dev] Dan's pacman tree build&test
In-Reply-To: <20081204154308.11a350d8@tux1.brauer.lan>
References: <20081204154308.11a350d8@tux1.brauer.lan>
Message-ID: <91752840812240955y454ef516m9e62c0ed6b33b5e0@mail.gmail.com>

On Thu, Dec 4, 2008 at 3:43 PM, Gerhard Brauer <gerbra at archlinux.de> wrote:
> Hi Dan,
>
> git clone http://code.toofishes.net/gitprojects/pacman.git
> (branch is master)
>
> 1. if have to install asciidox (for a2x), that's current not a
> (build)depend of pacman
>
> make stops with an error
>
> make[2]: Entering directory `/home/gerhard/al-de/pac-git/pacman/doc'
> a2x -d manpage -f manpage --xsltproc-opts='-param man.endnotes.list.enabled 0' --xsltproc-opts='-param man.en
> dnotes.are.numbered 0' --asciidoc-opts="-f asciidoc.conf -a pacman_version="3.2.1" -a pacman_date="`date +%Y-
> %m-%d`" -a sysconfdir=/etc" PKGBUILD.5.txt
> ./PKGBUILD.5.xml:148: element literal: validity error : Element emphasis is not declared in literal list of p
> ossible children
> e. The syntax is: <literal>source=(<emphasis>filename::url</emphasis>)</literal>
>                                                                               ^
> ./PKGBUILD.5.xml:780: element programlisting: validity error : No declaration for attribute language of eleme
> nt programlisting
> <programlisting language="sh" linenumbering="unnumbered"># Maintainer: Joe User
>                                                        ^
> ./PKGBUILD.5.xml:780: parser error : error parsing attribute name
> isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user
>                                                                               ^
> ./PKGBUILD.5.xml:780: parser error : attributes construct error
> isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user
>                                                                               ^
> ./PKGBUILD.5.xml:780: parser error : Couldn't find end of Start Tag joe.user line 780
> isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user
>                                                                               ^
> a2x: failed: xmllint --nonet --noout --valid "./PKGBUILD.5.xml"
> make[2]: *** [PKGBUILD.5] Fehler 1
> make[2]: Leaving directory `/home/gerhard/al-de/pac-git/pacman/doc'
> make[1]: *** [all-recursive] Fehler 1
> make[1]: Leaving directory `/home/gerhard/al-de/pac-git/pacman'
> make: *** [all] Fehler 2
>
> Seems any error in XML-Input doc file.
>
> Is it ok to post to this ML? Or better a private mail address?
>

I also have this problem, and I have not been able to figure it out myself.
I guess it is a problem if we can't build our man pages anymore :P

From allan at archlinux.org  Wed Dec 24 20:58:36 2008
From: allan at archlinux.org (Allan McRae)
Date: Thu, 25 Dec 2008 11:58:36 +1000
Subject: [pacman-dev] Dan's pacman tree build&test
In-Reply-To: <91752840812240955y454ef516m9e62c0ed6b33b5e0@mail.gmail.com>
References: <20081204154308.11a350d8@tux1.brauer.lan>
	<91752840812240955y454ef516m9e62c0ed6b33b5e0@mail.gmail.com>
Message-ID: <BAY102-DAV109B7265787F3410D0374486EA0@phx.gbl>

Xavier wrote:
> On Thu, Dec 4, 2008 at 3:43 PM, Gerhard Brauer <gerbra at archlinux.de> wrote:
>   
>> Hi Dan,
>>
>> git clone http://code.toofishes.net/gitprojects/pacman.git
>> (branch is master)
>>
>> 1. if have to install asciidox (for a2x), that's current not a
>> (build)depend of pacman
>>
>> make stops with an error
>>
>> make[2]: Entering directory `/home/gerhard/al-de/pac-git/pacman/doc'
>> a2x -d manpage -f manpage --xsltproc-opts='-param man.endnotes.list.enabled 0' --xsltproc-opts='-param man.en
>> dnotes.are.numbered 0' --asciidoc-opts="-f asciidoc.conf -a pacman_version="3.2.1" -a pacman_date="`date +%Y-
>> %m-%d`" -a sysconfdir=/etc" PKGBUILD.5.txt
>> ./PKGBUILD.5.xml:148: element literal: validity error : Element emphasis is not declared in literal list of p
>> ossible children
>> e. The syntax is: <literal>source=(<emphasis>filename::url</emphasis>)</literal>
>>                                                                               ^
>> ./PKGBUILD.5.xml:780: element programlisting: validity error : No declaration for attribute language of eleme
>> nt programlisting
>> <programlisting language="sh" linenumbering="unnumbered"># Maintainer: Joe User
>>                                                        ^
>> ./PKGBUILD.5.xml:780: parser error : error parsing attribute name
>> isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user
>>                                                                               ^
>> ./PKGBUILD.5.xml:780: parser error : attributes construct error
>> isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user
>>                                                                               ^
>> ./PKGBUILD.5.xml:780: parser error : Couldn't find end of Start Tag joe.user line 780
>> isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user
>>                                                                               ^
>> a2x: failed: xmllint --nonet --noout --valid "./PKGBUILD.5.xml"
>> make[2]: *** [PKGBUILD.5] Fehler 1
>> make[2]: Leaving directory `/home/gerhard/al-de/pac-git/pacman/doc'
>> make[1]: *** [all-recursive] Fehler 1
>> make[1]: Leaving directory `/home/gerhard/al-de/pac-git/pacman'
>> make: *** [all] Fehler 2
>>
>> Seems any error in XML-Input doc file.
>>
>> Is it ok to post to this ML? Or better a private mail address?
>>
>>     
>
> I also have this problem, and I have not been able to figure it out myself.
> I guess it is a problem if we can't build our man pages anymore :P

I have the asciidoc-8.2.7 package for i686 which works if someone wants 
a copy.  I think Dna has the x86_64 one.  There has been an 8.3.1 
release but that still breaks things...

Allan



From allan at archlinux.org  Wed Dec 24 21:06:52 2008
From: allan at archlinux.org (Allan McRae)
Date: Thu, 25 Dec 2008 12:06:52 +1000
Subject: [pacman-dev] makepkg developmental version
In-Reply-To: <df84014e0812240832q2d6a261eo378e9ca6e5aaf87a@mail.gmail.com>
References: <BAY102-DAV1B36A570F173A80998A4286EC0@phx.gbl>	<df84014e0812231410h4b35d2c2ue4724ad271b7b1a9@mail.gmail.com>	<BAY102-DAV7B9731B2960FAE9EA35F986ED0@phx.gbl>
	<df84014e0812240832q2d6a261eo378e9ca6e5aaf87a@mail.gmail.com>
Message-ID: <BAY102-DAV13A4AE8C6FBE6D018740A486EA0@phx.gbl>

Yun Zheng Hu wrote:
> Another small bug, on line 1500, if the BUILDSCRIPT filename contains
> whitespace it will fail:
> if [ -z $BUILDSCRIPT ]; then
>
> should be quoted like this:
> if [ -z "$BUILDSCRIPT" ]; then

I noticed this the other day when working on how to deal with options 
with spaces in them.   Is that the only change that needs made for 
PKGBUILDs with spaces to work?

Allan



From allan at archlinux.org  Wed Dec 24 21:22:26 2008
From: allan at archlinux.org (Allan McRae)
Date: Thu, 25 Dec 2008 12:22:26 +1000
Subject: [pacman-dev] makepkg developmental version
In-Reply-To: <BAY102-DAV13A4AE8C6FBE6D018740A486EA0@phx.gbl>
References: <BAY102-DAV1B36A570F173A80998A4286EC0@phx.gbl>	<df84014e0812231410h4b35d2c2ue4724ad271b7b1a9@mail.gmail.com>	<BAY102-DAV7B9731B2960FAE9EA35F986ED0@phx.gbl>	<df84014e0812240832q2d6a261eo378e9ca6e5aaf87a@mail.gmail.com>
	<BAY102-DAV13A4AE8C6FBE6D018740A486EA0@phx.gbl>
Message-ID: <BAY102-DAV43A1DA3E4FA777D5DE61786EA0@phx.gbl>

Allan McRae wrote:
> Yun Zheng Hu wrote:
>> Another small bug, on line 1500, if the BUILDSCRIPT filename contains
>> whitespace it will fail:
>> if [ -z $BUILDSCRIPT ]; then
>>
>> should be quoted like this:
>> if [ -z "$BUILDSCRIPT" ]; then
>
> I noticed this the other day when working on how to deal with options 
> with spaces in them.   Is that the only change that needs made for 
> PKGBUILDs with spaces to work?

And of course I don't really mean PKGBUILDs with spaces....

Allan



From shiningxc at gmail.com  Thu Dec 25 02:45:49 2008
From: shiningxc at gmail.com (Xavier Chantry)
Date: Thu, 25 Dec 2008 08:45:49 +0100
Subject: [pacman-dev] [PATCH] Fix asciidoc manpage creation.
Message-ID: <1230191149-4789-1-git-send-email-shiningxc@gmail.com>

As reported here, man pages could no longer be built :
http://archlinux.org/pipermail/pacman-dev/2008-December/007726.html

I found the explanation here :
http://www.methods.co.nz/asciidoc/source-highlight-filter.html

"If you use a2x(1) to generate PDF you need to include the --no-xmllint
option to suppress xmllint(1) checking???the programlisting language
attribute (required by the dblatex source highlighter) is not part of the
DocBook 4 specification (but it is in the newer DocBook 5 specification)."

Signed-off-by: Xavier Chantry <shiningxc at gmail.com>
---
 doc/Makefile.am    |    1 +
 doc/PKGBUILD.5.txt |    7 +++----
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/doc/Makefile.am b/doc/Makefile.am
index cce0a71..7992e54 100644
--- a/doc/Makefile.am
+++ b/doc/Makefile.am
@@ -62,6 +62,7 @@ ASCIIDOC_OPTS = \
 	-a pacman_date="`date +%Y-%m-%d`" \
 	-a sysconfdir=$(sysconfdir)
 A2X_OPTS = \
+	--no-xmllint \
 	-d manpage \
 	-f manpage \
 	--xsltproc-opts='-param man.endnotes.list.enabled 0' \
diff --git a/doc/PKGBUILD.5.txt b/doc/PKGBUILD.5.txt
index e42a8b8..466ef17 100644
--- a/doc/PKGBUILD.5.txt
+++ b/doc/PKGBUILD.5.txt
@@ -367,11 +367,10 @@ The following is an example PKGBUILD for the 'patch' package. For more
 examples, look through the build files of your distribution's packages. For
 those using Arch Linux, consult the ABS tree.
 
-[sh]
-source~~~~~
+[source,sh]
+-------------------------------
 include::PKGBUILD-example.txt[]
-source~~~~~
-
+-------------------------------
 
 See Also
 --------
-- 
1.6.0.6


From shiningxc at gmail.com  Thu Dec 25 03:24:54 2008
From: shiningxc at gmail.com (Xavier Chantry)
Date: Thu, 25 Dec 2008 09:24:54 +0100
Subject: [pacman-dev] [PATCH] Remove libdownload support and fix libfetch
	one.
Message-ID: <1230193494-14782-1-git-send-email-shiningxc@gmail.com>

Aaron said to consider libdownload a dead project so libdownload support was
removed to more easily fix libfetch one (otherwise many ifdef needed).

There was no direct replacement for ferror to detect an error while
downloading. So instead, I added a check at the end to see if the file was
fully downloaded, which is just a small chunk of code taken from here:
http://cvsweb.netbsd.org/bsdweb.cgi/pkgsrc/net/libfetch/files/fetch.c?only_with_tag=MAIN

Signed-off-by: Xavier Chantry <shiningxc at gmail.com>
---
 configure.ac        |   13 +++++--------
 lib/libalpm/dload.c |   37 ++++++++++++++++---------------------
 lib/libalpm/error.c |    5 +----
 3 files changed, 22 insertions(+), 33 deletions(-)

diff --git a/configure.ac b/configure.ac
index c7841b8..1b3d702 100644
--- a/configure.ac
+++ b/configure.ac
@@ -88,9 +88,9 @@ AC_ARG_WITH(db-ext,
 	AS_HELP_STRING([--with-db-ext=ext], [set the file extension used by the database]),
 	[DBEXT=$withval], [DBEXT=.db.tar.gz])
 
-# Help line for libdownload/libfetch
+# Help line for libfetch
 AC_ARG_ENABLE(internal-download,
-	AS_HELP_STRING([--disable-internal-download], [do not build with libdownload/libfetch support]),
+	AS_HELP_STRING([--disable-internal-download], [do not build with libfetch support]),
 	[internaldownload=$enableval], [internaldownload=yes])
 
 # Help line for documentation
@@ -131,17 +131,14 @@ AM_GNU_GETTEXT_VERSION(0.13.1)
 AC_CHECK_LIB([archive], [archive_read_data], ,
 	AC_MSG_ERROR([libarchive is needed to compile pacman!]))
 
-# Enable or disable usage of libdownload/libfetch
-# - this is a nested check- first see if we need a library, if we do then
-#   check for libdownload first, then fallback to libfetch, then die
+# Enable or disable usage of libfetch
 AC_MSG_CHECKING(whether to link with download library)
 if test "x$internaldownload" = "xyes" ; then
 	AC_MSG_RESULT(yes)
 	AC_DEFINE([INTERNAL_DOWNLOAD], , [Use internal download library])
 	# Check for a download library if it was actually requested
-	AC_CHECK_LIB([download], [downloadParseURL], ,
-		AC_CHECK_LIB([fetch], [fetchParseURL], ,
-			AC_MSG_ERROR([libdownload or libfetch are needed to compile with internal download support])) )
+	AC_CHECK_LIB([fetch], [fetchParseURL], ,
+		AC_MSG_ERROR([libfetch is needed to compile with internal download support]) )
 else
 	AC_MSG_RESULT(no)
 fi
diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c
index 9934f30..c0ee28d 100644
--- a/lib/libalpm/dload.c
+++ b/lib/libalpm/dload.c
@@ -34,15 +34,7 @@
 #include <sys/param.h> /* MAXHOSTNAMELEN */
 #endif
 
-#if defined(HAVE_LIBDOWNLOAD)
-#include <download.h>
-#define fetchFreeURL downloadFreeURL
-#define fetchLastErrCode downloadLastErrCode
-#define fetchLastErrString downloadLastErrString
-#define fetchParseURL downloadParseURL
-#define fetchTimeout downloadTimeout
-#define fetchXGet downloadXGet
-#elif defined(HAVE_LIBFETCH)
+#if defined(INTERNAL_DOWNLOAD)
 #include <fetch.h>
 #endif
 
@@ -109,7 +101,8 @@ static struct url *url_for_string(const char *url)
 
 static int download_internal(const char *url, const char *localpath,
 		time_t mtimeold, time_t *mtimenew) {
-	FILE *dlf, *localf = NULL;
+	fetchIO *dlf = NULL;
+	FILE *localf = NULL;
 	struct url_stat ust;
 	struct stat st;
 	int chk_resume = 0, ret = 0;
@@ -208,16 +201,8 @@ static int download_internal(const char *url, const char *localpath,
 		handle->dlcb(filename, 0, ust.size);
 	}
 
-	while((nread = fread(buffer, 1, PM_DLBUF_LEN, dlf)) > 0) {
+	while((nread = fetchIO_read(dlf, buffer, PM_DLBUF_LEN)) > 0) {
 		size_t nwritten = 0;
-		if(ferror(dlf)) {
-			pm_errno = PM_ERR_LIBFETCH;
-			_alpm_log(PM_LOG_ERROR, _("error downloading '%s': %s\n"),
-					filename, fetchLastErrString);
-			ret = -1;
-			goto cleanup;
-		}
-
 		while(nwritten < nread) {
 			nwritten += fwrite(buffer, 1, (nread - nwritten), localf);
 			if(ferror(localf)) {
@@ -233,12 +218,22 @@ static int download_internal(const char *url, const char *localpath,
 			handle->dlcb(filename, dl_thisfile, ust.size);
 		}
 	}
+
+	/* did the transfer complete normally? */
+	if (ust.size != -1 && dl_thisfile < ust.size) {
+		pm_errno = PM_ERR_LIBFETCH;
+		_alpm_log(PM_LOG_ERROR, _("%s appears to be truncated: %jd/%jd bytes\n"),
+				filename, (intmax_t)dl_thisfile, (intmax_t)ust.size);
+		ret = -1;
+		goto cleanup;
+	}
+
 	/* probably safer to close the file descriptors now before renaming the file,
 	 * for example to make sure the buffers are flushed.
 	 */
 	fclose(localf);
 	localf = NULL;
-	fclose(dlf);
+	fetchIO_close(dlf);
 	dlf = NULL;
 
 	rename(tempfile, destfile);
@@ -254,7 +249,7 @@ cleanup:
 		fclose(localf);
 	}
 	if(dlf != NULL) {
-		fclose(dlf);
+		fetchIO_close(dlf);
 	}
 	fetchFreeURL(fileurl);
 	return(ret);
diff --git a/lib/libalpm/error.c b/lib/libalpm/error.c
index 80a4fb1..df93cb7 100644
--- a/lib/libalpm/error.c
+++ b/lib/libalpm/error.c
@@ -30,10 +30,7 @@
 #include <sys/param.h> /* MAXHOSTNAMELEN */
 #endif
 
-#if defined(HAVE_LIBDOWNLOAD)
-#include <download.h> /* downloadLastErrString */
-#define fetchLastErrString downloadLastErrString
-#elif defined(HAVE_LIBFETCH)
+#if defined(INTERNAL_DOWNLOAD)
 #include <fetch.h> /* fetchLastErrString */
 #endif
 
-- 
1.6.0.6


From yunzheng.hu at gmail.com  Thu Dec 25 18:23:10 2008
From: yunzheng.hu at gmail.com (Yun Zheng Hu)
Date: Fri, 26 Dec 2008 00:23:10 +0100
Subject: [pacman-dev] makepkg developmental version
In-Reply-To: <BAY102-DAV43A1DA3E4FA777D5DE61786EA0@phx.gbl>
References: <BAY102-DAV1B36A570F173A80998A4286EC0@phx.gbl>
	<df84014e0812231410h4b35d2c2ue4724ad271b7b1a9@mail.gmail.com>
	<BAY102-DAV7B9731B2960FAE9EA35F986ED0@phx.gbl>
	<df84014e0812240832q2d6a261eo378e9ca6e5aaf87a@mail.gmail.com>
	<BAY102-DAV13A4AE8C6FBE6D018740A486EA0@phx.gbl>
	<BAY102-DAV43A1DA3E4FA777D5DE61786EA0@phx.gbl>
Message-ID: <df84014e0812251523x50b5344dga54e463750faebd7@mail.gmail.com>

Allan McRae wrote:
> I noticed this the other day when working on how to deal with options with
> spaces in them.   Is that the only change that needs made for PKGBUILDs with
> spaces to work?

Yes, besides the bug that the '-p' setting gets overriden when it
sources the makepkg.conf file. In other places where it uses
$BUILDSCRIPT it is already properly quoted.

From allan at archlinux.org  Fri Dec 26 02:55:54 2008
From: allan at archlinux.org (Allan McRae)
Date: Fri, 26 Dec 2008 17:55:54 +1000
Subject: [pacman-dev] [PATCH] makepkg: quote all uses of BUILDSCRIPT
Message-ID: <1230278154-5012-1-git-send-email-allan@archlinux.org>

Allows specifying alternative build script with spaces in name

Signed-off-by: Allan McRae <allan at archlinux.org>
---
 scripts/makepkg.sh.in |   10 +++++-----
 1 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in
index 1dc6415..a999e0b 100644
--- a/scripts/makepkg.sh.in
+++ b/scripts/makepkg.sh.in
@@ -1205,9 +1205,9 @@ devel_update() {
 	if [ "$newpkgver" != "" ]; then
 		if [ "$newpkgver" != "$pkgver" ]; then
 			if [ -f "./$BUILDSCRIPT" ]; then
-				sed -i "s/^pkgver=[^ ]*/pkgver=$newpkgver/" ./$BUILDSCRIPT
-				sed -i "s/^pkgrel=[^ ]*/pkgrel=1/" ./$BUILDSCRIPT
-				source $BUILDSCRIPT
+				sed -i "s/^pkgver=[^ ]*/pkgver=$newpkgver/" "./$BUILDSCRIPT"
+				sed -i "s/^pkgrel=[^ ]*/pkgrel=1/" "./$BUILDSCRIPT"
+				source "$BUILDSCRIPT"
 			fi
 		fi
 	fi
@@ -1496,7 +1496,7 @@ if [ "$CLEANCACHE" = "1" ]; then
 	fi
 fi
 
-if [ -z $BUILDSCRIPT ]; then
+if [ -z "$BUILDSCRIPT" ]; then
 	error "$(gettext "BUILDSCRIPT is undefined! Ensure you have updated %s.")" "$MAKEPKG_CONF"
 	exit 1
 fi
@@ -1556,7 +1556,7 @@ if [ ! -f "$BUILDSCRIPT" ]; then
 		BUILDSCRIPT=/dev/stdin
 	fi
 else
-	crlftest=$(file $BUILDSCRIPT | grep -F 'CRLF' || true)
+	crlftest=$(file "$BUILDSCRIPT" | grep -F 'CRLF' || true)
 	if [ "$crlftest" != "" ]; then
 		error "$(gettext "%s contains CRLF characters and cannot be sourced.")" "$BUILDSCRIPT"
 		exit 1
-- 
1.6.0.6


From dpmcgee at gmail.com  Fri Dec 26 15:40:11 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Fri, 26 Dec 2008 14:40:11 -0600
Subject: [pacman-dev] [PATCH] makepkg: quote all uses of BUILDSCRIPT
In-Reply-To: <1230278154-5012-1-git-send-email-allan@archlinux.org>
References: <1230278154-5012-1-git-send-email-allan@archlinux.org>
Message-ID: <449c10960812261240o2b013c66ga8b26b343b3d104@mail.gmail.com>

On Fri, Dec 26, 2008 at 1:55 AM, Allan McRae <allan at archlinux.org> wrote:
> Allows specifying alternative build script with spaces in name
>
> Signed-off-by: Allan McRae <allan at archlinux.org>
Signed-off-by: Dan McGee <dan at archlinux.org>

From allan at archlinux.org  Sat Dec 27 01:39:47 2008
From: allan at archlinux.org (Allan McRae)
Date: Sat, 27 Dec 2008 16:39:47 +1000
Subject: [pacman-dev] [PATCH] makepkg: move BUILDSCRIPT from makepkg.conf
Message-ID: <1230359987-27396-1-git-send-email-allan@archlinux.org>

Commit 4b183bf9 moved makepkg.conf sourcing to after the parsing
of options, broking the -p option and --help output.  The solution
is to move BUILDSCRIPT out of makepkg.conf.  This patch moves the
definition BUILDSCRIPT back to makepkg itself and adds configure
option to allow easy changing of this value during build time.

Signed-off-by: Allan McRae <allan at archlinux.org>
---
 configure.ac          |    9 +++++++++
 etc/makepkg.conf.in   |    3 +--
 scripts/Makefile.am   |    2 ++
 scripts/makepkg.sh.in |    1 +
 4 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/configure.ac b/configure.ac
index c7841b8..5f1a153 100644
--- a/configure.ac
+++ b/configure.ac
@@ -88,6 +88,11 @@ AC_ARG_WITH(db-ext,
 	AS_HELP_STRING([--with-db-ext=ext], [set the file extension used by the database]),
 	[DBEXT=$withval], [DBEXT=.db.tar.gz])
 
+# Help line for buildscript filename
+AC_ARG_WITH(buildscript,
+	AS_HELP_STRING([--with-buildscript=name], [set the build script name used by makepkg]),
+	[BUILDSCRIPT=$withval], [BUILDSCRIPT=PKGBUILD])
+
 # Help line for libdownload/libfetch
 AC_ARG_ENABLE(internal-download,
 	AS_HELP_STRING([--disable-internal-download], [do not build with libdownload/libfetch support]),
@@ -314,6 +319,9 @@ AC_DEFINE_UNQUOTED([SRCEXT], "$SRCEXT", [The file extension used by pacman sourc
 # Set database file extension
 AC_SUBST(DBEXT)
 AC_DEFINE_UNQUOTED([DBEXT], "$DBEXT", [The file extension used by pacman databases])
+# Set makepkg build script name
+AC_SUBST(BUILDSCRIPT)
+AC_DEFINE_UNQUOTED([BUILDSCRIPT], "$BUILDSCRIPT", [The build script name used by makepkg])
 
 # Configuration files
 AC_CONFIG_FILES([
@@ -362,6 +370,7 @@ ${PACKAGE_NAME}:
     package extension      : ${PKGEXT}
     source pkg extension   : ${SRCEXT}
     database extension     : ${DBEXT}
+    build script name      : ${BUILDSCRIPT}
 
   Compilation options:
     Run make in doc/ dir   : ${wantdoc}
diff --git a/etc/makepkg.conf.in b/etc/makepkg.conf.in
index 87b115c..9872d5d 100644
--- a/etc/makepkg.conf.in
+++ b/etc/makepkg.conf.in
@@ -95,13 +95,12 @@ PURGE_TARGETS=(usr/{,share}/info/dir .packlist *.pod)
 #PACKAGER="John Doe <john at doe.com>"
 
 #########################################################################
-# BUILDSCRIPT/EXTENSION DEFAULTS
+# EXTENSION DEFAULTS
 #########################################################################
 #
 # WARNING: Do NOT modify these variables unless you know what you are
 #          doing.
 #
-BUILDSCRIPT='PKGBUILD'
 PKGEXT='@PKGEXT@'
 SRCEXT='@SRCEXT@'
 
diff --git a/scripts/Makefile.am b/scripts/Makefile.am
index 014ae87..334b79a 100644
--- a/scripts/Makefile.am
+++ b/scripts/Makefile.am
@@ -37,9 +37,11 @@ edit = sed \
 	-e 's|@PACKAGE_BUGREPORT[@]|$(PACKAGE_BUGREPORT)|g' \
 	-e 's|@PACKAGE_NAME[@]|$(PACKAGE_NAME)|g' \
 	-e 's|@DBEXT[@]|$(DBEXT)|g' \
+	-e 's|@BUILDSCRIPT[@]|$(BUILDSCRIPT)|g' \
 	-e 's|@SIZECMD[@]|$(SIZECMD)|g' \
 	-e 's|@configure_input[@]|Generated from $@.in; do not edit by hand.|g'
 
+
 ## All the scripts depend on Makefile so that they are rebuilt when the
 ## prefix etc. changes. Use chmod -w to prevent people from editing the
 ## wrong file by accident.
diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in
index a999e0b..eae2c31 100644
--- a/scripts/makepkg.sh.in
+++ b/scripts/makepkg.sh.in
@@ -38,6 +38,7 @@ export COMMAND_MODE='legacy'
 
 myver='@PACKAGE_VERSION@'
 confdir='@sysconfdir@'
+BUILDSCRIPT='@BUILDSCRIPT@'
 startdir="$PWD"
 srcdir="$startdir/src"
 pkgdir="$startdir/pkg"
-- 
1.6.0.6


From roman.kyrylych at gmail.com  Sat Dec 27 02:58:59 2008
From: roman.kyrylych at gmail.com (Roman Kyrylych)
Date: Sat, 27 Dec 2008 09:58:59 +0200
Subject: [pacman-dev] small libdownload patch
In-Reply-To: <d64a48980811101035we70df16mcd4b915f4575e34c@mail.gmail.com>
References: <4ddb242c0811101024o733c3e46hda32de453a187f00@mail.gmail.com>
	<d64a48980811101035we70df16mcd4b915f4575e34c@mail.gmail.com>
Message-ID: <577f15290812262358x261588b8y1d6ed5c951d7862c@mail.gmail.com>

On Mon, Nov 10, 2008 at 20:35, Aaron Griffin <aaronmgriffin at gmail.com> wrote:
> On Mon, Nov 10, 2008 at 12:24 PM, Johannes Krampf
> <Johannes.Krampf at htw-dresden.de> wrote:
>> Hi,
>>
>> I've found a small compatibility problem and static checking a minor
>> buffer overflow in libdownload. Please excuse if this should already
>> be fixed in git.
>>
>> Here's the patch, <stdint.h> is included for uintptr_t and fscanf
>> writes a trailing \0, therefore requiring 1025 bytes in the worst
>> case:
>
> Just an FYI, NetBSD's libfetch code compiles on linux with a minor
> modification. So I think we're planning on switching to that.

...and that bug is fixed there long before in FreeBSD's libfetch.
Thanks for info, Aaron! I wasn't aware that NetBSD has their own libfetch.
Looks like NetBSD's libfetch was heavily reworked in some places,
including some new features added.
The only thing it's missing from FreeBSD's lbfetch is
support for HTTP 1.1 If-Modified-Since behavior
(which was only added less than 2 weeks ago).

There's one Linux-compatability fix after 2.20:
http://cvsweb.se.netbsd.org/cgi-bin/bsdweb.cgi/pkgsrc/net/libfetch/files/ftp.c.diff?r1=1.24;r2=1.25
Not sure if it's important enough for us though.

-- 
Roman Kyrylych (????? ???????)

From shiningxc at gmail.com  Sat Dec 27 06:15:49 2008
From: shiningxc at gmail.com (Xavier)
Date: Sat, 27 Dec 2008 12:15:49 +0100
Subject: [pacman-dev] small libdownload patch
In-Reply-To: <577f15290812262358x261588b8y1d6ed5c951d7862c@mail.gmail.com>
References: <4ddb242c0811101024o733c3e46hda32de453a187f00@mail.gmail.com>
	<d64a48980811101035we70df16mcd4b915f4575e34c@mail.gmail.com>
	<577f15290812262358x261588b8y1d6ed5c951d7862c@mail.gmail.com>
Message-ID: <91752840812270315x24452da1t37f56761060f6b6d@mail.gmail.com>

On Sat, Dec 27, 2008 at 8:58 AM, Roman Kyrylych
<roman.kyrylych at gmail.com> wrote:
>
> ...and that bug is fixed there long before in FreeBSD's libfetch.
> Thanks for info, Aaron! I wasn't aware that NetBSD has their own libfetch.
> Looks like NetBSD's libfetch was heavily reworked in some places,
> including some new features added.

Indeed NetBSD's libfetch looked pretty good last time I checked.

> The only thing it's missing from FreeBSD's lbfetch is
> support for HTTP 1.1 If-Modified-Since behavior
> (which was only added less than 2 weeks ago).
>

I believe they merge the changes from times to times, so it should be
included eventually, we just need to be patient :)

> There's one Linux-compatability fix after 2.20:
> http://cvsweb.se.netbsd.org/cgi-bin/bsdweb.cgi/pkgsrc/net/libfetch/files/ftp.c.diff?r1=1.24;r2=1.25
> Not sure if it's important enough for us though.
>

That is very cool, it is actually the only patch we needed :
http://code.phraktured.net/?p=libfetch.git;a=shortlog;h=refs/heads/linux
The only thing left is to provide a linux Makefile, which Aaron already did.

From sebnow at gmail.com  Sat Dec 27 09:46:13 2008
From: sebnow at gmail.com (Sebastian Nowicki)
Date: Sat, 27 Dec 2008 23:46:13 +0900
Subject: [pacman-dev] alpm_db_get_pkgcache and alpm_db_get_grpcache rename
Message-ID: <9564AD3D-81E0-4E49-83AE-284569DA3F88@gmail.com>

Hi,

Recently the alpm_db_get_{pkg,grp}cache() functions were renamed to  
alpm_db_get_{pkg,grp}cache()[1]. Since this is a public API change,  
shouldn't the old functions perhaps simply be deprecated and "alias"  
the new functions, instead of dropping them completely? I'd imagine  
this would potentially break some front-ends if libalpm gets updated,  
as was the case with a project of mine.

[1]: commit: d05882db9e417244fa580c4697b45333faffcc79


From shiningxc at gmail.com  Sat Dec 27 09:53:43 2008
From: shiningxc at gmail.com (Xavier)
Date: Sat, 27 Dec 2008 15:53:43 +0100
Subject: [pacman-dev] alpm_db_get_pkgcache and alpm_db_get_grpcache
	rename
In-Reply-To: <9564AD3D-81E0-4E49-83AE-284569DA3F88@gmail.com>
References: <9564AD3D-81E0-4E49-83AE-284569DA3F88@gmail.com>
Message-ID: <91752840812270653q4b2322dna963679c77aa82c9@mail.gmail.com>

On Sat, Dec 27, 2008 at 3:46 PM, Sebastian Nowicki <sebnow at gmail.com> wrote:
> Hi,
>
> Recently the alpm_db_get_{pkg,grp}cache() functions were renamed to
> alpm_db_get_{pkg,grp}cache()[1]. Since this is a public API change,
> shouldn't the old functions perhaps simply be deprecated and "alias" the new
> functions, instead of dropping them completely? I'd imagine this would
> potentially break some front-ends if libalpm gets updated, as was the case
> with a project of mine.
>
> [1]: commit: d05882db9e417244fa580c4697b45333faffcc79
>

This will only appear in 3.3 which does not even have any clear
roadmap so which looks quite far away from now.
It won't be in the next 3.2.2 release.

From shiningxc at gmail.com  Sat Dec 27 13:29:31 2008
From: shiningxc at gmail.com (Xavier)
Date: Sat, 27 Dec 2008 19:29:31 +0100
Subject: [pacman-dev] Aqpm, and some help needed
In-Reply-To: <200811200043.02591.drf54321@gmail.com>
References: <200811200043.02591.drf54321@gmail.com>
Message-ID: <91752840812271029q59ea4ec2l654de51137595869@mail.gmail.com>

On Thu, Nov 20, 2008 at 12:43 AM, Dario Freddi <drf54321 at gmail.com> wrote:
> Hello guys,
>
> I'm introducing you Aqpm, a QT/C++ wrapper to libalpm. I did this since Shaman
> backend was starting to get too much bloated, and since I needed to use Alpm
> in some other apps (and in Shaman 2 too).
>
> What's cool about it: I aim at a strong binary compatibility. This way, even
> when alpm introduces some binary-incompatible changes, modifying Aqpm
> internals will be enough to get all applications relying on Aqpm work again
> without a rebuild. This way we'll also avoid bad situation like Shaman crack
> this summer.
>
> It also wraps around the callback system and implements an easier Queue. It
> has solid internal threading where needed, and I'm planning to make it support
> PolicyKit, so that I'll also get rid of the problem of running Shaman with
> suid and other apps using Alpm running as root (even though Policykit could be
> directly added to alpm itself, I don't know).
>
> Enough talking, it's here: http://github.com/drf/aqpm/tree/master. If you want
> to test it, I moved Shaman to github and I have a branch there working with
> aqpm: http://github.com/drf/shaman1/tree/aqpm-work . Sure, it has some
> regressions, but really small stuff. Please note that Aqpm at the current
> status supports latest git revision of alpm, so it is not compilable with
> current stable pacman.
>
> I am writing you also to ask you for some help. I have one problem since
> libalpm 3, namely the download progress callback streaming always 0 in the
> "total" parameter. I got almost crazy over it, and I still can't find a reason
> why it does not work. I verified that when Alpm streams the callback, the
> value is correct, though when it gets received it becomes 0.
>
> So I kindly ask you a small review of my code, since I really don't know where
> to crash my head. It is the last missing bit to aqpm to work 100%, so I'd
> really appreciate if you could give some help. The relevant code is at:
>
> http://github.com/drf/aqpm/tree/master/trunk/libaqpm/Backend.cpp line 797-1010
> (Transaction thread, but in that file you'll find every action aqpm does)
>
> http://github.com/drf/aqpm/tree/master/trunk/libaqpm/callbacks.cpp Callbacks
> are here.
>
> Also some feedbacks/suggestions would be highly appreciated
>

In answer to http://bbs.archlinux.org/viewtopic.php?id=61794 :

You were not ignored, it is just that you mentioned this error to us
in the past :
http://bugs.archlinux.org/task/11355
The conclusion seemed to be that the bug was not in libalpm and that
you would find the error yourself. Apparently this was not the case :)

I am completely clueless about what could go wrong there. I had a look
several times at your code and never saw anything wrong. Though I
never did more than reading the code. Also you said that alpm
apparently gives the correct value, so it seems there is not much we
can do from our side. But then I have no idea how it could get lost...

Do you have this problem on both architectures? 686 and x86_64?
Also you could maybe try to play with different types on the libalpm
level. changing off_t to size_t or things like that.

One funny thing I noticed is that the call is the following :
handle->dlcb(filename, dl_thisfile, ust.size);
with dl_thisfile size_t and ust.size off_t
and the type of the function is this :
typedef void (*alpm_cb_download)(const char *filename,
off_t xfered, off_t total);

So we have a type mismatch for the xfered parameter, but not for the
total parameter. Yet you only have problems with total :P So it is
probably not the problem, but it is still something worth playing
with.

From drf54321 at gmail.com  Sat Dec 27 14:28:27 2008
From: drf54321 at gmail.com (Dario Freddi)
Date: Sat, 27 Dec 2008 20:28:27 +0100
Subject: [pacman-dev] Aqpm, and some help needed
In-Reply-To: <91752840812271029q59ea4ec2l654de51137595869@mail.gmail.com>
References: <200811200043.02591.drf54321@gmail.com>
	<91752840812271029q59ea4ec2l654de51137595869@mail.gmail.com>
Message-ID: <200812272028.27519.drf54321@gmail.com>

On Saturday 27 December 2008 19:29:31 Xavier wrote:
>
> In answer to http://bbs.archlinux.org/viewtopic.php?id=61794 :
>
> You were not ignored, it is just that you mentioned this error to us
> in the past :
> http://bugs.archlinux.org/task/11355
> The conclusion seemed to be that the bug was not in libalpm and that
> you would find the error yourself. Apparently this was not the case :)

Thanks for the answer. I apologize for such an irruent reaction anyway, 
realized that just after the post, so sorry once again.

>
> I am completely clueless about what could go wrong there. I had a look
> several times at your code and never saw anything wrong. Though I
> never did more than reading the code. Also you said that alpm
> apparently gives the correct value, so it seems there is not much we
> can do from our side. But then I have no idea how it could get lost...

What I can try to do is attach gdb and see what happens to that value.

>
> Do you have this problem on both architectures? 686 and x86_64?

Yeah

> Also you could maybe try to play with different types on the libalpm
> level. changing off_t to size_t or things like that.

I'll surely try this now.

>
> One funny thing I noticed is that the call is the following :
> handle->dlcb(filename, dl_thisfile, ust.size);
> with dl_thisfile size_t and ust.size off_t
> and the type of the function is this :
> typedef void (*alpm_cb_download)(const char *filename,
> off_t xfered, off_t total);
>
> So we have a type mismatch for the xfered parameter, but not for the
> total parameter. Yet you only have problems with total :P So it is
> probably not the problem, but it is still something worth playing
> with.

I really didn't notice that... it definitely could be. I'll try again and 
report as soon as I find out something.

Cheers
Dario

> _______________________________________________
> pacman-dev mailing list
> pacman-dev at archlinux.org
> http://archlinux.org/mailman/listinfo/pacman-dev


From drf54321 at gmail.com  Sat Dec 27 14:34:02 2008
From: drf54321 at gmail.com (Dario Freddi)
Date: Sat, 27 Dec 2008 20:34:02 +0100
Subject: [pacman-dev] Aqpm, and some help needed
In-Reply-To: <91752840812271029q59ea4ec2l654de51137595869@mail.gmail.com>
References: <200811200043.02591.drf54321@gmail.com>
	<91752840812271029q59ea4ec2l654de51137595869@mail.gmail.com>
Message-ID: <200812272034.03266.drf54321@gmail.com>

On Saturday 27 December 2008 19:29:31 Xavier wrote:
>
> One funny thing I noticed is that the call is the following :
> handle->dlcb(filename, dl_thisfile, ust.size);
> with dl_thisfile size_t and ust.size off_t
> and the type of the function is this :
> typedef void (*alpm_cb_download)(const char *filename,
> off_t xfered, off_t total);
>
> So we have a type mismatch for the xfered parameter, but not for the
> total parameter. Yet you only have problems with total :P So it is
> probably not the problem, but it is still something worth playing
> with.

Ok, after having a small round with it I found out that types are not 
compatible, so this have to be played into libalpm. I think this could be the 
only issue, since back in the days, when the function had standard types, 
everything worked. I'll try again, expect a patch in the case

> _______________________________________________
> pacman-dev mailing list
> pacman-dev at archlinux.org
> http://archlinux.org/mailman/listinfo/pacman-dev


From sebnow at gmail.com  Sat Dec 27 22:14:50 2008
From: sebnow at gmail.com (Sebastian Nowicki)
Date: Sun, 28 Dec 2008 12:14:50 +0900
Subject: [pacman-dev] alpm_db_get_pkgcache and alpm_db_get_grpcache
	rename
In-Reply-To: <91752840812270653q4b2322dna963679c77aa82c9@mail.gmail.com>
References: <9564AD3D-81E0-4E49-83AE-284569DA3F88@gmail.com>
	<91752840812270653q4b2322dna963679c77aa82c9@mail.gmail.com>
Message-ID: <D819E767-EFAA-4804-9A48-40936F6648BC@gmail.com>

On 27/12/2008, at 11:53 PM, Xavier wrote:

> On Sat, Dec 27, 2008 at 3:46 PM, Sebastian Nowicki  
> <sebnow at gmail.com> wrote:
>> Hi,
>>
>> Recently the alpm_db_get_{pkg,grp}cache() functions were renamed to
>> alpm_db_get_{pkg,grp}cache()[1]. Since this is a public API change,
>> shouldn't the old functions perhaps simply be deprecated and  
>> "alias" the new
>> functions, instead of dropping them completely? I'd imagine this  
>> would
>> potentially break some front-ends if libalpm gets updated, as was  
>> the case
>> with a project of mine.
>>
>> [1]: commit: d05882db9e417244fa580c4697b45333faffcc79
>>
>
> This will only appear in 3.3 which does not even have any clear
> roadmap so which looks quite far away from now.
> It won't be in the next 3.2.2 release.

There would probably still need to be some transition stage. As long  
as you guys are aware of the potential problem, I don't really mind ;).

From aaronmgriffin at gmail.com  Mon Dec 29 11:24:22 2008
From: aaronmgriffin at gmail.com (Aaron Griffin)
Date: Mon, 29 Dec 2008 10:24:22 -0600
Subject: [pacman-dev] Aqpm, and some help needed
In-Reply-To: <200812272034.03266.drf54321@gmail.com>
References: <200811200043.02591.drf54321@gmail.com>
	<91752840812271029q59ea4ec2l654de51137595869@mail.gmail.com>
	<200812272034.03266.drf54321@gmail.com>
Message-ID: <d64a48980812290824v59a830ar2b0f0c9ddcf5d3b4@mail.gmail.com>

On Sat, Dec 27, 2008 at 1:34 PM, Dario Freddi <drf54321 at gmail.com> wrote:
> On Saturday 27 December 2008 19:29:31 Xavier wrote:
>>
>> One funny thing I noticed is that the call is the following :
>> handle->dlcb(filename, dl_thisfile, ust.size);
>> with dl_thisfile size_t and ust.size off_t
>> and the type of the function is this :
>> typedef void (*alpm_cb_download)(const char *filename,
>> off_t xfered, off_t total);
>>
>> So we have a type mismatch for the xfered parameter, but not for the
>> total parameter. Yet you only have problems with total :P So it is
>> probably not the problem, but it is still something worth playing
>> with.
>
> Ok, after having a small round with it I found out that types are not
> compatible, so this have to be played into libalpm. I think this could be the
> only issue, since back in the days, when the function had standard types,
> everything worked. I'll try again, expect a patch in the case

To clarify something - I'm fairly certain off_t is specified in the C
standard, so this function _still_ has standard types

From aaronmgriffin at gmail.com  Mon Dec 29 14:45:43 2008
From: aaronmgriffin at gmail.com (Aaron Griffin)
Date: Mon, 29 Dec 2008 13:45:43 -0600
Subject: [pacman-dev] small libdownload patch
In-Reply-To: <91752840812270315x24452da1t37f56761060f6b6d@mail.gmail.com>
References: <4ddb242c0811101024o733c3e46hda32de453a187f00@mail.gmail.com>
	<d64a48980811101035we70df16mcd4b915f4575e34c@mail.gmail.com>
	<577f15290812262358x261588b8y1d6ed5c951d7862c@mail.gmail.com>
	<91752840812270315x24452da1t37f56761060f6b6d@mail.gmail.com>
Message-ID: <d64a48980812291145l7137a2aqbcd8eabe3ca787b9@mail.gmail.com>

On Sat, Dec 27, 2008 at 5:15 AM, Xavier <shiningxc at gmail.com> wrote:
> On Sat, Dec 27, 2008 at 8:58 AM, Roman Kyrylych
> <roman.kyrylych at gmail.com> wrote:
>>
>> ...and that bug is fixed there long before in FreeBSD's libfetch.
>> Thanks for info, Aaron! I wasn't aware that NetBSD has their own libfetch.
>> Looks like NetBSD's libfetch was heavily reworked in some places,
>> including some new features added.
>
> Indeed NetBSD's libfetch looked pretty good last time I checked.
>
>> The only thing it's missing from FreeBSD's lbfetch is
>> support for HTTP 1.1 If-Modified-Since behavior
>> (which was only added less than 2 weeks ago).
>>
>
> I believe they merge the changes from times to times, so it should be
> included eventually, we just need to be patient :)
>
>> There's one Linux-compatability fix after 2.20:
>> http://cvsweb.se.netbsd.org/cgi-bin/bsdweb.cgi/pkgsrc/net/libfetch/files/ftp.c.diff?r1=1.24;r2=1.25
>> Not sure if it's important enough for us though.
>>
>
> That is very cool, it is actually the only patch we needed :
> http://code.phraktured.net/?p=libfetch.git;a=shortlog;h=refs/heads/linux
> The only thing left is to provide a linux Makefile, which Aaron already did.

Actually, that patch isn't even needed if we build with
CFLAGS="-D_GNU_SOURCE". It still should be merged upstream though,
I'll poke the maintainer at some point.

From aaronmgriffin at gmail.com  Mon Dec 29 14:48:09 2008
From: aaronmgriffin at gmail.com (Aaron Griffin)
Date: Mon, 29 Dec 2008 13:48:09 -0600
Subject: [pacman-dev] alpm_db_get_pkgcache and alpm_db_get_grpcache
	rename
In-Reply-To: <D819E767-EFAA-4804-9A48-40936F6648BC@gmail.com>
References: <9564AD3D-81E0-4E49-83AE-284569DA3F88@gmail.com>
	<91752840812270653q4b2322dna963679c77aa82c9@mail.gmail.com>
	<D819E767-EFAA-4804-9A48-40936F6648BC@gmail.com>
Message-ID: <d64a48980812291148g15508eb3ha296e3d7f5b3e337@mail.gmail.com>

On Sat, Dec 27, 2008 at 9:14 PM, Sebastian Nowicki <sebnow at gmail.com> wrote:
> On 27/12/2008, at 11:53 PM, Xavier wrote:
>
>> On Sat, Dec 27, 2008 at 3:46 PM, Sebastian Nowicki <sebnow at gmail.com>
>> wrote:
>>>
>>> Hi,
>>>
>>> Recently the alpm_db_get_{pkg,grp}cache() functions were renamed to
>>> alpm_db_get_{pkg,grp}cache()[1]. Since this is a public API change,
>>> shouldn't the old functions perhaps simply be deprecated and "alias" the
>>> new
>>> functions, instead of dropping them completely? I'd imagine this would
>>> potentially break some front-ends if libalpm gets updated, as was the
>>> case
>>> with a project of mine.
>>>
>>> [1]: commit: d05882db9e417244fa580c4697b45333faffcc79
>>>
>>
>> This will only appear in 3.3 which does not even have any clear
>> roadmap so which looks quite far away from now.
>> It won't be in the next 3.2.2 release.
>
> There would probably still need to be some transition stage. As long as you
> guys are aware of the potential problem, I don't really mind ;).

If we stick a note in the ChangeLog, that's usually enough for most
projects. "API changed. Fix your crap". Lots of other projects do
that.

From shiningxc at gmail.com  Mon Dec 29 17:40:33 2008
From: shiningxc at gmail.com (Xavier)
Date: Mon, 29 Dec 2008 23:40:33 +0100
Subject: [pacman-dev] small libdownload patch
In-Reply-To: <d64a48980812291145l7137a2aqbcd8eabe3ca787b9@mail.gmail.com>
References: <4ddb242c0811101024o733c3e46hda32de453a187f00@mail.gmail.com>
	<d64a48980811101035we70df16mcd4b915f4575e34c@mail.gmail.com>
	<577f15290812262358x261588b8y1d6ed5c951d7862c@mail.gmail.com>
	<91752840812270315x24452da1t37f56761060f6b6d@mail.gmail.com>
	<d64a48980812291145l7137a2aqbcd8eabe3ca787b9@mail.gmail.com>
Message-ID: <91752840812291440s65d46263l1e57391f5c790097@mail.gmail.com>

On Mon, Dec 29, 2008 at 8:45 PM, Aaron Griffin <aaronmgriffin at gmail.com> wrote:
> On Sat, Dec 27, 2008 at 5:15 AM, Xavier <shiningxc at gmail.com> wrote:
>> On Sat, Dec 27, 2008 at 8:58 AM, Roman Kyrylych
>> <roman.kyrylych at gmail.com> wrote:
>>>
>>> ...and that bug is fixed there long before in FreeBSD's libfetch.
>>> Thanks for info, Aaron! I wasn't aware that NetBSD has their own libfetch.
>>> Looks like NetBSD's libfetch was heavily reworked in some places,
>>> including some new features added.
>>
>> Indeed NetBSD's libfetch looked pretty good last time I checked.
>>
>>> The only thing it's missing from FreeBSD's lbfetch is
>>> support for HTTP 1.1 If-Modified-Since behavior
>>> (which was only added less than 2 weeks ago).
>>>
>>
>> I believe they merge the changes from times to times, so it should be
>> included eventually, we just need to be patient :)
>>
>>> There's one Linux-compatability fix after 2.20:
>>> http://cvsweb.se.netbsd.org/cgi-bin/bsdweb.cgi/pkgsrc/net/libfetch/files/ftp.c.diff?r1=1.24;r2=1.25
>>> Not sure if it's important enough for us though.
>>>
>>
>> That is very cool, it is actually the only patch we needed :
>> http://code.phraktured.net/?p=libfetch.git;a=shortlog;h=refs/heads/linux
>> The only thing left is to provide a linux Makefile, which Aaron already did.
>
> Actually, that patch isn't even needed if we build with
> CFLAGS="-D_GNU_SOURCE". It still should be merged upstream though,
> I'll poke the maintainer at some point.

Maybe my sentence was a bit confusing.
What Roman just showed is that this patch was just merged in netbsd upstream.

From aaronmgriffin at gmail.com  Mon Dec 29 17:46:15 2008
From: aaronmgriffin at gmail.com (Aaron Griffin)
Date: Mon, 29 Dec 2008 16:46:15 -0600
Subject: [pacman-dev] small libdownload patch
In-Reply-To: <91752840812291440s65d46263l1e57391f5c790097@mail.gmail.com>
References: <4ddb242c0811101024o733c3e46hda32de453a187f00@mail.gmail.com>
	<d64a48980811101035we70df16mcd4b915f4575e34c@mail.gmail.com>
	<577f15290812262358x261588b8y1d6ed5c951d7862c@mail.gmail.com>
	<91752840812270315x24452da1t37f56761060f6b6d@mail.gmail.com>
	<d64a48980812291145l7137a2aqbcd8eabe3ca787b9@mail.gmail.com>
	<91752840812291440s65d46263l1e57391f5c790097@mail.gmail.com>
Message-ID: <d64a48980812291446t2c4234ddqaed435770d2ce559@mail.gmail.com>

On Mon, Dec 29, 2008 at 4:40 PM, Xavier <shiningxc at gmail.com> wrote:
> On Mon, Dec 29, 2008 at 8:45 PM, Aaron Griffin <aaronmgriffin at gmail.com> wrote:
>> On Sat, Dec 27, 2008 at 5:15 AM, Xavier <shiningxc at gmail.com> wrote:
>>> On Sat, Dec 27, 2008 at 8:58 AM, Roman Kyrylych
>>> <roman.kyrylych at gmail.com> wrote:
>>>>
>>>> ...and that bug is fixed there long before in FreeBSD's libfetch.
>>>> Thanks for info, Aaron! I wasn't aware that NetBSD has their own libfetch.
>>>> Looks like NetBSD's libfetch was heavily reworked in some places,
>>>> including some new features added.
>>>
>>> Indeed NetBSD's libfetch looked pretty good last time I checked.
>>>
>>>> The only thing it's missing from FreeBSD's lbfetch is
>>>> support for HTTP 1.1 If-Modified-Since behavior
>>>> (which was only added less than 2 weeks ago).
>>>>
>>>
>>> I believe they merge the changes from times to times, so it should be
>>> included eventually, we just need to be patient :)
>>>
>>>> There's one Linux-compatability fix after 2.20:
>>>> http://cvsweb.se.netbsd.org/cgi-bin/bsdweb.cgi/pkgsrc/net/libfetch/files/ftp.c.diff?r1=1.24;r2=1.25
>>>> Not sure if it's important enough for us though.
>>>>
>>>
>>> That is very cool, it is actually the only patch we needed :
>>> http://code.phraktured.net/?p=libfetch.git;a=shortlog;h=refs/heads/linux
>>> The only thing left is to provide a linux Makefile, which Aaron already did.
>>
>> Actually, that patch isn't even needed if we build with
>> CFLAGS="-D_GNU_SOURCE". It still should be merged upstream though,
>> I'll poke the maintainer at some point.
>
> Maybe my sentence was a bit confusing.
> What Roman just showed is that this patch was just merged in netbsd upstream.

Ah my fault. I misread. I looked at the link and thought that was the
_existing_ patch. At the time I sent the patch upstream, the define
was in one file (http.c) but not the other (ftp.c). I just confused
the two files 8)

So now all we need is a makefile, which I have already. Yay

From dpmcgee at gmail.com  Mon Dec 29 18:28:35 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Mon, 29 Dec 2008 17:28:35 -0600
Subject: [pacman-dev] alpm_db_get_pkgcache and alpm_db_get_grpcache
	rename
In-Reply-To: <d64a48980812291148g15508eb3ha296e3d7f5b3e337@mail.gmail.com>
References: <9564AD3D-81E0-4E49-83AE-284569DA3F88@gmail.com>
	<91752840812270653q4b2322dna963679c77aa82c9@mail.gmail.com>
	<D819E767-EFAA-4804-9A48-40936F6648BC@gmail.com>
	<d64a48980812291148g15508eb3ha296e3d7f5b3e337@mail.gmail.com>
Message-ID: <449c10960812291528u6bbad8dbg155c950d486c6a6e@mail.gmail.com>

On Mon, Dec 29, 2008 at 1:48 PM, Aaron Griffin <aaronmgriffin at gmail.com> wrote:
> On Sat, Dec 27, 2008 at 9:14 PM, Sebastian Nowicki <sebnow at gmail.com> wrote:
>> On 27/12/2008, at 11:53 PM, Xavier wrote:
>>
>>> On Sat, Dec 27, 2008 at 3:46 PM, Sebastian Nowicki <sebnow at gmail.com>
>>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> Recently the alpm_db_get_{pkg,grp}cache() functions were renamed to
>>>> alpm_db_get_{pkg,grp}cache()[1]. Since this is a public API change,
>>>> shouldn't the old functions perhaps simply be deprecated and "alias" the
>>>> new
>>>> functions, instead of dropping them completely? I'd imagine this would
>>>> potentially break some front-ends if libalpm gets updated, as was the
>>>> case
>>>> with a project of mine.
>>>>
>>>> [1]: commit: d05882db9e417244fa580c4697b45333faffcc79
>>>>
>>>
>>> This will only appear in 3.3 which does not even have any clear
>>> roadmap so which looks quite far away from now.
>>> It won't be in the next 3.2.2 release.
>>
>> There would probably still need to be some transition stage. As long as you
>> guys are aware of the potential problem, I don't really mind ;).
>
> If we stick a note in the ChangeLog, that's usually enough for most
> projects. "API changed. Fix your crap". Lots of other projects do
> that.

I've always stuck with the minor releases don't change the API, while
major releases probably will. I also ensure our libtool version number
incrementing happens, so you will have to rebuild anything that links
against pacman anyway do to an SO number bump.

This has happened before and will surely continue to happen,
especially if we get a true transactional API anytime soon.

-Dan

From sebnow at gmail.com  Tue Dec 30 11:17:05 2008
From: sebnow at gmail.com (Sebastian Nowicki)
Date: Wed, 31 Dec 2008 01:17:05 +0900
Subject: [pacman-dev] alpm_db_get_pkgcache and alpm_db_get_grpcache
	rename
In-Reply-To: <d64a48980812291148g15508eb3ha296e3d7f5b3e337@mail.gmail.com>
References: <9564AD3D-81E0-4E49-83AE-284569DA3F88@gmail.com>
	<91752840812270653q4b2322dna963679c77aa82c9@mail.gmail.com>
	<D819E767-EFAA-4804-9A48-40936F6648BC@gmail.com>
	<d64a48980812291148g15508eb3ha296e3d7f5b3e337@mail.gmail.com>
Message-ID: <100A9CAC-5094-4120-B8F9-8A4D618E092D@gmail.com>

On 30/12/2008, at 4:48 AM, Aaron Griffin wrote:

> If we stick a note in the ChangeLog, that's usually enough for most
> projects. "API changed. Fix your crap". Lots of other projects do
> that.

Indeed, it's no biggy, even a global sed would fix it :)

On 30/12/2008, at 8:28 AM, Dan McGee wrote:

> This has happened before and will surely continue to happen,
> especially if we get a true transactional API anytime soon.
>
> -Dan

Looking forward to it.


From bji-keyword-pacman.3644cb at www.ischo.com  Tue Dec 30 18:22:40 2008
From: bji-keyword-pacman.3644cb at www.ischo.com (Bryan Ischo)
Date: Wed, 31 Dec 2008 12:22:40 +1300
Subject: [pacman-dev] AUR pacman package fixes bug 9395 - comments?
Message-ID: <495AAD40.20000@www.ischo.com>

Hi all.  I'm sorry I didn't post to this list earlier about this issue; 
I kind of forgot that there was a pacman-dev list and I added my 
comments to the bug, not realizing that I should post here too.

Anyway, I have created a patch for pacman 3.2.1 which changes the 
behavior of pacman when it encounters unresolvable dependencies during a 
sync.

The current behavior is that any time a package to be upgraded has a 
dependency that cannot be resolved, pacman asks if the user would like 
to ignore the missing dependency and install the package anyway.  If the 
user answers yes, the sync proceeds, resulting in broken dependencies.  
If the user answers no, then the sync fails immediately.

This situation can be quite common if you use the IgnorePkg/IgnoreGroup 
feature, because any package which depends on an upgrade to an ignored 
package cannot be sync'd, and thus the user is forced to ignore more and 
more packages to prevent sync -Su from being impossible to complete.

My patch causes pacman to, instead of issuing a force/fail question when 
an unresolvable dependency is discovered, keep track of which packages 
that it would upgrade cannot be upgraded for this reason, and issue a 
single question to the user at the end of the package resolution step, 
of this form:

:: the following packages cannot be upgraded due to unresolvable 
dependencies:
        package1,
        package2,
        package3,
        etc.

Do you want to skip these packages for this upgrade? [Y/n]

If the user answers yes, then the offending packages will simply be 
removed from the sync operation and the remainder of the sync can occur 
as normal.  If the user answers no, then the sync cannot proceed, and it 
fails.

So this changes the "force/fail" question into a "skip/fail" question, 
which I think is better.  In fact, I can't think of any disadvantages to 
doing it this way - can you?

I don't have a patch to submit for this; instead I've created a patch as 
part of an AUR submission that produces a new version of pacman with the 
behavior I have described.  I think this is an easier way for users to 
test this feature.  But I would be more than happy to submit a patch to 
this list directly for this issue if it is the preferred means of 
getting this change into the official pacman sources.  A question I have 
though, is exactly what should be patched?  The NEWS file, configure.ac, 
etc?  Or just the code?  I ask because my patch introduces what I 
believe to be an incompatibility in libalpm with the previous version 
(because a new "question" has been added, that amounts to an addition to 
the API), which requires a new libalpm version number, so there are lots 
of files in the pacman source that need to be patched to effect this.  
Should I submit a patch to this list that takes pacman-3.2.1 to 
pacman-3.3.0 (with libalpm 4.0)?  Or should I just submit the actual 
header and source file changes (plus man page and other documentation 
changes, I am sure) in a patch to this list?

Any questions, concerns, comments, or other feedback about this patch 
would be most appreciated.

The bug in question is at:   http://bugs.archlinux.org/task/9395
The AUR package that I have created is at:   
http://aur.archlinux.org/packages.php?ID=11182

Thanks, and best wishes,
Bryan





From louipc.ist at gmail.com  Tue Dec 30 19:07:03 2008
From: louipc.ist at gmail.com (Loui Chang)
Date: Tue, 30 Dec 2008 19:07:03 -0500
Subject: [pacman-dev] AUR pacman package fixes bug 9395 - comments?
In-Reply-To: <495AAD40.20000@www.ischo.com>
References: <495AAD40.20000@www.ischo.com>
Message-ID: <20081231000703.GC2161@lynn>

>  The AUR package that I have created is at:   
>  http://aur.archlinux.org/packages.php?ID=11182

Hey Bryan,
Unsupported packages should not share the same name as official
packages, so I've deleted it.

There'd be no problem if you called it 'pacman-ischo' or something, to
denote that it's your fork of pacman.

Also, it's a good idea to include your patch on the list so it may be
discussed.


From bji-keyword-pacman.3644cb at www.ischo.com  Tue Dec 30 19:13:36 2008
From: bji-keyword-pacman.3644cb at www.ischo.com (Bryan Ischo)
Date: Wed, 31 Dec 2008 13:13:36 +1300
Subject: [pacman-dev] AUR pacman package fixes bug 9395 - comments?
In-Reply-To: <20081231000703.GC2161@lynn>
References: <495AAD40.20000@www.ischo.com> <20081231000703.GC2161@lynn>
Message-ID: <495AB930.8010104@www.ischo.com>

Loui Chang wrote:
>>  The AUR package that I have created is at:   
>>  http://aur.archlinux.org/packages.php?ID=11182
>>     
>
> Hey Bryan,
> Unsupported packages should not share the same name as official
> packages, so I've deleted it.
>
> There'd be no problem if you called it 'pacman-ischo' or something, to
> denote that it's your fork of pacman.
>
> Also, it's a good idea to include your patch on the list so it may be
> discussed.
>   
Hi.  Sorry about the invalid package I submitted to AUR, it was my first 
package submission and I did not realize the many ways in which it 
violates AUR policy :)  I agree that the best way is for me to change 
the name of the package, and I will re-submit a changed package to AUR 
(my purpose being to allow end-users to test my change easily).

I just got the git sources for pacman, and have applied my patch to it, 
which succeeded on the source/header files but failed miserably on many 
other files that are apparently specific to the release version of the 
source.  I will re-work my patch to fit properly into the git version of 
the source, and to include all documentation updates and such, but not 
to include changes to the pacman or libalpm versions (I will let a real 
developer handle those).  I'm very new to git so I have some reading to 
do ...

Thanks!
Bryan


From allan at archlinux.org  Tue Dec 30 19:17:58 2008
From: allan at archlinux.org (Allan McRae)
Date: Wed, 31 Dec 2008 10:17:58 +1000
Subject: [pacman-dev] AUR pacman package fixes bug 9395 - comments?
In-Reply-To: <495AAD40.20000@www.ischo.com>
References: <495AAD40.20000@www.ischo.com>
Message-ID: <BAY102-DAV13C95B79DA9E5EB66D7DE286E40@phx.gbl>

Bryan Ischo wrote:
> Hi all.  I'm sorry I didn't post to this list earlier about this 
> issue; I kind of forgot that there was a pacman-dev list and I added 
> my comments to the bug, not realizing that I should post here too.
>
> Anyway, I have created a patch for pacman 3.2.1 which changes the 
> behavior of pacman when it encounters unresolvable dependencies during 
> a sync.
>
> The current behavior is that any time a package to be upgraded has a 
> dependency that cannot be resolved, pacman asks if the user would like 
> to ignore the missing dependency and install the package anyway.  If 
> the user answers yes, the sync proceeds, resulting in broken 
> dependencies.  If the user answers no, then the sync fails immediately.
>
> This situation can be quite common if you use the 
> IgnorePkg/IgnoreGroup feature, because any package which depends on an 
> upgrade to an ignored package cannot be sync'd, and thus the user is 
> forced to ignore more and more packages to prevent sync -Su from being 
> impossible to complete.
>
> My patch causes pacman to, instead of issuing a force/fail question 
> when an unresolvable dependency is discovered, keep track of which 
> packages that it would upgrade cannot be upgraded for this reason, and 
> issue a single question to the user at the end of the package 
> resolution step, of this form:
>
> :: the following packages cannot be upgraded due to unresolvable 
> dependencies:
>        package1,
>        package2,
>        package3,
>        etc.
>
> Do you want to skip these packages for this upgrade? [Y/n]
>
> If the user answers yes, then the offending packages will simply be 
> removed from the sync operation and the remainder of the sync can 
> occur as normal.  If the user answers no, then the sync cannot 
> proceed, and it fails.
>
> So this changes the "force/fail" question into a "skip/fail" question, 
> which I think is better.  In fact, I can't think of any disadvantages 
> to doing it this way - can you?
>
> I don't have a patch to submit for this; instead I've created a patch 
> as part of an AUR submission that produces a new version of pacman 
> with the behavior I have described.  I think this is an easier way for 
> users to test this feature.  But I would be more than happy to submit 
> a patch to this list directly for this issue if it is the preferred 
> means of getting this change into the official pacman sources.  A 
> question I have though, is exactly what should be patched?  The NEWS 
> file, configure.ac, etc?  Or just the code?  I ask because my patch 
> introduces what I believe to be an incompatibility in libalpm with the 
> previous version (because a new "question" has been added, that 
> amounts to an addition to the API), which requires a new libalpm 
> version number, so there are lots of files in the pacman source that 
> need to be patched to effect this.  Should I submit a patch to this 
> list that takes pacman-3.2.1 to pacman-3.3.0 (with libalpm 4.0)?  Or 
> should I just submit the actual header and source file changes (plus 
> man page and other documentation changes, I am sure) in a patch to 
> this list?
>
> Any questions, concerns, comments, or other feedback about this patch 
> would be most appreciated.
>
> The bug in question is at:   http://bugs.archlinux.org/task/9395
> The AUR package that I have created is at:   
> http://aur.archlinux.org/packages.php?ID=11182
>
> Thanks, and best wishes,
> Bryan

As I expected, that package has already been deleted from the AUR.  
Submit your patch for the code here.  You don't need to update the NEWS, 
configure.ac files etc because the will be done by us at release time.

Allan





From bji-keyword-pacman.3644cb at www.ischo.com  Wed Dec 31 00:10:57 2008
From: bji-keyword-pacman.3644cb at www.ischo.com (Bryan Ischo)
Date: Wed, 31 Dec 2008 18:10:57 +1300
Subject: [pacman-dev] PATCH: Fix bug 9395 (allow sync to remove unresolvable
 packages if user wishes)
Message-ID: <495AFEE1.7000106@www.ischo.com>

Hi, attached is a patch to fix bug 9395, against the current git tree 
for pacman, as described in an earlier email to this list.  You may find 
a description of the changes I made in the Arch bug database for bug 
9395.  Please let me know if there are any questions.

Thank you, and best wishes,
Bryan

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: pacman-fixbug9395.patch
URL: <http://archlinux.org/pipermail/pacman-dev/attachments/20081231/6ae52696/attachment-0001.diff>

From shiningxc at gmail.com  Wed Dec 31 04:24:48 2008
From: shiningxc at gmail.com (Xavier)
Date: Wed, 31 Dec 2008 10:24:48 +0100
Subject: [pacman-dev] PATCH: Fix bug 9395 (allow sync to remove
	unresolvable packages if user wishes)
In-Reply-To: <495AFEE1.7000106@www.ischo.com>
References: <495AFEE1.7000106@www.ischo.com>
Message-ID: <91752840812310124i5c4a32f9l3576c7002905f5f3@mail.gmail.com>

On Wed, Dec 31, 2008 at 6:10 AM, Bryan Ischo
<bji-keyword-pacman.3644cb at www.ischo.com> wrote:
> Hi, attached is a patch to fix bug 9395, against the current git tree for
> pacman, as described in an earlier email to this list.  You may find a
> description of the changes I made in the Arch bug database for bug 9395.
>  Please let me know if there are any questions.
>
> Thank you, and best wishes,
> Bryan
>
>
> diff -rupN -x '*\.git*' pacman/doc/pacman.8.txt
> pacman-fixbug9395/doc/pacman.8.txt
> --- pacman/doc/pacman.8.txt     2008-12-31 16:45:31.000000000 +1300
> +++ pacman-fixbug9395/doc/pacman.8.txt  2008-12-31 17:52:33.000000000 +1300
> @@ -152,6 +152,11 @@ Options
>        If an install scriptlet exists, do not execute it. Do not use this
>        unless you know what you are doing.
>
> +*\--noignoreprompt*::
> +       Do not prompt the user to ask for confirmation that packages listed
> in
> +       IgnorePkg and IgnoreGroup really should be ignored; instead, assume
> +       that they should be and silently ignore them.
> +
>


Maybe we could simply get totally rid of this prompt, and avoid this
option altogether.
I think you already suggested that somewhere, and it might be fine. We
are usually against interaction.


>  Query Options[[QO]]
>  -------------------
> diff -rupN -x '*\.git*' pacman/lib/libalpm/alpm.h
> pacman-fixbug9395/lib/libalpm/alpm.h
> --- pacman/lib/libalpm/alpm.h   2008-12-31 16:45:31.000000000 +1300
> +++ pacman-fixbug9395/lib/libalpm/alpm.h        2008-12-31
> 17:59:38.000000000 +1300
> @@ -377,7 +377,7 @@ typedef enum _pmtransconv_t {
>        PM_TRANS_CONV_CONFLICT_PKG = 0x04,
>        PM_TRANS_CONV_CORRUPTED_PKG = 0x08,
>        PM_TRANS_CONV_LOCAL_NEWER = 0x10,
> -       /* 0x20 flag can go here */
> +    PM_TRANS_CONV_REMOVE_PKGS = 0x20,
>        PM_TRANS_CONV_REMOVE_HOLDPKG = 0x40
>  } pmtransconv_t;
>
> diff -rupN -x '*\.git*' pacman/lib/libalpm/deps.c
> pacman-fixbug9395/lib/libalpm/deps.c
> --- pacman/lib/libalpm/deps.c   2008-12-31 16:45:31.000000000 +1300
> +++ pacman-fixbug9395/lib/libalpm/deps.c        2008-12-31
> 17:52:41.000000000 +1300
> @@ -546,17 +546,92 @@ pmpkg_t *_alpm_resolvedep(pmdepend_t *de
>        return(NULL);
>  }
>
> +typedef struct __pkginfo_t
> +{
> +       /* The package for which this info is being kept */
> +       pmpkg_t *pkg;
> +       /* 0 if this package has been determined to be unresolvable, meaning
> that
> +          it has dependencies that cannot be resolved, nonzero otherwise */
> +       int unresolvable;
> +       /* 0 if this package was not pulled, nonzero if this package was
> pulled */
> +       int pulled;
> +       /* Packages that are immediately dependent on this package. */
> +       alpm_list_t *dependents;
> +} pkginfo_t;
> +
> +


I see that we need to keep tracks of all the parent dependencies, so
that when we want to remove a package from the targets, we remove all
the parent dependencies as well. We might want to re-use the existing
pmgraph_t structure we have for that, instead of adding a new one.

As for the rest, we could adopt a more general approach.
We add a SkipUnresolvable option in pacman.conf. If this option is
enabled, we simply skip all unresolvable packages from the target
list, no matter if it was caused by an ignored package or not.


> +static pkginfo_t *_alpm_findinfo(alpm_list_t *list, pmpkg_t *pkg)
> +{
> +       alpm_list_t *i;
> +
> +       for (i = list; i; i = i->next) {
> +               pkginfo_t *info = (pkginfo_t *) i->data;
> +               if (info->pkg == pkg) {
> +                       return info;
> +               }
> +       }
> +
> +       return NULL;
> +}
> +
> +
<snip>
> diff -rupN -x '*\.git*' pacman/lib/libalpm/deps.h
> pacman-fixbug9395/lib/libalpm/deps.h
> --- pacman/lib/libalpm/deps.h   2008-12-31 16:45:31.000000000 +1300
> +++ pacman-fixbug9395/lib/libalpm/deps.h        2008-12-31
> 17:52:41.000000000 +1300
> @@ -48,8 +48,8 @@ void _alpm_depmiss_free(pmdepmissing_t *
>  alpm_list_t *_alpm_sortbydeps(alpm_list_t *targets, int reverse);
>  void _alpm_recursedeps(pmdb_t *db, alpm_list_t *targs, int
> include_explicit);
>  pmpkg_t *_alpm_resolvedep(pmdepend_t *dep, alpm_list_t *dbs, alpm_list_t
> *excluding, pmpkg_t *tpkg);
> -int _alpm_resolvedeps(pmdb_t *local, alpm_list_t *dbs_sync, alpm_list_t
> *list,
> -                     alpm_list_t *remove, alpm_list_t **data);
> +int _alpm_resolvedeps(pmdb_t *local, alpm_list_t *dbs_sync, alpm_list_t
> **list,
> +                      alpm_list_t **pulled, alpm_list_t *remove,
> alpm_list_t **data);


Looks like these changes are indeed needed, to be able to remove
packages from "list".
Maybe for consistency we could do the same change for the "remove"
list, even if it is not needed.
Well, I am undecided on this one.


>  int _alpm_dep_edge(pmpkg_t *pkg1, pmpkg_t *pkg2);
>  pmdepend_t *_alpm_splitdep(const char *depstring);
>  pmpkg_t *_alpm_find_dep_satisfier(alpm_list_t *pkgs, pmdepend_t *dep);
> diff -rupN -x '*\.git*' pacman/lib/libalpm/sync.c
> pacman-fixbug9395/lib/libalpm/sync.c
> --- pacman/lib/libalpm/sync.c   2008-12-31 16:45:31.000000000 +1300
> +++ pacman-fixbug9395/lib/libalpm/sync.c        2008-12-31
> 18:01:17.000000000 +1300
> @@ -417,9 +417,10 @@ int _alpm_sync_prepare(pmtrans_t *trans,
>        }
>
>        if(!(trans->flags & PM_TRANS_FLAG_NODEPS)) {
> -               /* store a pointer to the last original target so we can
> tell what was
> -                * pulled by resolvedeps */
> -               alpm_list_t *pulled = alpm_list_last(list);
> +               /* resolvedeps returns a pointer to the first element of the
> +                * list which is a pulled element, and all elements after
> that
> +                * are pulled as well */
> +               alpm_list_t *pulled;
>                /* Resolve targets dependencies */
>                EVENT(trans, PM_TRANS_EVT_RESOLVEDEPS_START, NULL, NULL);
>                _alpm_log(PM_LOG_DEBUG, "resolving target's dependencies\n");
> @@ -432,13 +433,13 @@ int _alpm_sync_prepare(pmtrans_t *trans,
>                        }
>                }
>
> -               if(_alpm_resolvedeps(db_local, dbs_sync, list, remove, data)
> == -1) {
> +               if(_alpm_resolvedeps(db_local, dbs_sync, &list, &pulled,
> remove, data) == -1) {
>                        /* pm_errno is set by resolvedeps */
>                        ret = -1;
>                        goto cleanup;
>                }
>
> -               for(i = pulled->next; i; i = i->next) {
> +               for(i = pulled; i; i = i->next) {
>                        pmpkg_t *spkg = i->data;
>                        pmsyncpkg_t *sync =
> _alpm_sync_new(PM_PKG_REASON_DEPEND, spkg, NULL);
>                        if(sync == NULL) {
> diff -rupN -x '*\.git*' pacman/src/pacman/callback.c
> pacman-fixbug9395/src/pacman/callback.c
> --- pacman/src/pacman/callback.c        2008-12-31 16:45:31.000000000 +1300
> +++ pacman-fixbug9395/src/pacman/callback.c     2008-12-31
> 18:04:16.000000000 +1300
> @@ -248,7 +248,10 @@ void cb_trans_conv(pmtransconv_t event,
>  {
>        switch(event) {
>                case PM_TRANS_CONV_INSTALL_IGNOREPKG:
> -                       if(data2) {
> +                       if(config->noignoreprompt) {
> +                               *response = 0;
> +                       }
> +                       else if(data2) {
>                                /* TODO we take this route based on data2
> being not null? WTF */
>                                *response = yesno(_(":: %s requires
> installing %s from IgnorePkg/IgnoreGroup. Install anyway?"),
>                                                alpm_pkg_get_name(data2),
> @@ -274,6 +277,31 @@ void cb_trans_conv(pmtransconv_t event,
>                                        (char *)data2,
>                                        (char *)data2);
>                        break;
> +                case PM_TRANS_CONV_REMOVE_PKGS:
> +                    {
> +                        /* Allocate a buffer big enough to hold all of the
> +                           package names */
> +                        char *packagenames;
> +                        alpm_list_t *unresolved = (alpm_list_t *) data1;
> +                        alpm_list_t *i;
> +                        int len = 1, /* for trailing \0 */ where = 0, count
> = 0;
> +                        for (i = unresolved; i; i = i->next) {
> +                            count += 1;
> +                            len += 3 /* for \t, comma, and \n */ +
> +                                strlen(alpm_pkg_get_name(i->data));
> +                        }
> +                        packagenames = (char *) malloc(len);
> +                        for (i = unresolved; i; i = i->next) {
> +                            where += snprintf(&(packagenames[where]), len -
> where, "\t%s%s\n",
> +                                              alpm_pkg_get_name(i->data),
> (i->next) ? "," : "");
> +                        }
> +                        *response = yesno(_(":: the following package%s
> cannot be upgraded due to unresolvable "
> +
>             "dependencies:\n%s\nDo you want to skip %s package%s for this
> upgrade?"),
> +                                          (count > 1) ? "s" : "",
> packagenames, (count > 1) ? "these" : "this",
> +                                          (count > 1) ? "s" : "");
> +                        free(packagenames);
> +                    }
> +                    break;


We must have already something to display a list of targets, somewhere
in the pacman frontend code. If the existing functions are not
directly usable for this use case, we will probably need to factor out
some code.


> diff -rupN -x '*\.git*' pacman/src/pacman/pacman.c
> pacman-fixbug9395/src/pacman/pacman.c
> --- pacman/src/pacman/pacman.c  2008-12-31 16:45:31.000000000 +1300
> +++ pacman-fixbug9395/src/pacman/pacman.c       2008-12-31
> 17:53:00.000000000 +1300
> @@ -137,15 +137,16 @@ static void usage(int op, const char * c
>                                 "                       ignore a group
> upgrade (can be used more than once)\n"));
>                        printf(_("  -q, --quiet          show less
> information for query and search\n"));
>                }
> -               printf(_("      --config <path>  set an alternate
> configuration file\n"));
> -               printf(_("      --logfile <path> set an alternate log
> file\n"));
> -               printf(_("      --noconfirm      do not ask for any
> confirmation\n"));
> -               printf(_("      --noprogressbar  do not show a progress bar
> when downloading files\n"));
> -               printf(_("      --noscriptlet    do not execute the install
> scriptlet if one exists\n"));
> -               printf(_("  -v, --verbose        be verbose\n"));
> -               printf(_("  -r, --root <path>    set an alternate
> installation root\n"));
> -               printf(_("  -b, --dbpath <path>  set an alternate database
> location\n"));
> -               printf(_("      --cachedir <dir> set an alternate package
> cache location\n"));
> +               printf(_("      --config <path>   set an alternate
> configuration file\n"));
> +               printf(_("      --logfile <path>  set an alternate log
> file\n"));
> +               printf(_("      --noconfirm       do not ask for any
> confirmation\n"));
> +               printf(_("      --noprogressbar   do not show a progress bar
> when downloading files\n"));
> +               printf(_("      --noscriptlet     do not execute the install
> scriptlet if one exists\n"));
> +               printf(_("      --noignoreprompt  don't prompt to confirm
> packages in IngorePkg and IgnoreGroup\n"));
> +               printf(_("  -v, --verbose         be verbose\n"));
> +               printf(_("  -r, --root <path>     set an alternate
> installation root\n"));
> +               printf(_("  -b, --dbpath <path>   set an alternate database
> location\n"));
> +               printf(_("      --cachedir <dir>  set an alternate package
> cache location\n"));
>        }
>  }
>


These changes are strange. Maybe a whitespace / tab issue?

From shiningxc at gmail.com  Wed Dec 31 04:33:26 2008
From: shiningxc at gmail.com (Xavier)
Date: Wed, 31 Dec 2008 10:33:26 +0100
Subject: [pacman-dev] PATCH: Fix bug 9395 (allow sync to remove
	unresolvable packages if user wishes)
In-Reply-To: <495AFEE1.7000106@www.ischo.com>
References: <495AFEE1.7000106@www.ischo.com>
Message-ID: <91752840812310133i6ee6450dq1b49de0c3ea8875b@mail.gmail.com>

On Wed, Dec 31, 2008 at 6:10 AM, Bryan Ischo
<bji-keyword-pacman.3644cb at www.ischo.com> wrote:
> Hi, attached is a patch to fix bug 9395, against the current git tree for
> pacman, as described in an earlier email to this list.  You may find a
> description of the changes I made in the Arch bug database for bug 9395.
>  Please let me know if there are any questions.
>

Another thing, it looks like you are quite able to read and write code :)
So in case you didn't lose your motivation for that, you should really
have a look at these wiki pages :
http://wiki.archlinux.org/index.php/Super_Quick_Git_Guide
http://wiki.archlinux.org/index.php/Pacman_Development

Using git might be painful in the beginning, but after the initial
period, it gets very nice to work with.

From allan at archlinux.org  Wed Dec 31 07:28:08 2008
From: allan at archlinux.org (Allan McRae)
Date: Wed, 31 Dec 2008 22:28:08 +1000
Subject: [pacman-dev] small pacman code clarification
Message-ID: <BAY102-DAV8EC4CD0C58EF933AD998B86E40@phx.gbl>

Hi,

I am fixing FS#12263 (problem with umask and db creation), which only 
requires moving the creation of the db directory for the package before 
the package archive starts getting extracted.

This occurs in the if loop in libalpm/add.c at line 699.  However, I am 
confused about that if loop.  The archive is extraction is in a loop:

    if(!(trans->flags & PM_TRANS_FLAG_DBONLY)) {

Why is there a need for that if?  As far as I can tell, 
PM_TRANS_FLAG_DBONLY only happens with a -Rk operation.  Is this here in 
case we want to add the -k flag to Sync operations?  Or am I missing its 
point completely...

Cheers,
Allan



From dpmcgee at gmail.com  Wed Dec 31 08:33:32 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Wed, 31 Dec 2008 07:33:32 -0600
Subject: [pacman-dev] small pacman code clarification
In-Reply-To: <BAY102-DAV8EC4CD0C58EF933AD998B86E40@phx.gbl>
References: <BAY102-DAV8EC4CD0C58EF933AD998B86E40@phx.gbl>
Message-ID: <449c10960812310533l77cd8ec3h1ea71e5a6720423@mail.gmail.com>

On Wed, Dec 31, 2008 at 6:28 AM, Allan McRae <allan at archlinux.org> wrote:
> Hi,
>
> I am fixing FS#12263 (problem with umask and db creation), which only
> requires moving the creation of the db directory for the package before the
> package archive starts getting extracted.
>
> This occurs in the if loop in libalpm/add.c at line 699.  However, I am
> confused about that if loop.  The archive is extraction is in a loop:
>
>   if(!(trans->flags & PM_TRANS_FLAG_DBONLY)) {
>
> Why is there a need for that if?  As far as I can tell, PM_TRANS_FLAG_DBONLY
> only happens with a -Rk operation.  Is this here in case we want to add the
> -k flag to Sync operations?  Or am I missing its point completely...

Did we used to have it on -A/-U and it got silently lost? That would
be my best guess.

I've never really been sure that this option is needed anyway, but I
don't want to sidetrack this thread.

-Dan

From ngaba at bibl.u-szeged.hu  Wed Dec 31 08:43:12 2008
From: ngaba at bibl.u-szeged.hu (Nagy Gabor)
Date: Wed, 31 Dec 2008 14:43:12 +0100
Subject: [pacman-dev] PATCH: Fix bug 9395 (allow sync to remove
 unresolvable packages if user wishes)
In-Reply-To: <495AFEE1.7000106@www.ischo.com>
References: <495AFEE1.7000106@www.ischo.com>
Message-ID: <20081231144312.6fd26a94@athlon.localdomain>

> Hi, attached is a patch to fix bug 9395, against the current git tree 
> for pacman, as described in an earlier email to this list.  You may
> find a description of the changes I made in the Arch bug database for
> bug 9395.  Please let me know if there are any questions.
> 
> Thank you, and best wishes,
> Bryan
> 

Basically this is a neat (and readable) job imho.

Some comments:
1. There is a typo in the description of unresolvable field.
2. A new list for pkginfos looks ugly imho.
3. I like the concept of pulled, maybe (pmpkg_t *) pulledby would be
even better, then we could print more informative (error) messages (see
FS#12536 for example). So we again reached to a ~graph structure.
(Hm. Maybe dependents can be also used instead of pulledby...)
4. It is not easy to determine which package is unresolvable. If a
pulled dependency satisfier of foo is unresolvable we may could find an
other (resolvable) satisfier. But this is not handled in the current
code neither, so your _alpm_mark_unresolvable() is OK.
5. I may overlook something, but as I see, the following situation can
happen:
# pacman -S foo bar, where foo depends on bar.
During resolvedeps foo may become unresolvable, but bar not. In
this case pacman asks the user whether he wants to remove foo from
target list. User answers yes. Since your _alpm_is_needed code (btw,
_alpm prefix is not needed here) doesn't check pulled flag, it will
determine that bar is not needed (which is not true, bar was an explicit
package), and pacman will also remove bar as well, and the user wasn't
informed about this.
6. I think a dependency cycle can lead to an infinite loop due to
_alpm_is_needed.

Bye


From shiningxc at gmail.com  Wed Dec 31 10:50:36 2008
From: shiningxc at gmail.com (Xavier)
Date: Wed, 31 Dec 2008 16:50:36 +0100
Subject: [pacman-dev] PATCH: Fix bug 9395 (allow sync to remove
	unresolvable packages if user wishes)
In-Reply-To: <20081231144312.6fd26a94@athlon.localdomain>
References: <495AFEE1.7000106@www.ischo.com>
	<20081231144312.6fd26a94@athlon.localdomain>
Message-ID: <91752840812310750u4eab669bn577a01f234363459@mail.gmail.com>

On Wed, Dec 31, 2008 at 2:43 PM, Nagy Gabor <ngaba at bibl.u-szeged.hu> wrote:
> Since your _alpm_is_needed code (btw,
> _alpm prefix is not needed here)

Oh indeed, all the static functions should not have any prefix.
Maybe the README file is not perfectly clear about this, but it can be
understood implicitly.

  26 In a general manner, public library functions are named
"alpm_<type>_<action>"
  27 (examples: alpm_trans_commit(), alpm_release(), alpm_pkg_get_name(), ...).
  28 Internal (and thus private) functions should be named "_alpm_XXX"
for instance
  29 (examples: _alpm_needbackup(), _alpm_runscriplet(), ...).
Functions defined and
  30 used inside a single file should be defined as "static".

From bji-keyword-pacman.3644cb at www.ischo.com  Wed Dec 31 14:27:39 2008
From: bji-keyword-pacman.3644cb at www.ischo.com (Bryan Ischo)
Date: Thu, 01 Jan 2009 08:27:39 +1300
Subject: [pacman-dev] PATCH: Fix bug 9395 (allow sync to
 remove	unresolvable packages if user wishes)
In-Reply-To: <91752840812310124i5c4a32f9l3576c7002905f5f3@mail.gmail.com>
References: <495AFEE1.7000106@www.ischo.com>
	<91752840812310124i5c4a32f9l3576c7002905f5f3@mail.gmail.com>
Message-ID: <495BC7AB.7040502@www.ischo.com>

Xavier wrote:
> On Wed, Dec 31, 2008 at 6:10 AM, Bryan Ischo
> <bji-keyword-pacman.3644cb at www.ischo.com> wrote:
>   

Thank you very much for your feedback, Xavier!  Comments inline below ...

> Maybe we could simply get totally rid of this prompt, and avoid this
> option altogether.
> I think you already suggested that somewhere, and it might be fine. We
> are usually against interaction.
>   
(Xavier is referring here to the prompt where libalpm issues a query 
(via a callback) about whether or not the user wants to upgrade a 
package which was in IgnorePkg/IgnoreGroup should that package be needed 
to upgrade another package in the transaction.  My changes add a config 
flag allowing this prompt to be skipped.

I personally would like to see that prompt removed entirely, as you have 
suggested.  My reasoning is that if the user has put a package in 
IgnorePkg/IgnoreGroup, it's because they really want it to be ignored, 
so asking them if they are sure is unnecessary.  However, I don't feel 
that I can make a call like this - I think it's up to the main pacman 
devs, so I'd like to hear others' opinions on this.  I can certainly 
remove that prompt entirely in a new patch, and would in fact like to do 
that, but I won't unless instructed to by the pacman dev team.

One thing that I think could help make that question even more redundant 
is if the information given to the user in the new prompt I have created 
was more thorough.  Right now it just tells the user what packages need 
to be removed from the transaction, but it doesn't say what packages 
were unresolvable or why.  If the prompt looked something more like this:

:: the following packages cannot be upgraded due to unresolvable 
dependencies:
          firefox 3.0.5, requires xulrunner 1.0.9
          xulrunner 1.0.9, requires pango-1.6
          pango-1.6, ignored by user configuration

Then at least the user would always be aware of what effect their 
IgnorePkg/IgnoreGroup settings is having on the sync operation.  I 
wanted to do something like this, but was not ambitious enough, as it 
would require quite a more sophisticated data structure being passed to 
the query callback, and for the query callback to be significantly more 
complicated.

> I see that we need to keep tracks of all the parent dependencies, so
> that when we want to remove a package from the targets, we remove all
> the parent dependencies as well. We might want to re-use the existing
> pmgraph_t structure we have for that, instead of adding a new one.
>   
Thanks for the suggestion, I will look into that.  I was unaware of the 
existence of this structure; if it can replace pkginfo_t, I will do so.

> As for the rest, we could adopt a more general approach.
> We add a SkipUnresolvable option in pacman.conf. If this option is
> enabled, we simply skip all unresolvable packages from the target
> list, no matter if it was caused by an ignored package or not.
>   
I am not sure I understand what you are saying here.  Packages are 
already marked unresolvable whether it was due to being in 
IgnorePkg/IgnoreGroup or for some other reason, the reason is not kept 
track of or used in the resolution process.  I think that what you're 
saying is that the skipping should be made automatic instead of 
prompting the user whether or not to do it.  Actually I would agree with 
that also; perhaps this information can simply be provided in the final 
"Proceed with installation?" prompt.  Something like this:

Targets (29): openssl-0.9.8i-3 ca-certificates-20080809-5 libpng-1.2.34-1
xcb-util-0.3.2-1 cairo-1.8.6-1 dbus-glib-0.78-1
diffutils-2.8.1-6 libtasn1-1.7-1 gnutls-2.6.3-1
hal-info-0.20081219-1 hdparm-9.3-1 hwdetect-2008.12-3
intel-dri-7.2-2 klibc-udev-135-1 libdmx-1.0.2-2
libxml2-2.7.2-1 libgsf-1.14.10-1 libxfont-1.3.4-1
nss-3.12.2-1 pacman-3.2.1-2 pycairo-1.8.0-1
subversion-1.5.5-1 sudo-1.7.0-1 tar-1.20-3 ttf-dejavu-2.28-1
udev-135-1 xextproto-7.0.4-1 xkeyboard-config-1.4-2
xorg-font-utils-7.4-1

Skipped Targets (4): firefox-3.0.5, xulrunner-1.0.9, pango-1.6

Total Download Size: 23.64 MB
Total Installed Size: 80.21 MB

Proceed with installation? [Y/n] n

This would be nice because it would avoid the need for another 
confirmation question (since the "Proceed with installation" serves the 
same purpose).  However, adding the Skipped Targets to this prompt might 
a) confuse the user, and b) result in too much verbage (could be a very 
large list if IgnorePkg/IgnoreGroup has packages "low down" in the 
dependency tree).

Also, it would require much bigger changes to the code.  Also, the 
changes currently are nicely self-contained because they simply change 
the behavior of _alpm_resolvedeps, they don't reach out into other 
functions, forcing them to keep track of this information so that they 
can ultimately produce the new Proceed prompt.  And finally, I haven't 
looked to see all of the places that _alpm_resolvedeps is called, but 
it's possible that there are other situations that it's called in that 
really should issue that "skip these packages" prompt immediately rather 
than deferring to some later prompt, which might not even happen for 
that particular code path.

All in all, I like the idea, but I think it might be better left to a 
v2.0 version of this feature.

>> +static pkginfo_t *_alpm_findinfo(alpm_list_t *list, pmpkg_t *pkg)
>> +{
>> +       alpm_list_t *i;
>> +
>> +       for (i = list; i; i = i->next) {
>> +               pkginfo_t *info = (pkginfo_t *) i->data;
>> +               if (info->pkg == pkg) {
>> +                       return info;
>> +               }
>> +       }
>> +
>> +       return NULL;
>> +}
>> +
>> +
>>     
> <snip>
>   
>> diff -rupN -x '*\.git*' pacman/lib/libalpm/deps.h
>> pacman-fixbug9395/lib/libalpm/deps.h
>> --- pacman/lib/libalpm/deps.h   2008-12-31 16:45:31.000000000 +1300
>> +++ pacman-fixbug9395/lib/libalpm/deps.h        2008-12-31
>> 17:52:41.000000000 +1300
>> @@ -48,8 +48,8 @@ void _alpm_depmiss_free(pmdepmissing_t *
>>  alpm_list_t *_alpm_sortbydeps(alpm_list_t *targets, int reverse);
>>  void _alpm_recursedeps(pmdb_t *db, alpm_list_t *targs, int
>> include_explicit);
>>  pmpkg_t *_alpm_resolvedep(pmdepend_t *dep, alpm_list_t *dbs, alpm_list_t
>> *excluding, pmpkg_t *tpkg);
>> -int _alpm_resolvedeps(pmdb_t *local, alpm_list_t *dbs_sync, alpm_list_t
>> *list,
>> -                     alpm_list_t *remove, alpm_list_t **data);
>> +int _alpm_resolvedeps(pmdb_t *local, alpm_list_t *dbs_sync, alpm_list_t
>> **list,
>> +                      alpm_list_t **pulled, alpm_list_t *remove,
>> alpm_list_t **data);
>>     
>
>
> Looks like these changes are indeed needed, to be able to remove
> packages from "list".
> Maybe for consistency we could do the same change for the "remove"
> list, even if it is not needed.
> Well, I am undecided on this one.
>   

I'm afraid I don't quite understand this comment.  Can you please clarify?

> We must have already something to display a list of targets, somewhere
> in the pacman frontend code. If the existing functions are not
> directly usable for this use case, we will probably need to factor out
> some code.
>   
I will take a look; if there is code that I can easily re-use, I will do 
so.  However, one of my goals with this patch was to keep it very 
self-contained (it just modifies the behavior of _alpm_resolvedeps and 
makes a minor change to its signature, also it adds a callback handler 
to pacman for the new question callback, otherwise it doesn't change 
anything), because I don't know the pacman code base well enough to feel 
confident changing it around too much.

>> diff -rupN -x '*\.git*' pacman/src/pacman/pacman.c
>> pacman-fixbug9395/src/pacman/pacman.c
>> --- pacman/src/pacman/pacman.c  2008-12-31 16:45:31.000000000 +1300
>> +++ pacman-fixbug9395/src/pacman/pacman.c       2008-12-31
>> 17:53:00.000000000 +1300
>> @@ -137,15 +137,16 @@ static void usage(int op, const char * c
>>                                 "                       ignore a group
>> upgrade (can be used more than once)\n"));
>>                        printf(_("  -q, --quiet          show less
>> information for query and search\n"));
>>                }
>> -               printf(_("      --config <path>  set an alternate
>> configuration file\n"));
>> -               printf(_("      --logfile <path> set an alternate log
>> file\n"));
>> -               printf(_("      --noconfirm      do not ask for any
>> confirmation\n"));
>> -               printf(_("      --noprogressbar  do not show a progress bar
>> when downloading files\n"));
>> -               printf(_("      --noscriptlet    do not execute the install
>> scriptlet if one exists\n"));
>> -               printf(_("  -v, --verbose        be verbose\n"));
>> -               printf(_("  -r, --root <path>    set an alternate
>> installation root\n"));
>> -               printf(_("  -b, --dbpath <path>  set an alternate database
>> location\n"));
>> -               printf(_("      --cachedir <dir> set an alternate package
>> cache location\n"));
>> +               printf(_("      --config <path>   set an alternate
>> configuration file\n"));
>> +               printf(_("      --logfile <path>  set an alternate log
>> file\n"));
>> +               printf(_("      --noconfirm       do not ask for any
>> confirmation\n"));
>> +               printf(_("      --noprogressbar   do not show a progress bar
>> when downloading files\n"));
>> +               printf(_("      --noscriptlet     do not execute the install
>> scriptlet if one exists\n"));
>> +               printf(_("      --noignoreprompt  don't prompt to confirm
>> packages in IngorePkg and IgnoreGroup\n"));
>> +               printf(_("  -v, --verbose         be verbose\n"));
>> +               printf(_("  -r, --root <path>     set an alternate
>> installation root\n"));
>> +               printf(_("  -b, --dbpath <path>   set an alternate database
>> location\n"));
>> +               printf(_("      --cachedir <dir>  set an alternate package
>> cache location\n"));
>>        }
>>  }
>>
>>     
>
>
> These changes are strange. Maybe a whitespace / tab issue?
>
>   

Almost certainly.  My xxdiff showed only the last bits, surrounding the 
--noignoreprompt option that I had added, as changing, so I'm not sure 
why diff pulled all the rest in.  I will investigate.

Thank you again for your comments.  I will be issuing an updated patch 
shortly which incorporates your suggetsions as well as those of later 
posters (who I will respond to in turn soon).

Best wishes,
Bryan



From bji-keyword-pacman.3644cb at www.ischo.com  Wed Dec 31 15:55:24 2008
From: bji-keyword-pacman.3644cb at www.ischo.com (Bryan Ischo)
Date: Thu, 01 Jan 2009 09:55:24 +1300
Subject: [pacman-dev] PATCH: Fix bug 9395 (allow sync to remove
 unresolvable packages if user wishes)
In-Reply-To: <20081231144312.6fd26a94@athlon.localdomain>
References: <495AFEE1.7000106@www.ischo.com>
	<20081231144312.6fd26a94@athlon.localdomain>
Message-ID: <495BDC3C.9050904@www.ischo.com>

Nagy Gabor wrote:
>
> Basically this is a neat (and readable) job imho.
>   
Thanks for your kind words and your input.  Comments inline below.
> Some comments:
> 1. There is a typo in the description of unresolvable field.
>   
Check, will fix.

> 2. A new list for pkginfos looks ugly imho.
>   

What do you mean by 'ugly'?  Is it redundant given other data structures 
in libalpm?  If so, I will be happy to switch to another data 
structure.  Xavier already pointed out that pmgraph_t may fit the bill.  
I'll definitely look into it.  Or maybe you meant something else by 
'ugly'?  I'm not sure ...

> 3. I like the concept of pulled, maybe (pmpkg_t *) pulledby would be
> even better, then we could print more informative (error) messages (see
> FS#12536 for example). So we again reached to a ~graph structure.
> (Hm. Maybe dependents can be also used instead of pulledby...)
>   
This sounds like a good idea.  But it would involve greater changes to 
libalpm then I felt comfortable doing.  Unfortunately, when one is just 
a patch contributor rather than a real developer, one feels the need to 
keep all changes as minimal as possible, instead of reaching into lots 
of parts of the code and doing cleanups and refactoring and such.  But I 
would be happy to do some of that, once I feel more comfortable with the 
code base.

> 4. It is not easy to determine which package is unresolvable. If a
> pulled dependency satisfier of foo is unresolvable we may could find an
> other (resolvable) satisfier. But this is not handled in the current
> code neither, so your _alpm_mark_unresolvable() is OK.
>   
I see what you mean.  Is this something that should/could be fixed in 
_alpm_resolvedeps?  If so, perhaps it can be deferred to a separate bug 
incident and fixed separately from my change?

> 5. I may overlook something, but as I see, the following situation can
> happen:
> # pacman -S foo bar, where foo depends on bar.
> During resolvedeps foo may become unresolvable, but bar not. In
> this case pacman asks the user whether he wants to remove foo from
> target list. User answers yes. Since your _alpm_is_needed code (btw,
> _alpm prefix is not needed here) doesn't check pulled flag, it will
> determine that bar is not needed (which is not true, bar was an explicit
> package), and pacman will also remove bar as well, and the user wasn't
> informed about this.
>   
I will remove _alpm prefix from my static functions, I hadn't read the 
code guidelines carefully enough to realize the convention, mea culpa.

As to your problem situation: good catch, I missed that case in my 
thought process and in testing.  Thanks for pointing it out, I will fix 
it (should be an easy fix in _alpm_is_needed).

> 6. I think a dependency cycle can lead to an infinite loop due to
> _alpm_is_needed.
>   
I didn't realize that dependency cycles were possible.  But you are 
right, they would lead to an infinite loop.  I'll have to figure out 
some way to detect and break such cycles.  Thanks for pointing that out 
to me.

Best wishes,
Bryan


From aaronmgriffin at gmail.com  Wed Dec 31 16:03:09 2008
From: aaronmgriffin at gmail.com (Aaron Griffin)
Date: Wed, 31 Dec 2008 15:03:09 -0600
Subject: [pacman-dev] PATCH: Fix bug 9395 (allow sync to remove
	unresolvable packages if user wishes)
In-Reply-To: <495BDC3C.9050904@www.ischo.com>
References: <495AFEE1.7000106@www.ischo.com>
	<20081231144312.6fd26a94@athlon.localdomain>
	<495BDC3C.9050904@www.ischo.com>
Message-ID: <d64a48980812311303t2beb991bjb21def25270d247c@mail.gmail.com>

On Wed, Dec 31, 2008 at 2:55 PM, Bryan Ischo
<bji-keyword-pacman.3644cb at www.ischo.com> wrote:
> I didn't realize that dependency cycles were possible.  But you are right,
> they would lead to an infinite loop.  I'll have to figure out some way to
> detect and break such cycles.  Thanks for pointing that out to me.

We have a few cycles in core alone. Pacman deals with it elegantly
enough, in that it just kinda decides on an order and warns us "A is
being installed before its dependency B". If you want to see this in
action, try:

$ pacman -r some-dir/ -Sy base

You should get 2 or 3, mainly relating to things like glibc and bash

From dpmcgee at gmail.com  Wed Dec 31 16:19:28 2008
From: dpmcgee at gmail.com (Dan McGee)
Date: Wed, 31 Dec 2008 15:19:28 -0600
Subject: [pacman-dev] Pacman contributors/developers (was: Fix bug 9395)
Message-ID: <449c10960812311319r2d750f60vddde441047d06366@mail.gmail.com>

On Wed, Dec 31, 2008 at 2:55 PM, Bryan Ischo
<bji-keyword-pacman.3644cb at www.ischo.com> wrote:
> This sounds like a good idea.  But it would involve greater changes to
> libalpm then I felt comfortable doing.  Unfortunately, when one is just a
> patch contributor rather than a real developer, one feels the need to keep
> all changes as minimal as possible, instead of reaching into lots of parts
> of the code and doing cleanups and refactoring and such.  But I would be
> happy to do some of that, once I feel more comfortable with the code base.

Just wanted to set the record straight here- if you are a patch
contributor, you are a developer in my mind. I don't think the old
timers or new timers are perfectly happy with the shape libalpm is in,
so feel free to hack things up if it results in long term improvement.

We definitely have done a lot of refactoring and cleanup to the code,
but there is no shortage of places that could still use a good rework
or rewrite.

Obviously it is best if you can separate your work into patch chunks
that are easier to digest, but sometimes that just isn't possible. GIT
makes it real easy to go back and refactor your refactorings as you
can easily mix and match your patches.

-Dan