Merge lp:~cjwatson/britney/fix-sru-branching into lp:britney

Proposed by Colin Watson
Status: Superseded
Proposed branch: lp:~cjwatson/britney/fix-sru-branching
Merge into: lp:britney
Diff against target: 7295 lines (+7146/-0) (has conflicts)
28 files modified
README (+107/-0)
britney (+516/-0)
fauxpkg/FauxPackages (+59/-0)
fauxpkg/README (+50/-0)
fauxpkg/fauxpkg.py (+181/-0)
fauxpkg/noremove.d/README (+3/-0)
scripts/backlog-report (+86/-0)
scripts/ood_report.py (+101/-0)
scripts/udeb-report (+49/-0)
update_out/.gitignore (+6/-0)
update_out/Makefile (+33/-0)
update_out/README (+32/-0)
update_out/assert.c (+11/-0)
update_out/britney-py.c (+871/-0)
update_out/checklib.c (+185/-0)
update_out/dpkg-lib.cpp (+34/-0)
update_out/dpkg.c (+2013/-0)
update_out/dpkg.h (+207/-0)
update_out/freelist.c (+188/-0)
update_out/freelist.h (+14/-0)
update_out/index.html (+18/-0)
update_out/memory.c (+389/-0)
update_out/memory.h (+47/-0)
update_out/memory2.c (+20/-0)
update_out/memory3.c (+209/-0)
update_out/templates.h (+277/-0)
update_out/thoughts (+13/-0)
update_out/update_out.py (+1427/-0)
Text conflict in README
To merge this branch: bzr merge lp:~cjwatson/britney/fix-sru-branching
Reviewer Review Type Date Requested Status
Ubuntu Package Archive Administrators Pending
Review via email: mp+261537@code.launchpad.net

This proposal has been superseded by a proposal from 2015-06-09.

Commit message

Make sure the hints branch always points to the right place, even if it's changed due to e.g. a series being released.

Description of the change

Make sure the hints branch always points to the right place, even if it's changed due to e.g. a series being released.

To post a comment you must log in.

Unmerged revisions

281. By Colin Watson

Make sure the hints branch always points to the right place, even if it's changed due to e.g. a series being released.

280. By Adam Conrad

vivid -> wily

279. By Iain Lane

merge lp:~cjwatson/britney/dynamic-arches

278. By Steve Langasek

Merge lp:~cjwatson/britney/sru-lock

277. By Steve Langasek

Merge lp:~cjwatson/britney/honour-updates

276. By Steve Langasek

Merge lp:~cjwatson/britney/fix-blocks-function

275. By Steve Langasek

Merge lp:~cjwatson/britney/more-output-prefix

274. By Steve Langasek

Merge lp:~cjwatson/britney/output-prefix

273. By Steve Langasek

Merge lp:~cjwatson/britney/sru-config

272. By Colin Watson

Publish update_excuses.yaml and archive copies of it.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'README'
--- README 2006-06-17 13:45:56 +0000
+++ README 2015-06-09 16:34:27 +0000
@@ -1,5 +1,112 @@
1<<<<<<< TREE
1README for britney v2.02README for britney v2.0
2=======================3=======================
34
4Please write here some useful stuff.5Please write here some useful stuff.
56
7=======
8Britney is run from /org/release.debian.org/britney. Some notes about
9how it works, what to do and what not:
10
11 * the code is in code/b1, which is a git repo. Changes must be
12 approved by Release Managers first, and all changes must be commited
13 (i.e. nothing uncommited is allowed there).
14
15 Commit emails will be send to britney@release.debian.org, you can
16 subscribe to that if you want them (along with cron mail from
17 britney).
18
19 * the main script is code/b1/britney. It accepts a list of "actions"
20 to perform. See ~release/etc/crontab.release for the default list,
21 and see the "Actions" and "Re-running" sections below.
22
23 * input/ contains FauxPackages, edit if if you need, and an urgencies/
24 subdirectory where dinstall places a summary of the uploads of that
25 day.
26
27 * Heidi/, ssh/: see the "save" action below.
28
29 * d-i/: udeb handling is done by the release team too. See d-i/README
30 for details (when Dato writes it; Dato takes care of udebs until then).
31
32
33Actions
34=======
35
36The main actions (read, the ones that are invoked from cron) are:
37
38 * urgencies: updates var/data/testing/Urgency with the data provided
39 by dinstall.
40
41 * bugs: fetches bug counts from bugs.debian.org.
42
43 * pkglists: generates all needed Packages files according to the
44 latest dinstall.
45
46 * run: the main action, running the update_out.py script. Takes
47 hints, packages files, and tries to migrate stuff. When done, it
48 writes a HeidiResult file used by "save" below *and rewrites the
49 var/data/testing/Packags_* files*. That is important.
50
51 * save: commits the HeidiResult provided by update_out above. This is
52 done by:
53
54 - taking HeidiResult, stripping unwanted stuff (faux packages),
55 and put the result into a file under Heidi/set, named after the
56 current timestamp.
57
58 - changing the "current" symlink under Heidi/set to point to this
59 new file, gzipping the previous current.
60
61 - signaling ftpmaster scripts to import the new file. This is done
62 with a ssh trigger, whose key is in ssh/.
63
64 IF YOU NEED TO UNDO ONE IMPORT: gunzip the previous file, point
65 the "current" symlink to it, and:
66
67 % ssh -2 -i % ~release/britney/ssh/ftp-master_trigger_key dak@ries.debian.org
68
69 * stats: copies the resulting update.OUTPUT and update.EXCUSES files
70 to www/britney, and checks for uninstallabilities.
71
72
73Re-running
74==========
75
76Re-runs can be incremental or not:
77
78 * incremental means that you run britney against the results of the
79 previous run. This is done by just issuing `./britney run`.
80
81 WARNING: even if you don't save the results of an incremental
82 re-run, following re-runs will start with the state the previous one
83 left. This means it's a very very bad idea to use incremental
84 re-runs to test force-hint hints; please always use a private
85 britney instead.
86
87 * non-incremental means that you start from scratch against the state
88 of the previous install. To do this, you need to do "pkglists" prior
89 to "run".
90
91 IF THERE WERE ANY SUCCESSFUL HINTS IN THE PREVIOUS RUN, YOU MUST
92 CHECK WITH THEIR OWNERS WHETHER IT'S OK TO RUN FROM SCRATCH. Or, in
93 the worst case, you should check whether they have cleaned up their
94 hints already, since in that case their work will be lost.
95
96 (For this reason it's not very wise to clean your hints before your
97 stuff is dinstalled.)
98
99If you're satisfied with the result of your re-run, run `./britney save`.
100
101
102Private britneys
103================
104
105It is ok to run private britneys if you nice them, though Ganneff
106prefers if you don't do it while dinstall is running.
107
108Do NOT run your britney against britney/var/data directly, use a
109copy of your own.
110
111TODO(dato): Some HOWTO here.
112>>>>>>> MERGE-SOURCE
6113
=== added file 'britney'
--- britney 1970-01-01 00:00:00 +0000
+++ britney 2015-06-09 16:34:27 +0000
@@ -0,0 +1,516 @@
1#!/bin/bash
2
3set -e
4set -u
5
6ulimit -d 8000000 -m 8000000 -v 8000000
7umask 002
8
9
10OPTIONS="$@"
11qoption () {
12 for a in $OPTIONS; do if [ "$a" = "$1" ]; then return 0; fi; done
13 return 1
14}
15
16option () {
17 for a in $OPTIONS; do if [ "$a" = "$1" ]; then date -uR; return 0; fi; done
18 return 1
19}
20
21LP_SERVICE="${LP_SERVICE:-production}"
22DISTRIBUTION="${DISTRIBUTION:-ubuntu}"
23DEFAULT_SERIES=wily
24SERIES="${SERIES:-$DEFAULT_SERIES}"
25
26# DATE
27NOW=`date +"%F/%T" -u`
28
29# Dirs:
30BASE=/home/ubuntu-archive/proposed-migration
31DAK_LOCKDIR=/srv/ftp-master.debian.org/lock
32FTP_MIRROR=/home/ubuntu-archive/mirror/$DISTRIBUTION
33
34D_I=$BASE/d-i
35VAR=$BASE/var
36SSH=$BASE/ssh
37HEIDI=$BASE/Heidi
38INPUT=$BASE/input
39CODE=$BASE/code/b1
40
41DATA=$VAR/data
42LOCKDIR=$VAR/lock
43HEIDI_SET=$HEIDI/set
44SCRIPTS=$CODE/scripts
45FAUXDIR=$CODE/fauxpkg
46UPDATE_OUT=$CODE/update_out
47URGENCIES=$INPUT/urgencies
48
49LOCK=$LOCKDIR/britney-$DISTRIBUTION-$SERIES.lock
50DAK_LOCK=$DAK_LOCKDIR/daily.lock
51DAK_STOP=$DAK_LOCKDIR/archive.stop
52
53TESTING=$DATA/$SERIES
54UNSTABLE=$DATA/$SERIES-proposed
55TPU=$DATA/testing-proposed-updates
56
57FAUXPKG_SCRIPT=$FAUXDIR/fauxpkg.py
58SSH_KEY=$SSH/ftp-master_trigger_key
59
60HTML=/home/ubuntu-archive/public_html/proposed-migration
61if [ "$DISTRIBUTION" != ubuntu ]; then
62 HTML="$HTML/$DISTRIBUTION"
63fi
64BOOTSTRAP_DIR=/home/ubuntu-archive/public_html/bootstrap
65
66# Britney 2
67DATA_B2=$VAR/data-b2
68CODE_B2=$BASE/code/b2
69B2_OUTPUT=$DATA_B2/output
70B2_CONFIG=${B2_CONFIG:-$CODE_B2/britney.conf}
71B2_CONFIG_NOBREAKALL=${B2_CONFIG_NOBREAKALL:-$CODE_B2/britney_nobreakall.conf}
72
73if [ -f $DAK_STOP ]; then
74 echo "$DAK_STOP exists, not running"
75 exit 1
76fi
77
78# 10800 seconds = 3 hours
79# 129600 seconds = 36 hours
80if ! lockfile -r0 -l129600 $LOCK 2>/dev/null >/dev/null; then
81 echo "Could not get britney lockfile!"
82 ls -l "$LOCK"
83 exit 1
84fi
85
86trap exit_function 0
87exit_function () {
88 rm -f $LOCK
89}
90
91#eval $( dak admin config db-shell )
92
93suite_info () {
94 python -c '
95from __future__ import print_function
96
97import os
98import sys
99
100from launchpadlib.launchpad import Launchpad as _Launchpad
101
102# Work around non-multiple-instance-safety of launchpadlib (bug #459418).
103class Launchpad(_Launchpad):
104 @classmethod
105 def _get_paths(cls, service_root, launchpadlib_dir=None):
106 service_root, launchpadlib_dir, cache_path, service_root_dir = (
107 _Launchpad._get_paths(
108 service_root, launchpadlib_dir=launchpadlib_dir))
109 cache_path += "-proposed-migration"
110 if not os.path.exists(cache_path):
111 os.makedirs(cache_path, 0o700)
112 return service_root, launchpadlib_dir, cache_path, service_root_dir
113
114launchpad = Launchpad.login_anonymously("proposed-migration", sys.argv[1])
115distro = launchpad.distributions[sys.argv[2]]
116series_name = sys.argv[3].split("-")[0]
117series = distro.getSeries(name_or_version=series_name)
118'"$1" "$LP_SERVICE" "$DISTRIBUTION" "$2"
119}
120
121suite_arches () {
122 suite_info 'print(" ".join(arch.architecture_tag for arch in series.architectures))' "$1"
123}
124
125suite_archindep () {
126 suite_info 'print(series.nominatedarchindep.architecture_tag)' "$1"
127}
128
129if ! qoption allowdaklock; then
130 while [ -f $DAK_LOCK ]; do
131 echo `date` $DAK_LOCK exists. Sleeping in 10 more minutes.
132 sleep 600
133 done
134fi
135
136urgencies () {
137 echo URGENCIES:
138 cd $BASE
139 for u in $URGENCIES/install-urgencies-*; do
140 [ -e "$u" ] || continue
141 cat "$u" >> $1/$SERIES/Urgency # XXX I'd rather not have this in $DATA --dato
142 rm "$u"
143 done
144}
145
146if option urgencies; then
147 urgencies $DATA_B2
148fi
149
150bugs () {
151 for suite in $SERIES $SERIES-proposed; do
152 x="$1/${suite}/BugsV"
153 wget --quiet -O "${x}.new" http://bugs.debian.org/release-critical/britney/${suite}-nr
154 if [ -s ${x}.new ]; then mv ${x}.new $x; else rm $x.new; exit 1; fi
155 done
156}
157if option bugs; then
158 echo BUGS:
159 bugs $DATA_B2
160fi
161
162if option bugs_b1; then
163 echo BUGS_B1:
164 bugs $DATA
165fi
166
167if option bugs_b2; then
168 echo BUGS_B2:
169 bugs $DATA_B2
170fi
171
172blocks () {
173 local tags
174 [ "$DISTRIBUTION" = ubuntu ] || return 0
175 mkdir -p "$1/$SERIES-proposed"
176 if [ "$SERIES" = "$DEFAULT_SERIES" ]; then
177 tags="block-proposed block-proposed-$SERIES"
178 else
179 tags="block-proposed-$SERIES"
180 fi
181 python -c '
182from __future__ import print_function
183
184import calendar
185import os
186import sys
187try:
188 from urllib.parse import urlsplit
189except ImportError:
190 from urlparse import urlsplit
191
192from launchpadlib.launchpad import Launchpad as _Launchpad
193
194# Work around non-multiple-instance-safety of launchpadlib (bug #459418).
195class Launchpad(_Launchpad):
196 @classmethod
197 def _get_paths(cls, service_root, launchpadlib_dir=None):
198 service_root, launchpadlib_dir, cache_path, service_root_dir = (
199 _Launchpad._get_paths(
200 service_root, launchpadlib_dir=launchpadlib_dir))
201 cache_path += "-proposed-migration"
202 if not os.path.exists(cache_path):
203 os.makedirs(cache_path, 0o700)
204 return service_root, launchpadlib_dir, cache_path, service_root_dir
205
206launchpad = Launchpad.login_with("proposed-migration", sys.argv[1])
207distro = launchpad.distributions[sys.argv[2]]
208tags = sys.argv[3].split()
209for task in distro.searchTasks(omit_targeted=False, tags=tags):
210 target = task.target
211 bug = task.bug
212 if urlsplit(target.resource_type_link).fragment in (
213 "distribution_source_package", "source_package"):
214 for action in reversed(
215 [a for a in bug.activity if a.whatchanged == "tags"]):
216 oldtags = action.oldvalue.split()
217 newtags = action.newvalue.split()
218 gained_block = False
219 for tag in tags:
220 if tag not in oldtags and tag in newtags:
221 gained_block = True
222 break
223 if gained_block:
224 date = action.datechanged
225 break
226 else:
227 date = bug.date_created
228 print("%s %d %d" %
229 (os.path.basename(target.self_link), bug.id,
230 calendar.timegm(date.timetuple())))
231' "$LP_SERVICE" "$DISTRIBUTION" "$tags" >"$1/$SERIES-proposed/Blocks.new"
232 mv "$1/$SERIES-proposed/Blocks.new" "$1/$SERIES-proposed/Blocks"
233}
234if option blocks; then
235 echo BLOCKS:
236 blocks $DATA_B2
237fi
238
239pkg_lists () {
240 for suite in $SERIES $SERIES-proposed; do
241 mkdir -p $1/$suite
242 suite_dirs=$FTP_MIRROR/dists/$suite
243 if [ "$suite" = "$SERIES" ]; then
244 suite_dirs="$suite_dirs $FTP_MIRROR/dists/$SERIES-updates"
245 fi
246 >$1/$suite/Sources
247 for suite_dir in $suite_dirs; do
248 gzip -t $suite_dir/{main,restricted,universe,multiverse}/source/Sources.gz
249 zcat $suite_dir/{main,restricted,universe,multiverse}/source/Sources.gz >>$1/$suite/Sources
250 done
251 for arch in `suite_arches $suite`; do
252 >$1/$suite/Packages_$arch
253 for suite_dir in $suite_dirs; do
254 gzip -t $suite_dir/{main,restricted,universe,multiverse}/binary-$arch/Packages.gz
255 zcat $suite_dir/{main,restricted,universe,multiverse}/binary-$arch/Packages.gz >>$1/$suite/Packages_$arch
256 gzip -t $suite_dir/{main,restricted,universe,multiverse}/debian-installer/binary-$arch/Packages.gz
257 zcat $suite_dir/{main,restricted,universe,multiverse}/debian-installer/binary-$arch/Packages.gz >>$1/$suite/Packages_$arch
258 done
259 done
260 done
261 $FAUXPKG_SCRIPT generate $1/$SERIES $1/$SERIES-proposed
262}
263
264lite_sync () {
265 cat <<EOF | rsync -a --files-from=- $1 $2
266$SERIES/Dates
267$SERIES/BugsV
268$SERIES/Urgency
269$SERIES-proposed/BugsV
270$SERIES-proposed/Hints
271EOF
272
273}
274
275if option pkglists; then
276 echo PKGLISTS:
277 cd $DATA_B2
278 pkg_lists $DATA_B2
279fi
280
281if option sync_b2; then
282 rsync -ar $DATA/ $DATA_B2
283fi
284
285# Give options to run a b2 from scratch after b1 has finished
286if option pkglists_b2; then
287 echo PKGLISTS_B2:
288 cd $DATA_B2
289 pkg_lists $DATA_B2
290fi
291
292if option pkglists_b1; then
293 echo PKGLISTS_B1:
294 cd $DATA
295 pkg_lists $DATA
296fi
297
298if option sync_b2_lite; then
299 lite_sync $DATA $DATA_B2
300fi
301
302if option sync_b1; then
303 rsync -ar $DATA_B2/ $DATA
304fi
305
306if option sync_b1_lite; then
307 lite_sync $DATA_B2 $DATA
308fi
309
310make_hints_branch () {
311 if [ -d "$2" ]; then
312 bzr pull -q -d "$2" --remember --overwrite "$1"
313 else
314 bzr branch "$1" "$2"
315 fi
316}
317
318if option hints_bzr; then
319 echo HINTS_BZR:
320 if [ "$DISTRIBUTION" = ubuntu ]; then
321 if [ "$SERIES" = "$DEFAULT_SERIES" ]; then
322 branch="lp:~ubuntu-release/britney/hints-ubuntu"
323 else
324 branch="lp:~ubuntu-sru/britney/hints-ubuntu-$SERIES"
325 fi
326 make_hints_branch "$DATA_B2/$SERIES-proposed/Hints" "$branch"
327 if [ "$SERIES" = "$DEFAULT_SERIES" ]; then
328 make_hints_branch \
329 "$DATA_B2/$SERIES-proposed/Hints/ubuntu-touch" \
330 "lp:~ubuntu-touch-release/britney/hints-ubuntu-touch"
331 else
332 rm -rf "$DATA_B2/$SERIES-proposed/Hints/ubuntu-touch"
333 fi
334 elif [ "$DISTRIBUTION" = ubuntu-rtm ]; then
335 make_hints_branch \
336 "$DATA_B2/$SERIES-proposed/Hints" \
337 lp:~ubuntu-release/britney/hints-ubuntu-rtm
338 fi
339fi
340
341run_b1 () {
342 cd $UPDATE_OUT
343 ./update_out.py $TESTING $UNSTABLE $TPU
344}
345
346b2_diff () {
347 tmp_dir=`mktemp -d`
348 master="b2"
349 [ "$master" = "b1" ] && secondary="b2" || secondary="b1"
350 (cd $tmp_dir
351
352 sort $TESTING/HeidiResult >HeidiResult_b1
353 sort $B2_OUTPUT/$SERIES/HeidiResult >HeidiResult_b2
354
355 sed -e '/^<p>Generated: /d' $CODE/update_out/update.EXCUSES_py >excuses_b1.html
356 sed -e '/^<p>Generated: /d' $B2_OUTPUT/$SERIES/excuses.html >excuses_b2.html
357
358 touch -m -r $TESTING/HeidiResult HeidiResult_b1
359 touch -m -r $B2_OUTPUT/$SERIES/HeidiResult HeidiResult_b2
360
361 touch -m -r $CODE/update_out/update.EXCUSES_py excuses_b1.html
362 touch -m -r $B2_OUTPUT/$SERIES/excuses.html excuses_b2.html
363
364 # NB: If you remove the "head" below, add a "|| true"; otherwise
365 # set -e may not allow the HeidiResult diff to execute.
366 diff -u excuses_${master}.html excuses_${secondary}.html | head -n 1000
367 diff -u HeidiResult_${master} HeidiResult_${secondary} | head -n 1000)
368 rm -rf $tmp_dir
369}
370
371b2_diff_with_mail () {
372 b2_diff
373 b2_diff | ifne mail -s "britney2 diff `date -u +'%Y-%m-%d %H:%M'`" \
374 -a "Reply-To: britney2@release.debian.org" britney2@release.debian.org
375}
376
377make_b2_config () {
378 local arches archindep
379 arches=`suite_arches $SERIES`
380 archindep=`suite_archindep $SERIES`
381 sed -e "s/^\\(ARCHITECTURES *= \\).*/\\1$arches/" \
382 -e "s/^\\(NOBREAKALL_ARCHES *= \\).*/\\1$archindep/" \
383 "$1" >"$2"
384}
385
386run_b2 () {
387 cd $BASE
388 make_b2_config "$B2_CONFIG" "$B2_CONFIG.$DISTRIBUTION.$SERIES"
389 $CODE_B2/britney.py -c "$B2_CONFIG.$DISTRIBUTION.$SERIES" -v --distribution=$DISTRIBUTION --series=$SERIES
390}
391
392if option run; then
393 echo RUN:
394 run_b2
395fi
396
397if option run_b1; then
398 echo RUN_B1:
399 run_b1
400 # assume that this will have been run after a "run" which is b2
401 b2_diff_with_mail
402fi
403
404if option run_b2; then
405 echo RUN_B2:
406 run_b2
407 b2_diff_with_mail
408fi
409
410if option b2_diff; then
411 b2_diff
412fi
413
414lp_import() {
415 # tell LP about the new delta
416 promote-to-release -l "$LP_SERVICE" -d "$DISTRIBUTION" -s "$SERIES" -v "$1"
417}
418
419save () {
420 cd $BASE
421 echo RESULTS:
422 # process a delta
423 if [ "$1" = "b1" ]; then
424 DELTA="$TESTING/HeidiResultDelta"
425 else
426 DELTA="$B2_OUTPUT/$SERIES/HeidiResultDelta"
427 fi
428 echo Using data from $DELTA
429 lp_import "$DELTA"
430 printf " done\n"
431}
432
433if option save_b1; then
434 save b1
435fi
436if option save || option save_b2; then
437 save b2
438fi
439
440create_uninst_report () {
441 $CODE_B2/britney.py -c $1 --distribution=$DISTRIBUTION --series=$SERIES --print-uninst >$HTML/$2.new
442 echo -e "\n# Generated: `date -uR`" >>$HTML/$2.new
443 mv $HTML/$2.new $HTML/$2
444}
445
446create_ood_report () {
447 $SCRIPTS/ood_report.py $1 >$HTML/$2.new
448 echo -e "\n# Generated: `date -uR`" >>$HTML/$2.new
449 mv $HTML/$2.new $HTML/$2
450}
451
452stats () {
453 echo STATS:
454 mkdir -p "$HTML/$SERIES"
455 if [ "$1" = "b1" ]; then
456 cp $UPDATE_OUT/update.EXCUSES_py $HTML/$SERIES/update_excuses.html
457 cp $UPDATE_OUT/update.OUTPUT_py $HTML/$SERIES/update_output.txt
458 else
459 cp $DATA_B2/output/$SERIES/excuses.html $HTML/$SERIES/update_excuses.html
460 cp $DATA_B2/output/$SERIES/excuses.yaml $HTML/$SERIES/update_excuses.yaml
461 cp $DATA_B2/output/$SERIES/output.txt $HTML/$SERIES/update_output.txt
462 cp $DATA_B2/$SERIES-proposed/Blocks $HTML/$SERIES/blocks.txt
463 fi
464 #gzip -9 < $HTML/$SERIES/update_excuses.html > $HTML/$SERIES/update_excuses.html.gz
465 #gzip -9 < $HTML/$SERIES/update_output.txt > $HTML/$SERIES/update_output.txt.gz
466 mkdir -p "$HTML/update_output/$SERIES/${NOW%/*}"
467 mkdir -p "$HTML/update_excuses/$SERIES/${NOW%/*}"
468 gzip -9c $HTML/$SERIES/update_output.txt > $HTML/update_output/$SERIES/$NOW.txt.gz
469 gzip -9c $HTML/$SERIES/update_excuses.html > $HTML/update_excuses/$SERIES/$NOW.html.gz
470 gzip -9c $HTML/$SERIES/update_excuses.yaml > $HTML/update_excuses/$SERIES/$NOW.yaml.gz
471
472 # TODO: {stable,unstable}_uninst.txt -- needs support in b2, talk to Fabio
473 # TODO: this creates uninstallability reports against b2 results, not b1's
474 make_b2_config "$B2_CONFIG" "$B2_CONFIG.$DISTRIBUTION.$SERIES"
475 make_b2_config "$B2_CONFIG_NOBREAKALL" "$B2_CONFIG_NOBREAKALL.$DISTRIBUTION.$SERIES"
476 create_uninst_report "$B2_CONFIG.$DISTRIBUTION.$SERIES" ${SERIES}_uninst.txt
477 create_uninst_report "$B2_CONFIG_NOBREAKALL.$DISTRIBUTION.$SERIES" ${SERIES}_uninst_full.txt
478
479 #if grep -q -e '-meta-faux' $HTML/testing_uninst.txt; then
480 # echo >&2 'Warning! Some -meta-faux package is uninstallable!'
481 #fi
482
483 #create_ood_report $DATA_B2/stable stable_outdate.txt
484 #create_ood_report $DATA_B2/testing testing_outdate.txt
485
486 $SCRIPTS/backlog-report -o $HTML/$SERIES/update_excuses.csv $HTML/$SERIES/update_excuses.html
487}
488
489if option stats_b1; then
490 stats b1
491fi
492if option stats || option stats_b2; then
493 stats b2
494fi
495
496if option summary; then
497 cd $BASE
498 echo "Out of dates holding up testing:"
499 TERM=vt100 w3m -dump $HTML/$SERIES/update_excuses.html | sed -n 's/^ *[^ ] *[^ ]*out of date on \([^ ]*\): .*$/\1/p' | sort | uniq -c | sort -n
500 echo "Uninstallables holding up testing:"
501 sed < $HTML/$SERIES/update_excuses.html -n 's/^ *<li>[^ ]* (\([^, ]*\),.*) uninstallable.*$/\1/p' | sort | uniq -c | sort -n
502 #echo "wanna-build stats:"
503 #for a in alpha arm hppa hurd-i386 i386 ia64 m68k mips mipsel powerpc s390 sparc; do
504 # printf " %-12s " "$a:"
505 # /srv/wanna-build/bin/wanna-build-statistics --database=$a/build-db |
506 # grep "if also counting" || echo "-"
507 #done
508fi
509
510if option archivedata; then
511 echo ARCHIVE:
512 cd $VAR
513 tar czf data_bkup/data-$(date +"%Y.%m.%d").tgz data/{stable,testing,unstable,testing-proposed-updates}
514fi
515
516echo -n "Finished at: "; date -uR
0517
=== added directory 'fauxpkg'
=== added file 'fauxpkg/FauxPackages'
--- fauxpkg/FauxPackages 1970-01-01 00:00:00 +0000
+++ fauxpkg/FauxPackages 2015-06-09 16:34:27 +0000
@@ -0,0 +1,59 @@
1Package: wine1.4-i386
2Source: wine1.6
3Version: ${unstable-version}
4Architecture: amd64
5Provides: wine-i386
6Multi-Arch: foreign
7
8Package: wine1.6-i386
9Source: wine1.6
10Version: ${unstable-version}
11Architecture: amd64
12Provides: wine-i386
13Multi-Arch: foreign
14
15Package: nspluginviewer
16Source: nspluginwrapper
17Version: ${unstable-version}
18Architecture: amd64 arm64 armhf powerpc ppc64el
19Multi-Arch: foreign
20
21Package: ocaml-mingw-w64-x86-64
22Source: mingw-ocaml
23Version: ${unstable-version}
24Architecture: i386
25Multi-Arch: foreign
26
27Package: claws-mail-extra-plugins-dbg
28Version: 1:1
29
30Package: claws-mail-html2-viewer
31Version: 1:1
32
33Package: ubuntu-emulator-images
34Source: android
35Version: ${unstable-version}
36Architecture: amd64 armhf
37Multi-Arch: foreign
38
39Package: ubuntu-emulator-runtime
40Source: android
41Version: ${unstable-version}
42Architecture: amd64 armhf
43Multi-Arch: foreign
44
45Package: libnss-mdns-i386
46Source: nss-mdns
47Version: ${unstable-version}
48Architecture: amd64
49Multi-Arch: foreign
50
51Package: unity8
52Version: ${unstable-version}
53Architecture: arm64 powerpc ppc64el
54
55Package: liboxideqt-qmlplugin
56Source: oxide-qt
57Version: ${unstable-version}
58Architecture: arm64 powerpc ppc64el
59
060
=== added file 'fauxpkg/README'
--- fauxpkg/README 1970-01-01 00:00:00 +0000
+++ fauxpkg/README 2015-06-09 16:34:27 +0000
@@ -0,0 +1,50 @@
1Files and scripts in this directory are in charge on creating a list of
2"faux" packages for britney. These are packages that do not exist in the
3archive, but which we want britney to think they exist. At the moment
4they are used for:
5
6 - allowing packages in contrib to migrate even if their dependencies
7 are not satisfied within the archive. This is done by creating a
8 faux package for each missing dependency.
9
10 - ensure some packages do not get accidentally removed, because they
11 are needed by parts of testing which don't declare a Depends on
12 them. In particular, to ensure packages needed by the installer and
13 tasksel are not removed. This is done by creating faux packages that
14 do depend on all packages which are not to be removed.
15
16To create packages of the first kind, just append suitable entries to
17the toplevel FauxPackages file. These should have at least a Package and
18Version fields. You can restrict their existance to certain architectures
19by providing an Architecture field with a list of architectures. Other
20fields are propagated verbatim to britney, eg. Provides.
21
22As for the second kind, you don't directly create faux packages, but
23just append the packages which shouldn't be removed to the appropriate
24file under the "noremove.d" directory. Those files just contain a list
25of *binary* packages, one per line. A couple of important notes:
26
27 - if the binary package is not in testing, do not add it: wait until
28 it migrates.
29
30 - if the binary package is not available on i386 (being arch:all would
31 suffice), you have to qualify it with [arch], mentioning one arch in
32 which it is available.
33
34 - the names of the files under noremove.d must be valid as package
35 names, e.g. not contain underscores.
36
37The noremove.d/tasksel file is special: it gets generated automatically
38by the fauxpkg.py script, by running the script with the "update-tasksel"
39command. (Currently is it to be run by hand whenever the tasksel
40maintainer notices our file is out of date.)
41
42One further caveat: if one of these faux meta-packages becomes uninstallable,
43they stop working (since the removal of their dependencies does not increase
44the uninstallability count). The britney script has a mechanism in place to
45warn in case they become uninstallable. When adding packages to the lists,
46make sure to watch the next britney mail, in case you added conflicting
47packages. (In that case, you want to add such packages in different files
48under noremove.d.)
49
50Please don't forget to commit your changes to files in this directory!
051
=== added file 'fauxpkg/fauxpkg.py'
--- fauxpkg/fauxpkg.py 1970-01-01 00:00:00 +0000
+++ fauxpkg/fauxpkg.py 2015-06-09 16:34:27 +0000
@@ -0,0 +1,181 @@
1#! /usr/bin/python
2## encoding: utf-8
3#
4# Copyright (c) 2008 Adeodato Simó (dato@net.com.org.es)
5# Licensed under the terms of the MIT license.
6
7"""Handle the creation of britney faux packages.
8
9This program gets called from the "britney" script in order to append to the
10Packages_<arch> files a list of faux packages. This is done with the "generate"
11command, passing a list of britney suite directories:
12
13 % fauxpkg.py generate /home/release/britney/var/data/{testing,unstable}
14
15This automatically appeds to the Packages files the list of faux packages. See
16the README file in this directory for the input files from which such list is
17generated.
18"""
19
20import os
21import re
22import sys
23import glob
24import tempfile
25import subprocess
26
27import apt_pkg
28apt_pkg.init()
29
30##
31
32BASEDIR = os.path.dirname(__file__)
33
34NOREMOVE_DIR = os.path.join(BASEDIR, 'noremove.d')
35FAUX_PACKAGES = os.path.join(BASEDIR, 'FauxPackages')
36
37DEFAULT_NOREMOVE_ARCH = 'amd64'
38
39##
40
41def main():
42 if not sys.argv[1:]:
43 print >>sys.stderr, 'Usage: %s <generate | update-tasksel> [ britney_suite_dir1 ... ]' % (
44 os.path.basename(sys.argv[0]))
45 sys.exit(1)
46 else:
47 command = sys.argv.pop(1)
48
49 if command == 'generate':
50 if not sys.argv[1:]:
51 print >>sys.stderr, 'E: need at least one britney suite directory'
52 sys.exit(1)
53 else:
54 do_generate(sys.argv[1:])
55 elif command == 'update-tasksel':
56 if sys.argv[1:]:
57 print >>sys.stderr, 'E: extra arguments not allowed'
58 sys.exit(1)
59 else:
60 do_update_tasksel()
61 else:
62 print >>sys.stderr, 'E: unknown command %s' % (command,)
63
64##
65
66def do_generate(directories):
67 arches = set()
68 allfaux = {}
69
70 for dir_ in directories:
71 arches.update([ re.sub(r'^.*/Packages_', '', x)
72 for x in glob.glob(os.path.join(dir_, 'Packages_*')) ])
73
74 unstable_versions = {}
75 for dir_ in directories:
76 with open(os.path.join(dir_, 'Sources')) as f:
77 parser = apt_pkg.TagFile(f)
78 for section in parser:
79 if 'Package' in section and 'Version' in section:
80 unstable_versions[section['Package']] = section['Version']
81
82 # First, FauxPackages
83 try:
84 parser = apt_pkg.TagFile(file(FAUX_PACKAGES))
85 step = parser.step
86 section = parser.section
87 except AttributeError, e:
88 parser = apt_pkg.ParseTagFile(file(FAUX_PACKAGES))
89 step = parser.Step
90 section = parser.Section
91 while step():
92 d = dict(section)
93 d['Section'] = 'faux' # crucial; britney filters HeidiResult based on section
94
95 if not d.has_key('Architecture'):
96 these_arches = arches
97 else:
98 these_arches = set(re.split(r'[, ]+', d['Architecture']))
99
100 d['Architecture'] = 'all' # same everywhere
101
102 if d.get('Version') == '${unstable-version}':
103 source = d.get('Source', d.get('Package'))
104 if source in unstable_versions:
105 d['Version'] = unstable_versions[source]
106
107 for arch in these_arches:
108 allfaux.setdefault(arch, []).append(d)
109
110 # Now, noremove.d
111 for f in glob.glob(os.path.join(NOREMOVE_DIR, '*.list')):
112 pkgs = {}
113 basename = re.sub(r'.+/(.+)\.list', r'\1', f)
114
115 for line in file(f):
116 line = line.strip()
117 if re.match(r'^#', line):
118 continue
119 elif re.match(r'\S+$', line):
120 pkg = line
121 arch = DEFAULT_NOREMOVE_ARCH
122 else:
123 m = re.match(r'(\S+)\s+\[(.+)\]', line)
124 if m:
125 pkg, arch = m.groups()
126 else:
127 print >>sys.stderr, 'W: could not parse line %r' % (line,)
128
129 arch = re.split(r'[, ]+', arch)[0] # just in case
130 pkgs.setdefault(arch, set()).add(pkg)
131
132 for arch in pkgs.keys():
133 d = { 'Package': '%s-meta-faux' % (basename,), 'Version': '1',
134 'Section': 'faux', 'Architecture': '%s' % (arch,),
135 'Depends': ', '.join(pkgs[arch]) }
136 allfaux.setdefault(arch, []).append(d)
137
138 # Write the result
139 for arch in arches:
140 if arch not in allfaux:
141 continue
142 for dir_ in directories:
143 f = os.path.join(dir_, 'Packages_' + arch)
144 if not os.path.exists(f):
145 continue
146 else:
147 f = file(f, 'a')
148 for d in allfaux[arch]:
149 f.write('\n'.join('%s: %s' % (k, v) for k, v in d.iteritems()) + '\n\n')
150
151##
152
153def do_update_tasksel():
154 p = subprocess.Popen('dak ls -f control-suite -s unstable -a source tasksel',
155 shell=True, stdout=subprocess.PIPE)
156 p.wait()
157 version = p.stdout.readline().split()[1]
158
159 p = subprocess.Popen('dak ls -f control-suite -s unstable -S -a i386,all tasksel',
160 shell=True, stdout=subprocess.PIPE)
161 p.wait()
162 tasks = []
163
164 for line in p.stdout:
165 pkg = line.split()[0]
166
167 if pkg.startswith('task-'):
168 tasks.append(pkg)
169
170 # Write the new file
171 tmpfd, tmpname = tempfile.mkstemp(dir=NOREMOVE_DIR)
172 os.write(tmpfd, '# Generated from tasksel-data %s\n' % (version,))
173 os.write(tmpfd, '\n'.join(sorted(tasks)) + '\n')
174 os.close(tmpfd)
175 os.chmod(tmpname, 0644)
176 os.rename(tmpname, os.path.join(NOREMOVE_DIR, 'tasksel.list'))
177
178##
179
180if __name__ == '__main__':
181 main()
0182
=== added directory 'fauxpkg/noremove.d'
=== added file 'fauxpkg/noremove.d/README'
--- fauxpkg/noremove.d/README 1970-01-01 00:00:00 +0000
+++ fauxpkg/noremove.d/README 2015-06-09 16:34:27 +0000
@@ -0,0 +1,3 @@
1Packages listed in these files are protected against accidental removals
2from testing. Please mail debian-release@lists.debian.org if you think
3some package should be added to any of these lists.
04
=== added directory 'scripts'
=== added file 'scripts/backlog-report'
--- scripts/backlog-report 1970-01-01 00:00:00 +0000
+++ scripts/backlog-report 2015-06-09 16:34:27 +0000
@@ -0,0 +1,86 @@
1#! /usr/bin/python
2#
3# Generate a report of the backlog in -proposed.
4
5from __future__ import print_function
6
7__metaclass__ = type
8
9import calendar
10import csv
11import gzip
12import io
13from optparse import OptionParser
14import os
15import re
16import sys
17import time
18
19
20generated_re = re.compile(r"^<p>Generated: (.*)</p>$")
21
22
23def import_excuses(csv_writer, path):
24 generated_time = None
25 valid_candidates = 0
26 not_considered = 0
27
28 if path.endswith(".gz"):
29 binary = gzip.open(path)
30 else:
31 binary = io.open(path, mode="rb")
32 try:
33 with io.BufferedReader(binary) as buffered:
34 with io.TextIOWrapper(buffered) as text:
35 for line in text:
36 match = generated_re.match(line)
37 if match:
38 generated_time = time.strptime(
39 match.group(1), "%Y.%m.%d %H:%M:%S +0000")
40 elif line.startswith("<li>Valid candidate"):
41 valid_candidates += 1
42 elif line.startswith("<li>Not considered"):
43 not_considered += 1
44 finally:
45 binary.close()
46
47 assert generated_time is not None
48
49 csv_writer.writerow({
50 "time": calendar.timegm(generated_time) * 1000,
51 "valid candidates": valid_candidates,
52 "not considered": not_considered,
53 "total": valid_candidates + not_considered,
54 })
55
56
57def main():
58 parser = OptionParser(description="Generate a backlog report.")
59 parser.add_option(
60 "-o", "--output", help="Write output to this file (default: stdout).")
61 options, args = parser.parse_args()
62
63 if options.output is not None:
64 csv_is_new = not os.path.exists(options.output)
65 if sys.version < "3":
66 output = open(options.output, "ab")
67 else:
68 output = open(options.output, "a", newline="")
69 else:
70 csv_is_new = True
71 output = sys.stdout
72
73 try:
74 csv_writer = csv.DictWriter(
75 output, ["time", "valid candidates", "not considered", "total"])
76 if csv_is_new:
77 csv_writer.writeheader()
78 for arg in args:
79 import_excuses(csv_writer, arg)
80 finally:
81 if options.output is not None:
82 output.close()
83
84
85if __name__ == "__main__":
86 main()
087
=== added file 'scripts/ood_report.py'
--- scripts/ood_report.py 1970-01-01 00:00:00 +0000
+++ scripts/ood_report.py 2015-06-09 16:34:27 +0000
@@ -0,0 +1,101 @@
1#! /usr/bin/python
2## encoding: utf-8
3#
4# Copyright (c) 2008 Adeodato Simó (dato@net.com.org.es)
5# Licensed under the terms of the MIT license.
6
7"""Create a report of packages that are out-of-date in each architecture.
8
9It expects a single "directory" argument, that should be a britney directory
10for a distribution, i.e. containing Packages_<arch> files and Sources.
11"""
12
13import os
14import re
15import sys
16import glob
17
18import apt_pkg
19apt_pkg.init()
20
21##
22
23def main():
24 if len(sys.argv) != 2:
25 print >>sys.stderr, 'Usage: %s <directory>'
26 sys.exit(1)
27 else:
28 directory = sys.argv[1]
29
30 pkgfiles = glob.glob(os.path.join(directory, 'Packages_*'))
31 versions = get_src_versions(os.path.join(directory, 'Sources'))
32
33 ood = {} # { arch1: { srcpk1: (oldver, [binpkg1, binpkg2, ...]), ... }, ... }
34
35 for pkgfile in pkgfiles:
36 arch = re.sub(r'^.*/Packages_', '', pkgfile)
37 try:
38 parser = apt_pkg.TagFile(file(pkgfile))
39 step = parser.step
40 get_section = parser.section
41 get_field = parser.section.get
42 except AttributeError, e:
43 parser = apt_pkg.ParseTagFile(file(pkgfile))
44 step = parser.Step
45 get_section = parser.Section
46 get_field = parser.Section.get
47 d = ood[arch] = {}
48
49 while step():
50 pkg = get_section['Package']
51 src = get_field('Source') or pkg
52
53 if ' ' in src:
54 m = re.match(r'(\S+) \((\S+)\)$', src)
55 src = m.group(1)
56 ver = m.group(2)
57 else:
58 ver = re.sub(r'\+b\d+$', '', get_section['Version'])
59
60 try:
61 distver = versions[src]
62 except KeyError:
63 pass # faux package
64 else:
65 if ver != distver:
66 d.setdefault(src, (ver, []))[1].append(pkg)
67
68 arches = sorted(ood.keys())
69
70 for arch in arches:
71 print '* %s' % (arch,)
72 for src, (oldver, binpkgs) in sorted(ood[arch].iteritems()):
73 # do not print binpkgs, I think it clutters the view too much
74 print ' %s (%s)' % (src, oldver)
75 print
76
77 print '* summary'
78 print '\n'.join(map(lambda x: '%4d %s' % (len(ood[x]), x), arches))
79
80##
81
82def get_src_versions(sources_file):
83 """Return a dict { srcname: version, ... }."""
84 mydict = {}
85 try:
86 parser = apt_pkg.TagFile(file(sources_file))
87
88 while parser.step():
89 mydict[parser.section['Package']] = parser.section['Version']
90 except AttributeError, e:
91 parser = apt_pkg.ParseTagFile(file(sources_file))
92
93 while parser.Step():
94 mydict[parser.Section['Package']] = parser.Section['Version']
95
96 return mydict
97
98##
99
100if __name__ == '__main__':
101 main()
0102
=== added file 'scripts/udeb-report'
--- scripts/udeb-report 1970-01-01 00:00:00 +0000
+++ scripts/udeb-report 2015-06-09 16:34:27 +0000
@@ -0,0 +1,49 @@
1#!/bin/sh
2
3# Generate a report with out-of-date udebs. Queries come from the
4# original "d-i" script by Jeroen van Wolffelaar. Modified not to emit
5# output for queries that return 0 rows by Adeodato Simó.
6
7# TODO(dato): rewrite these queries.
8
9export PAGER="cat"
10
11eval $(dak admin config db-shell)
12
13##
14
15maybe_print () {
16 if ! echo "$2" | grep -q -E '^\(0 rows\)$'; then
17 echo "$1"
18 echo "$2"
19 echo
20 fi
21}
22
23maybe_print "udeb's in testing that don't (anymore) correspond to any testing source:" \
24"`psql -c \"select b.package, b.version, (SELECT arch_string from
25architecture where b.architecture=architecture.id) as arch, s.source from
26bin_associations ba LEFT JOIN binaries b on (ba.bin=b.id) LEFT JOIN source s
27on (b.source=s.id) WHERE ba.suite=4 AND s.id NOT IN (SELECT source from
28src_associations WHERE suite=4) AND b.type = 'udeb' ORDER BY s.source,
29b.package, b.architecture;\"`"
30
31maybe_print "udeb's in unstable that should be in testing too:" \
32"`psql -c \"select b.package, b.version, (SELECT arch_string from
33architecture where b.architecture=architecture.id) as arch, s.source from
34bin_associations ba LEFT JOIN binaries b on (ba.bin=b.id) LEFT JOIN source s
35on (b.source=s.id) WHERE ba.suite=5 AND NOT EXISTS (SELECT 1 FROM
36bin_associations ba2 WHERE ba2.suite=4 AND ba2.bin=ba.bin) AND s.id IN (SELECT
37source from src_associations WHERE suite=4) AND b.type = 'udeb' AND
38b.architecture IN (SELECT architecture from suite_architectures where suite = 4)
39ORDER BY s.source, b.package, b.architecture;\"`"
40
41maybe_print "udeb's in t-p-u that should be in testing too:" \
42"`psql -c \"select b.package, b.version, (SELECT arch_string from
43architecture where b.architecture=architecture.id) as arch, s.source from
44bin_associations ba LEFT JOIN binaries b on (ba.bin=b.id) LEFT JOIN source s
45on (b.source=s.id) WHERE ba.suite=3 AND NOT EXISTS (SELECT 1 FROM
46bin_associations ba2 WHERE ba2.suite=4 AND ba2.bin=ba.bin) AND s.id IN (SELECT
47source from src_associations WHERE suite=4) AND b.type = 'udeb' AND
48b.architecture IN (SELECT architecture from suite_architectures where suite = 4)
49ORDER BY s.source, b.package, b.architecture;\"`"
050
=== added directory 'update_out'
=== added file 'update_out/.gitignore'
--- update_out/.gitignore 1970-01-01 00:00:00 +0000
+++ update_out/.gitignore 2015-06-09 16:34:27 +0000
@@ -0,0 +1,6 @@
1*.o
2*.so
3/oldstuff
4/Makefile.dep
5/update.EXCUSES_py
6/update.OUTPUT_py
07
=== added file 'update_out/Makefile'
--- update_out/Makefile 1970-01-01 00:00:00 +0000
+++ update_out/Makefile 2015-06-09 16:34:27 +0000
@@ -0,0 +1,33 @@
1
2CC = gcc
3CXX = g++
4CFLAGS = -fPIC -Wall -W -O2 -DNDEBUG -DMDEBUG0 -g -p # -DDIAGNOSE
5CXXFLAGS = $(CFLAGS)
6
7all : britneymodule.so # libajdpkg.a aptvercmp checklib freelist
8
9clean :
10 rm -f *.so *.o *~ Makefile.dep gmon.out
11 rm -f freelist aptvercmp checklib libajdpkg.a
12
13checklib : checklib.o dpkg.o dpkg-lib.o memory3.o freelist.o assert.o
14 $(CC) $(CFLAGS) -o checklib $^ -lapt-pkg # -lccmalloc -ldl
15
16aptvercmp : dpkg-lib.cpp
17 $(CXX) $(CFLAGS) -DTESTBIN -o aptvercmp dpkg-lib.cpp -lapt-pkg
18
19freelist : freelist.c assert.o
20 $(CC) $(CFLAGS) -DTESTBIN -o $@ $^
21
22#libajdpkg.a : dpkg.o dpkg-lib.o memory3.o freelist.o assert.o
23# ar rv $@ $^
24# ranlib $@
25
26britneymodule.so : britney-py.o dpkg.o dpkg-lib.o memory3.o freelist.o assert.o
27 $(CXX) -shared -o britneymodule.so $^ -lapt-pkg
28
29Makefile.dep :
30 @gcc -MM *.c *.cpp > Makefile.dep
31 @echo Makefile.dep : Makefile *.c *.h >> Makefile.dep
32
33-include Makefile.dep
034
=== added file 'update_out/README'
--- update_out/README 1970-01-01 00:00:00 +0000
+++ update_out/README 2015-06-09 16:34:27 +0000
@@ -0,0 +1,32 @@
1
2BUILDING
3========
4
5Install libapt-pkg-dev
6
7testing/ $ make
8testing/ $ mkdir old cur out
9testing/ $ cd testing
10testing/testing/ $ perl Makefile.PL
11testing/testing/ $ make
12
13Add old and new packages files into old and cur, respectively.
14
15testing/ $ ./checklib i386 alpha
16
17Will generate some test stuff in out/
18
19TODO
20====
21
22Need some way of actually updating archives.
23Need some way of judging differences between Packages files.
24 (so I can see what hasn't been updated and work out why;
25 so I can check that Packages.gz matches dpkg-scanpackages output)
26Need some way of automatically explaining why packages aren't upgraded.
27 (shouldn't be hard?)
28
29BUGS
30====
31
32out/ directory must exist for checklib, or segfault
033
=== added file 'update_out/assert.c'
--- update_out/assert.c 1970-01-01 00:00:00 +0000
+++ update_out/assert.c 2015-06-09 16:34:27 +0000
@@ -0,0 +1,11 @@
1#include <stdio.h>
2#include <stdlib.h>
3
4int _myassertbug(int line, char *file, char *err) {
5 fprintf(stderr, "Assertion failed: %s:%d: %s\n", file, line, err);
6 fprintf(stderr, "I HATE YOU!!!");
7 ((void(*)())0)();
8 abort();
9 return 0;
10}
11
012
=== added file 'update_out/britney-py.c'
--- update_out/britney-py.c 1970-01-01 00:00:00 +0000
+++ update_out/britney-py.c 2015-06-09 16:34:27 +0000
@@ -0,0 +1,871 @@
1#include <python2.7/Python.h>
2
3#include "dpkg.h"
4
5#define MAKE_PY_LIST(L,S,E,I,V) \
6 do { \
7 L = PyList_New(0); \
8 if (!L) break; \
9 for (S; E; I) { \
10 PyObject *EL; \
11 EL = Py_BuildValue V; \
12 if (!EL) { \
13 Py_DECREF(L); \
14 L = NULL; \
15 break; \
16 } \
17 PyList_Append(L, EL); \
18 Py_DECREF(EL); \
19 } \
20 if (L) PyList_Sort(L); \
21 } while(0)
22
23/**************************************************************************
24 * britney.Packages -- dpkg_packages wrapper
25 *******************************************/
26
27typedef enum { DONTFREE, FREE } dpkgpackages_freeme;
28typedef struct {
29 PyObject_HEAD
30 dpkg_packages *pkgs;
31 PyObject *ref; /* object packages are "in" */
32 dpkgpackages_freeme freeme; /* free pkgs when deallocing? */
33} dpkgpackages;
34
35staticforward PyTypeObject Packages_Type;
36
37static PyObject *dpkgpackages_new(dpkg_packages *pkgs,
38 dpkgpackages_freeme freeme, PyObject *ref)
39{
40 dpkgpackages *res;
41
42 res = PyObject_NEW(dpkgpackages, &Packages_Type);
43 if (res == NULL) return NULL;
44
45 res->pkgs = pkgs;
46 res->ref = ref; Py_INCREF(res->ref);
47 res->freeme = freeme;
48
49 return (PyObject *) res;
50}
51
52static void dpkgpackages_dealloc(dpkgpackages *self) {
53 if (self->freeme == FREE) free_packages(self->pkgs);
54 Py_XDECREF(self->ref);
55 self->pkgs = NULL;
56 self->ref = NULL;
57 PyObject_DEL(self);
58}
59
60
61static dpkg_collected_package *dpkgpackages_lookuppkg(dpkgpackages *self,
62 char *pkgname)
63{
64 dpkg_collected_package *cpkg = NULL;
65 cpkg = lookup_packagetbl(self->pkgs->packages, pkgname);
66 if (!cpkg) {
67 PyErr_SetString(PyExc_ValueError, "Not a valid package");
68 }
69 return cpkg;
70}
71
72static PyObject *dpkgpackages_ispresent(dpkgpackages *self, PyObject *args) {
73 dpkg_collected_package *cpkg;
74 char *pkgname;
75 if (!PyArg_ParseTuple(args, "s", &pkgname)) return NULL;
76 cpkg = lookup_packagetbl(self->pkgs->packages, pkgname);
77 return cpkg ? Py_BuildValue("i", 1) : Py_BuildValue("i", 0);
78}
79
80static PyObject *dpkgpackages_getversion(dpkgpackages *self, PyObject *args) {
81 dpkg_collected_package *cpkg;
82 char *pkgname;
83 if (!PyArg_ParseTuple(args, "s", &pkgname)) return NULL;
84 cpkg = lookup_packagetbl(self->pkgs->packages, pkgname);
85 if (cpkg) return Py_BuildValue("s", cpkg->pkg->version);
86 else return Py_BuildValue("");
87}
88static PyObject *dpkgpackages_getsource(dpkgpackages *self, PyObject *args) {
89 dpkg_collected_package *cpkg;
90 char *pkgname;
91 if (!PyArg_ParseTuple(args, "s", &pkgname)) return NULL;
92 cpkg = lookup_packagetbl(self->pkgs->packages, pkgname);
93 if (cpkg) return Py_BuildValue("s", cpkg->pkg->source);
94 else return Py_BuildValue("");
95}
96static PyObject *dpkgpackages_getsourcever(dpkgpackages *self, PyObject *args) {
97 dpkg_collected_package *cpkg;
98 char *pkgname;
99 if (!PyArg_ParseTuple(args, "s", &pkgname)) return NULL;
100 cpkg = lookup_packagetbl(self->pkgs->packages, pkgname);
101 if (cpkg) return Py_BuildValue("s", cpkg->pkg->source_ver);
102 else return Py_BuildValue("");
103}
104static PyObject *dpkgpackages_isarchall(dpkgpackages *self, PyObject *args) {
105 dpkg_collected_package *cpkg;
106 char *pkgname;
107 if (!PyArg_ParseTuple(args, "s", &pkgname)) return NULL;
108 cpkg = lookup_packagetbl(self->pkgs->packages, pkgname);
109 if (cpkg) return Py_BuildValue("i", cpkg->pkg->arch_all);
110 else return Py_BuildValue("");
111}
112static PyObject *dpkgpackages_isntarchall(dpkgpackages *self, PyObject *args) {
113 dpkg_collected_package *cpkg;
114 char *pkgname;
115 if (!PyArg_ParseTuple(args, "s", &pkgname)) return NULL;
116 cpkg = lookup_packagetbl(self->pkgs->packages, pkgname);
117 if (cpkg) return Py_BuildValue("i", !cpkg->pkg->arch_all);
118 else return Py_BuildValue("");
119}
120static PyObject *dpkgpackages_getfield(dpkgpackages *self, PyObject *args) {
121 char *field;
122 char *pkgname;
123 int i;
124 dpkg_collected_package *cpkg;
125 dpkg_paragraph *para;
126 if (!PyArg_ParseTuple(args, "ss", &pkgname, &field)) return NULL;
127 cpkg = dpkgpackages_lookuppkg(self, pkgname);
128 if (!cpkg) return NULL;
129 para = cpkg->pkg->details;
130 for (i = 0; i < para->n_entries; i++) {
131 if (strcasecmp(para->entry[i].name, field) == 0) {
132 return Py_BuildValue("s", para->entry[i].value);
133 }
134 }
135 return Py_BuildValue("");
136}
137static PyObject *dpkgpackages_isinstallable(dpkgpackages *self, PyObject *args)
138{
139 char *pkgname;
140 if (!PyArg_ParseTuple(args, "s", &pkgname)) return NULL;
141 if (checkinstallable2(self->pkgs, pkgname)) {
142 return Py_BuildValue("i", 1);
143 } else {
144 return Py_BuildValue("");
145 }
146}
147static PyObject *dpkgpackages_isuninstallable(dpkgpackages *self,
148 PyObject *args)
149{
150 char *pkgname;
151 if (!PyArg_ParseTuple(args, "s", &pkgname)) return NULL;
152 if (!checkinstallable2(self->pkgs, pkgname)) {
153 return Py_BuildValue("i", 1);
154 } else {
155 return Py_BuildValue("");
156 }
157}
158static PyObject *dpkgpackages_unsatdeps(dpkgpackages *self, PyObject *args) {
159 /* arguments are:
160 * testingpkgs[arch].unsatisfiable_deps(unstablepkgs[arch], "netbase", "Depends")
161 * exciting, huh?
162 */
163
164 dpkgpackages *pkgpkgs;
165 char *pkgname, *fieldname;
166 dpkg_collected_package *cpkg;
167 int fieldidx;
168 int buflen = 100;
169 char *buf = malloc(buflen);
170 const char *fields[] = { "Pre-Depends", "Depends", "Recommends",
171 "Suggests", NULL };
172 satisfieddeplist *unsatdeps, *dl;
173 PyObject *res = Py_BuildValue("[]");
174
175 if (!PyArg_ParseTuple(args, "O!ss", &Packages_Type, &pkgpkgs, &pkgname, &fieldname)) return NULL;
176
177 cpkg = lookup_packagetbl(pkgpkgs->pkgs->packages, pkgname);
178 if (!cpkg) {
179 PyErr_SetString(PyExc_ValueError, "Not a valid package");
180 return NULL;
181 }
182
183 for (fieldidx = 0; fields[fieldidx]; fieldidx++) {
184 if (strcmp(fields[fieldidx], fieldname) == 0) break;
185 }
186 if (!fields[fieldidx]) {
187 PyErr_SetString(PyExc_ValueError, "Not a valid dependency field");
188 return NULL;
189 }
190
191 unsatdeps = checkunsatisfiabledeps(self->pkgs,
192 cpkg->pkg->depends[fieldidx]);
193 for (dl = unsatdeps; dl != NULL; dl = dl->next) {
194 int len;
195 packagelist *it;
196 PyObject *pkglist;
197 deplist *depl;
198 dependency *dep;
199
200 len = 0;
201 buf[0] = '\0';
202 for (depl = dl->value->depl; depl; depl = depl->next) {
203 dep = depl->value;
204 len += strlen(dep->package) + 4;
205 /* 4 = strlen(" | ") + 1 */
206 if (dep->op != dr_NOOP) {
207 len += strlen(dep->version) + 6;
208 /* 6 = strlen(" (>= )") */
209 }
210 if (len >= buflen) {
211 char *newbuf;
212 newbuf = realloc(buf, len + 100);
213 if (newbuf == NULL) {
214 free_satisfieddeplist(unsatdeps);
215 free(buf);
216 Py_DECREF(res);
217 PyErr_SetFromErrno(PyExc_MemoryError);
218 return NULL;
219 }
220 buf = newbuf;
221 buflen = len + 100;
222 }
223 if (buf[0] != '\0') strcat(buf, " | ");
224 strcat(buf, dep->package);
225 if (dep->op != dr_NOOP) {
226 sprintf(buf + strlen(buf), " (%s %s)",
227 dependency_relation_sym[dep->op],
228 dep->version);
229 }
230 }
231
232 MAKE_PY_LIST(pkglist, it = dl->value->pkgs, it, it = it->next,
233 ("s", it->value->package)
234 );
235
236 {
237 PyObject *depel = Py_BuildValue("(sN)", buf, pkglist);
238 PyList_Append(res, depel);
239 Py_DECREF(depel);
240 }
241 }
242
243 free_satisfieddeplist(unsatdeps);
244 free(buf);
245
246 return res;
247}
248
249static PyObject *dpkgpackages_getattr(dpkgpackages *self, char *name) {
250 static struct PyMethodDef dpkgsources_methods[] = {
251 { "is_present", (binaryfunc) dpkgpackages_ispresent,
252 METH_VARARGS, NULL },
253 { "get_version", (binaryfunc) dpkgpackages_getversion,
254 METH_VARARGS, NULL },
255 { "get_source", (binaryfunc) dpkgpackages_getsource,
256 METH_VARARGS, NULL },
257 { "get_sourcever", (binaryfunc) dpkgpackages_getsourcever,
258 METH_VARARGS, NULL },
259 { "is_arch_all", (binaryfunc) dpkgpackages_isarchall,
260 METH_VARARGS, NULL },
261 { "isnt_arch_all", (binaryfunc) dpkgpackages_isntarchall,
262 METH_VARARGS, NULL },
263 { "get_field", (binaryfunc) dpkgpackages_getfield,
264 METH_VARARGS, NULL },
265 { "is_installable", (binaryfunc) dpkgpackages_isinstallable,
266 METH_VARARGS, NULL },
267 { "is_uninstallable", (binaryfunc)dpkgpackages_isuninstallable,
268 METH_VARARGS, NULL },
269 { "unsatisfiable_deps", (binaryfunc) dpkgpackages_unsatdeps,
270 METH_VARARGS, NULL },
271 { NULL, NULL, 0, NULL }
272 };
273
274 if (strcmp(name, "packages") == 0) {
275 PyObject *packages;
276 packagetbl_iter it;
277 MAKE_PY_LIST(packages,
278 it = first_packagetbl(self->pkgs->packages),
279 !done_packagetbl(it), it = next_packagetbl(it),
280 ("s", it.k)
281 );
282 return packages;
283 }
284
285 return Py_FindMethod(dpkgsources_methods, (PyObject *)self, name);
286}
287
288static PyTypeObject Packages_Type = {
289 PyObject_HEAD_INIT(&PyType_Type)
290
291 0, /* ob_size (0) */
292 "Packages", /* type name */
293 sizeof(dpkgpackages), /* basicsize */
294 0, /* itemsize (0) */
295
296 (destructor) dpkgpackages_dealloc,
297 (printfunc) 0,
298 (getattrfunc) dpkgpackages_getattr,
299 (setattrfunc) 0,
300 (cmpfunc) 0,
301 (reprfunc) 0,
302
303 0, /* number methods */
304 0, /* sequence methods */
305 0, /* mapping methods */
306
307 (hashfunc) 0, /* dict[x] ?? */
308 (ternaryfunc) 0, /* x() */
309 (reprfunc) 0 /* str(x) */
310};
311
312/**************************************************************************
313 * britney.Sources -- dpkg_sources wrapper
314 *****************************************/
315
316typedef struct {
317 PyObject_HEAD
318 dpkg_sources *srcs;
319} dpkgsources;
320
321staticforward PyTypeObject Sources_Type;
322
323static PyObject *dpkgsources_new(PyObject *self, PyObject *args) {
324 dpkgsources *res = NULL;
325 char *dir;
326 PyObject *arches;
327 char **archesStr = NULL;
328 int i, count;
329
330 (void)self; /* unused */
331
332 if (!PyArg_ParseTuple(args, "sO!", &dir, &PyList_Type, &arches)) {
333 goto end;
334 }
335
336 count = PyList_Size(arches);
337 if (count <= 0) {
338 PyErr_SetString(PyExc_TypeError, "No architectures specified");
339 goto end;
340 }
341
342 archesStr = malloc(sizeof(char *) * count);
343 if (!archesStr) {
344 PyErr_SetFromErrno(PyExc_MemoryError);
345 goto end;
346 }
347
348 for (i = 0; i < count; i++) {
349 PyObject *arch = PyList_GetItem(arches, i);
350 if (!PyString_Check(arch)) {
351 goto end;
352 }
353 archesStr[i] = PyString_AsString(arch);
354 }
355
356 res = PyObject_NEW(dpkgsources, &Sources_Type);
357 if (res == NULL) goto end;
358
359 res->srcs = read_directory(dir, count, archesStr);
360 if (!res->srcs) {
361 Py_DECREF(res);
362 res = NULL;
363 goto end;
364 }
365
366end:
367 if (archesStr) free(archesStr);
368 return (PyObject *) res;
369}
370
371static void dpkgsources_dealloc(dpkgsources *self) {
372 free_sources(self->srcs);
373 self->srcs = NULL;
374 PyObject_DEL(self);
375}
376
377static PyObject *dpkgsources_packages(dpkgsources *self, PyObject *args)
378{
379 char *arch;
380 dpkg_packages *pkgs;
381 if (!PyArg_ParseTuple(args, "s", &arch)) return NULL;
382 pkgs = get_architecture(self->srcs, arch);
383 return dpkgpackages_new(pkgs, FREE, (PyObject *) self);
384}
385
386static PyObject *dpkgsources_isfake(dpkgsources *self, PyObject *args) {
387 char *srcname;
388 dpkg_source *src;
389
390 if (!PyArg_ParseTuple(args, "s", &srcname)) return NULL;
391 src = lookup_sourcetbl(self->srcs->sources, srcname);
392 if (src) return Py_BuildValue("i", src->fake);
393 else return Py_BuildValue("");
394}
395
396static PyObject *dpkgsources_getversion(dpkgsources *self, PyObject *args) {
397 char *srcname;
398 dpkg_source *src;
399
400 if (!PyArg_ParseTuple(args, "s", &srcname)) return NULL;
401 src = lookup_sourcetbl(self->srcs->sources, srcname);
402 if (src) return Py_BuildValue("s", src->version);
403 else return Py_BuildValue("");
404}
405
406static PyObject *dpkgsources_getfield(dpkgsources *self, PyObject *args) {
407 char *srcname, *field;
408 dpkg_source *src;
409 int i;
410 dpkg_paragraph *para;
411
412 if (!PyArg_ParseTuple(args, "ss", &srcname, &field)) return NULL;
413 src = lookup_sourcetbl(self->srcs->sources, srcname);
414 if (!src) {
415 PyErr_SetString(PyExc_ValueError, "Not a valid source package");
416 return NULL;
417 }
418 para = src->details;
419 if (para) {
420 for (i = 0; i < para->n_entries; i++) {
421 if (strcasecmp(para->entry[i].name, field) == 0) {
422 return Py_BuildValue("s", para->entry[i].value);
423 }
424 }
425 }
426 return Py_BuildValue("");
427}
428
429static PyObject *dpkgsources_ispresent(dpkgsources *self, PyObject *args) {
430 char *srcname;
431 if (!PyArg_ParseTuple(args, "s", &srcname)) return NULL;
432 if (lookup_sourcetbl(self->srcs->sources, srcname)) {
433 return Py_BuildValue("i", 1);
434 } else {
435 return Py_BuildValue("i", 0);
436 }
437}
438
439static PyObject *dpkgsources_binaries(dpkgsources *self, PyObject *args) {
440 char *srcname, *arch;
441 int archnum;
442 dpkg_source *src;
443 PyObject *res;
444 ownedpackagelist *p;
445
446 if (!PyArg_ParseTuple(args, "ss", &srcname, &arch)) return NULL;
447
448 for (archnum = 0; archnum < self->srcs->n_arches; archnum++) {
449 if (strcmp(arch, self->srcs->archname[archnum]) == 0) break;
450 }
451 if (archnum == self->srcs->n_arches) {
452 PyErr_SetString(PyExc_ValueError, "Not a valid architecture");
453 return NULL;
454 }
455
456 src = lookup_sourcetbl(self->srcs->sources, srcname);
457 if (src == NULL) {
458 PyErr_SetString(PyExc_ValueError, "Not a valid source package");
459 return NULL;
460 }
461
462 MAKE_PY_LIST(res, p = src->packages[archnum], p, p = p->next,
463 ("s", p->value->package)
464 );
465 return res;
466}
467
468static PyObject *dpkgsources_getattr(dpkgsources *self, char *name) {
469 static struct PyMethodDef dpkgsources_methods[] = {
470 { "Packages", (binaryfunc) dpkgsources_packages,
471 METH_VARARGS, NULL },
472 { "is_fake", (binaryfunc) dpkgsources_isfake,
473 METH_VARARGS, NULL },
474 { "get_version", (binaryfunc) dpkgsources_getversion,
475 METH_VARARGS, NULL },
476 { "get_field", (binaryfunc) dpkgsources_getfield,
477 METH_VARARGS, NULL },
478 { "is_present", (binaryfunc) dpkgsources_ispresent,
479 METH_VARARGS, NULL },
480 { "binaries", (binaryfunc) dpkgsources_binaries,
481 METH_VARARGS, NULL },
482 { NULL, NULL, 0, NULL }
483 };
484
485 if (strcmp(name, "arches") == 0) {
486 PyObject *arches;
487 int i;
488 MAKE_PY_LIST(arches, i = 0, i < self->srcs->n_arches, i++,
489 ("s", self->srcs->archname[i])
490 );
491 return arches;
492 } else if (strcmp(name, "sources") == 0) {
493 PyObject *sources;
494 sourcetbl_iter it;
495 MAKE_PY_LIST(sources,
496 it = first_sourcetbl(self->srcs->sources),
497 !done_sourcetbl(it), it = next_sourcetbl(it),
498 ("s", it.k)
499 );
500 return sources;
501 }
502
503 return Py_FindMethod(dpkgsources_methods, (PyObject *)self, name);
504}
505
506static PyTypeObject Sources_Type = {
507 PyObject_HEAD_INIT(&PyType_Type)
508
509 0, /* ob_size (0) */
510 "Sources", /* type name */
511 sizeof(dpkgsources), /* basicsize */
512 0, /* itemsize (0) */
513
514 (destructor) dpkgsources_dealloc,
515 (printfunc) 0,
516 (getattrfunc) dpkgsources_getattr,
517 (setattrfunc) 0,
518 (cmpfunc) 0,
519 (reprfunc) 0,
520
521 0, /* number methods */
522 0, /* sequence methods */
523 0, /* mapping methods */
524
525 (hashfunc) 0, /* dict[x] ?? */
526 (ternaryfunc) 0, /* x() */
527 (reprfunc) 0 /* str(x) */
528};
529
530/**************************************************************************
531 * britney.SourcesNote -- dpkg_sourcesnote wrapper
532 *************************************************/
533
534typedef struct {
535 PyObject_HEAD
536 dpkg_sources_note *srcsn;
537 PyObject *refs; /* list of referenced dpkgsources */
538} dpkgsrcsn;
539
540staticforward PyTypeObject SourcesNote_Type;
541
542static PyObject *dpkgsrcsn_new(PyObject *self, PyObject *args) {
543 dpkgsrcsn *res = NULL;
544 PyObject *arches;
545 char **archesStr = NULL;
546 int i, count;
547
548 (void)self; /* unused */
549
550 if (!PyArg_ParseTuple(args, "O!", &PyList_Type, &arches)) {
551 goto end;
552 }
553
554 count = PyList_Size(arches);
555 if (count <= 0) {
556 PyErr_SetString(PyExc_TypeError, "No architectures specified");
557 goto end;
558 }
559
560 archesStr = malloc(sizeof(char *) * count);
561 if (!archesStr) {
562 PyErr_SetFromErrno(PyExc_MemoryError);
563 goto end;
564 }
565
566 for (i = 0; i < count; i++) {
567 PyObject *arch = PyList_GetItem(arches, i);
568 if (!PyString_Check(arch)) {
569 goto end;
570 }
571 archesStr[i] = PyString_AsString(arch);
572 }
573
574 res = PyObject_NEW(dpkgsrcsn, &SourcesNote_Type);
575 if (res == NULL) goto end;
576
577 res->refs = PyList_New(0);
578 res->srcsn = new_sources_note(count, archesStr);
579 if (!res->refs || !res->srcsn) {
580 Py_DECREF(res);
581 res = NULL;
582 goto end;
583 }
584
585end:
586 if (archesStr) free(archesStr);
587 return (PyObject *) res;
588}
589
590static void dpkgsrcsn_dealloc(dpkgsrcsn *self) {
591 if (self->srcsn) free_sources_note(self->srcsn);
592 self->srcsn = NULL;
593 Py_XDECREF(self->refs);
594 self->refs = NULL;
595
596 PyObject_DEL(self);
597}
598
599static PyObject *dpkgsrcsn_removesource(dpkgsrcsn *self, PyObject *args) {
600 char *name;
601 if (!PyArg_ParseTuple(args, "s", &name)) return NULL;
602 remove_source(self->srcsn, name);
603 return Py_BuildValue("");
604}
605static PyObject *dpkgsrcsn_upgradesource(dpkgsrcsn *self, PyObject *args) {
606 char *name;
607 dpkgsources *srcs;
608 dpkg_source *src;
609 if (!PyArg_ParseTuple(args, "O!s", &Sources_Type, &srcs, &name))
610 return NULL;
611 src = lookup_sourcetbl(srcs->srcs->sources, name);
612 if (!src) {
613 PyErr_SetString(PyExc_ValueError, "Source does not exist");
614 return NULL;
615 }
616 if (!PySequence_In(self->refs, (PyObject *)srcs))
617 PyList_Append(self->refs, (PyObject *)srcs);
618 upgrade_source(self->srcsn, src);
619 return Py_BuildValue("");
620}
621static PyObject *dpkgsrcsn_upgradearch(dpkgsrcsn *self, PyObject *args) {
622 char *name, *arch;
623 dpkgsources *srcs;
624 dpkg_source *src;
625 if (!PyArg_ParseTuple(args, "O!ss", &Sources_Type, &srcs, &name, &arch))
626 return NULL;
627 src = lookup_sourcetbl(srcs->srcs->sources, name);
628 if (!src) {
629 PyErr_SetString(PyExc_ValueError, "Source does not exist");
630 return NULL;
631 }
632 if (!PySequence_In(self->refs, (PyObject *)srcs))
633 PyList_Append(self->refs, (PyObject *)srcs);
634 upgrade_arch(self->srcsn, src, arch);
635 return Py_BuildValue("");
636}
637
638static PyObject *dpkgsrcsn_undochange(dpkgsrcsn *self, PyObject *args) {
639 if (!PyArg_ParseTuple(args, "")) return NULL;
640 undo_change(self->srcsn);
641 return Py_BuildValue("");
642}
643
644static PyObject *dpkgsrcsn_commitchanges(dpkgsrcsn *self, PyObject *args) {
645 if (!PyArg_ParseTuple(args, "")) return NULL;
646 commit_changes(self->srcsn);
647 return Py_BuildValue("");
648}
649
650static PyObject *dpkgsrcsn_writenotes(dpkgsrcsn *self, PyObject *args) {
651 char *dir;
652 if (!PyArg_ParseTuple(args, "s", &dir)) return NULL;
653 write_notes(dir, self->srcsn);
654 return Py_BuildValue("");
655}
656
657static PyObject *dpkgsrcsn_packages(dpkgsrcsn *self, PyObject *args) {
658 char *arch;
659 int archnum;
660 if (!PyArg_ParseTuple(args, "s", &arch)) return NULL;
661 for (archnum = 0; archnum < self->srcsn->n_arches; archnum++) {
662 if (strcmp(arch, self->srcsn->archname[archnum]) == 0) break;
663 }
664 if (archnum == self->srcsn->n_arches) {
665 PyErr_SetString(PyExc_ValueError, "Not a valid architecture");
666 return NULL;
667 }
668 return dpkgpackages_new(self->srcsn->pkgs[archnum], DONTFREE,
669 (PyObject *) self);
670}
671
672static PyObject *dpkgsrcsn_getversion(dpkgsrcsn *self, PyObject *args) {
673 char *srcname;
674 dpkg_source_note *srcn;
675
676 if (!PyArg_ParseTuple(args, "s", &srcname)) return NULL;
677 srcn = lookup_sourcenotetbl(self->srcsn->sources, srcname);
678 if (srcn) return Py_BuildValue("s", srcn->source->version);
679 else return Py_BuildValue("");
680}
681static PyObject *dpkgsrcsn_getfield(dpkgsrcsn *self, PyObject *args) {
682 char *srcname, *field;
683 dpkg_source_note *srcn;
684 int i;
685 dpkg_paragraph *para;
686
687 if (!PyArg_ParseTuple(args, "ss", &srcname, &field)) return NULL;
688 srcn = lookup_sourcenotetbl(self->srcsn->sources, srcname);
689 if (!srcn) {
690 PyErr_SetString(PyExc_ValueError, "Not a valid source package");
691 return NULL;
692 }
693 para = srcn->source->details;
694 if (para) {
695 for (i = 0; i < para->n_entries; i++) {
696 if (strcasecmp(para->entry[i].name, field) == 0) {
697 return Py_BuildValue("s", para->entry[i].value);
698 }
699 }
700 }
701 return Py_BuildValue("");
702}
703static PyObject *dpkgsrcsn_ispresent(dpkgsrcsn *self, PyObject *args) {
704 char *srcname;
705 if (!PyArg_ParseTuple(args, "s", &srcname)) return NULL;
706 if (lookup_sourcenotetbl(self->srcsn->sources, srcname)) {
707 return Py_BuildValue("i", 1);
708 } else {
709 return Py_BuildValue("i", 0);
710 }
711}
712
713static PyObject *dpkgsrcsn_isfake(dpkgsrcsn *self, PyObject *args) {
714 char *srcname;
715 dpkg_source_note *srcn;
716
717 if (!PyArg_ParseTuple(args, "s", &srcname)) return NULL;
718 srcn = lookup_sourcenotetbl(self->srcsn->sources, srcname);
719 if (srcn) return Py_BuildValue("i", srcn->source->fake);
720 else return Py_BuildValue("");
721}
722
723static PyObject *dpkgsrcsn_binaries(dpkgsrcsn *self, PyObject *args) {
724 char *srcname, *arch;
725 int archnum;
726 dpkg_source_note *srcn;
727 PyObject *res;
728 packagelist *p;
729
730 if (!PyArg_ParseTuple(args, "ss", &srcname, &arch)) return NULL;
731
732 for (archnum = 0; archnum < self->srcsn->n_arches; archnum++) {
733 if (strcmp(arch, self->srcsn->archname[archnum]) == 0) break;
734 }
735 if (archnum == self->srcsn->n_arches) {
736 PyErr_SetString(PyExc_ValueError, "Not a valid architecture");
737 return NULL;
738 }
739
740 srcn = lookup_sourcenotetbl(self->srcsn->sources, srcname);
741 if (srcn == NULL) {
742 PyErr_SetString(PyExc_ValueError, "Not a valid source package");
743 return NULL;
744 }
745
746 MAKE_PY_LIST(res, p = srcn->binaries[archnum], p, p = p->next,
747 ("s", p->value->package)
748 );
749 return res;
750}
751
752static PyObject *dpkgsrcsn_getattr(dpkgsrcsn *self, char *name) {
753 static struct PyMethodDef dpkgsrcsn_methods[] = {
754 { "remove_source", (binaryfunc) dpkgsrcsn_removesource,
755 METH_VARARGS, NULL },
756 { "upgrade_source", (binaryfunc) dpkgsrcsn_upgradesource,
757 METH_VARARGS, NULL },
758 { "upgrade_arch", (binaryfunc) dpkgsrcsn_upgradearch,
759 METH_VARARGS, NULL },
760
761 { "undo_change", (binaryfunc) dpkgsrcsn_undochange,
762 METH_VARARGS, NULL },
763 { "commit_changes", (binaryfunc) dpkgsrcsn_commitchanges,
764 METH_VARARGS, NULL },
765
766 { "write_notes", (binaryfunc) dpkgsrcsn_writenotes,
767 METH_VARARGS, NULL },
768
769 { "Packages", (binaryfunc) dpkgsrcsn_packages,
770 METH_VARARGS, NULL },
771
772 { "get_version", (binaryfunc) dpkgsrcsn_getversion,
773 METH_VARARGS, NULL },
774 { "get_field", (binaryfunc) dpkgsrcsn_getfield,
775 METH_VARARGS, NULL },
776 { "is_present", (binaryfunc) dpkgsrcsn_ispresent,
777 METH_VARARGS, NULL },
778 { "is_fake", (binaryfunc) dpkgsrcsn_isfake,
779 METH_VARARGS, NULL },
780 { "binaries", (binaryfunc) dpkgsrcsn_binaries,
781 METH_VARARGS, NULL },
782 { NULL, NULL, 0, NULL }
783 };
784
785 if (strcmp(name, "arches") == 0) {
786 PyObject *arches;
787 int i;
788 MAKE_PY_LIST(arches, i = 0, i < self->srcsn->n_arches, i++,
789 ("s", self->srcsn->archname[i])
790 );
791 return arches;
792 } else if (strcmp(name, "sources") == 0) {
793 PyObject *sources;
794 sourcenotetbl_iter it;
795 MAKE_PY_LIST(sources,
796 it = first_sourcenotetbl(self->srcsn->sources),
797 !done_sourcenotetbl(it),
798 it = next_sourcenotetbl(it),
799 ("s", it.k)
800 );
801 return sources;
802 } else if (strcmp(name, "can_undo") == 0) {
803 if (can_undo(self->srcsn)) {
804 return Py_BuildValue("i", 1);
805 } else {
806 return Py_BuildValue("");
807 }
808 }
809
810 return Py_FindMethod(dpkgsrcsn_methods, (PyObject *)self, name);
811}
812
813static PyTypeObject SourcesNote_Type = {
814 PyObject_HEAD_INIT(&PyType_Type)
815
816 0, /* ob_size (0) */
817 "SourcesNote", /* type name */
818 sizeof(dpkgsrcsn), /* basicsize */
819 0, /* itemsize (0) */
820
821 (destructor) dpkgsrcsn_dealloc,
822 (printfunc) 0,
823 (getattrfunc) dpkgsrcsn_getattr,
824 (setattrfunc) 0,
825 (cmpfunc) 0,
826 (reprfunc) 0,
827
828 0, /* number methods */
829 0, /* sequence methods */
830 0, /* mapping methods */
831
832 (hashfunc) 0, /* dict[x] ?? */
833 (ternaryfunc) 0, /* x() */
834 (reprfunc) 0 /* str(x) */
835};
836
837/**************************************************************************
838 * britney.versioncmp() -- apt version compare function
839 ******************************************************/
840
841static PyObject *apt_versioncmp(PyObject *self, PyObject *args) {
842 char *l, *r;
843 int res;
844
845 (void)self; /* unused */
846
847 if (!PyArg_ParseTuple(args, "ss", &l, &r)) {
848 return NULL;
849 }
850
851 res = versioncmp(l,r);
852 return Py_BuildValue("i", res);
853}
854
855/**************************************************************************
856 * module initialisation
857 ***********************/
858
859static PyMethodDef britneymethods[] = {
860 { "Sources", dpkgsources_new, METH_VARARGS, NULL },
861 { "SourcesNote", dpkgsrcsn_new, METH_VARARGS, NULL },
862
863 { "versioncmp", apt_versioncmp, METH_VARARGS, NULL },
864
865 { NULL, NULL, 0, NULL }
866};
867
868void initbritney(void) {
869 Py_InitModule("britney", britneymethods);
870}
871
0872
=== added file 'update_out/checklib.c'
--- update_out/checklib.c 1970-01-01 00:00:00 +0000
+++ update_out/checklib.c 2015-06-09 16:34:27 +0000
@@ -0,0 +1,185 @@
1#include <stdlib.h>
2#include <unistd.h>
3
4#include <assert.h>
5
6#include "dpkg.h"
7
8#if 0
9static void checknewsrc(sourcetbl *srcstbl, dpkg_source *cur, void *data) {
10 dpkg_sources *oldsrc = data;
11 dpkg_source *old;
12 old = lookup_sourcetbl(oldsrc->sources, cur->package);
13 if (old == NULL) {
14 printf("New: %s (%s)\n", cur->package, cur->version );
15 } else if (strcmp(old->version, cur->version) != 0) {
16 printf("Updated: %s (%s, was %s)\n",
17 cur->package, cur->version, old->version );
18 } else {
19 dpkg_source *src2;
20 src2 = remove_sourcetbl(srcstbl, cur->package);
21 assert(cur == src2);
22 free_source(cur);
23 }
24}
25
26static void checkoldsrc(sourcetbl *oldsrctbl, dpkg_source *old, void *data) {
27 dpkg_sources *src = data;
28 dpkg_source *cur;
29 (void)oldsrctbl;
30 cur = lookup_sourcetbl(src->sources, old->package);
31 if (cur == NULL) {
32 printf("Removed: %s (was %s)\n", old->package, old->version );
33 }
34}
35
36static void checkuptodate(sourcetbl *srctbl, dpkg_source *src, void *data) {
37 int i;
38 int remove;
39 ownedpackagelist **p;
40 dpkg_sources *srcs = data;
41
42 (void)srctbl;
43
44 remove = 0;
45 for (i = 0; i < srcs->n_arches; i++) {
46 p = &src->packages[i];
47 while(*p != NULL) {
48 if (strcmp((*p)->value->source_ver, src->version) != 0) {
49 if (cmpversions((*p)->value->source_ver, GT, src->version)) {
50 printf("ALERT: old source: ");
51 } else {
52 printf("WARN: out of date: ");
53 }
54 printf("%s %s: %s binary: %s %s from %s\n",
55 src->package, src->version, srcs->archname[i],
56 (*p)->value->package, (*p)->value->version,
57 (*p)->value->source_ver);
58 delete_ownedpackagelist(p);
59 } else {
60 p = &(*p)->next;
61 }
62 }
63 if (src->packages[i] == NULL) {
64 printf("%s missing uptodate binaries for %s\n",
65 src->package, srcs->archname[i]);
66 remove = 1;
67 }
68 }
69 if (remove) {
70 dpkg_source *src2;
71 src2 = remove_sourcetbl(srcs->sources, src->package);
72 assert(src == src2);
73 free_source(src);
74 }
75}
76#endif
77
78static void upgrade(sourcetbl *srctbl, dpkg_source *src, void *data) {
79 static int i = 0;
80 dpkg_sources_note *srcsn = data;
81 (void)srctbl;
82 i++; i %= 1000;
83 if (can_undo(srcsn)) {
84 if (i % 29 == 1 || i % 31 == 1 || i % 7 == 5)
85 undo_change(srcsn);
86 if (i % 33 == 0) commit_changes(srcsn);
87 }
88 upgrade_source(data, src);
89}
90
91static void checkpkgs(packagetbl *pkgtbl, dpkg_collected_package *cpkg,
92 void *data)
93{
94 dpkg_packages *pkgs = data;
95 assert(pkgs->packages == pkgtbl);
96 printf("Trying %s (%s, %s)\n", cpkg->pkg->package, cpkg->pkg->version, pkgs->arch);
97 if (!checkinstallable2(pkgs, cpkg->pkg->package)) {
98 printf("Package: %s (%s, %s) is uninstallable\n",
99 cpkg->pkg->package, cpkg->pkg->version, pkgs->arch);
100 }
101}
102
103void print_memblock_summary(void);
104
105int main(int argc, char **argv) {
106 dpkg_sources *src = NULL, *oldsrc = NULL;
107 dpkg_sources_note *srcsn;
108 dpkg_source *srcpkg;
109 dpkg_packages *pkgs[10];
110 int n_pkgs;
111 int i,j;
112 int reps;
113
114 if (argc < 3) {
115 printf("Usage: %s <reps> <arch>...\n", argv[0]);
116 exit(EXIT_FAILURE);
117 }
118
119 reps = atoi(argv[1]);
120 if (reps < 1) {
121 printf("reps must be >= 1\n");
122 exit(EXIT_FAILURE);
123 }
124
125 src = read_directory("cur", argc - 2, argv + 2);
126 oldsrc = read_directory("old", argc - 2, argv + 2);
127 srcsn = new_sources_note(argc - 2, argv + 2);
128
129 printf("FINISHED LOADING\n"); fflush(stdout); /* sleep(5); */
130
131#if 0
132 iterate_sourcetbl(oldsrc->sources, checkoldsrc, src);
133
134 printf("FIRST\n");
135 iterate_sourcetbl(src->sources, checkuptodate, src);
136 printf("SECOND\n");
137 iterate_sourcetbl(src->sources, checkuptodate, src);
138 printf("END\n");
139
140 iterate_sourcetbl(src->sources, checknewsrc, oldsrc);
141#endif
142
143 n_pkgs = 0;
144 for (i = argc - 1; i > 1; i--) {
145 pkgs[n_pkgs++] = get_architecture(oldsrc, argv[i]);
146 }
147 for (j = 0; j < reps; j++) {
148 printf("Round %d/%d starting...\n", j + 1, reps);
149 for (i = 0; i < n_pkgs; i++) {
150 iterate_packagetbl(pkgs[i]->packages, checkpkgs, pkgs[i]);
151 }
152 printf("Round %d ended.\n", j+1);
153 }
154 iterate_sourcetbl(src->sources, upgrade, srcsn);
155 iterate_sourcetbl(oldsrc->sources, upgrade, srcsn);
156
157 for (i = 0; i < n_pkgs; i++) {
158 free_packages(pkgs[i]);
159 }
160
161 srcpkg = lookup_sourcetbl(oldsrc->sources, "omirr");
162 if (srcpkg != NULL) {
163 printf("Adding old\n");
164 upgrade_source(srcsn, srcpkg);
165 }
166 srcpkg = lookup_sourcetbl(src->sources, "omirr");
167 if (srcpkg != NULL) {
168 printf("Adding cur\n");
169 upgrade_source(srcsn, srcpkg);
170 }
171
172 printf("FINISHED PROCESSING\n"); fflush(stdout); /* sleep(5); */
173
174 write_directory("out", oldsrc);
175
176 printf("FINISHED WRITING\n"); fflush(stdout); /* sleep(5); */
177
178 free_sources_note(srcsn);
179 free_sources(src);
180 free_sources(oldsrc);
181
182 DEBUG_ONLY( print_memblock_summary(); )
183
184 return 0;
185}
0186
=== added file 'update_out/dpkg-lib.cpp'
--- update_out/dpkg-lib.cpp 1970-01-01 00:00:00 +0000
+++ update_out/dpkg-lib.cpp 2015-06-09 16:34:27 +0000
@@ -0,0 +1,34 @@
1
2#include <apt-pkg/debversion.h>
3
4extern "C" {
5
6#include "dpkg.h"
7
8int versioncmp(char *left, char *right) {
9 return debVS.CmpVersion(left, right);
10}
11
12int cmpversions(char *left, int op, char *right) {
13 int i = debVS.CmpVersion(left, right);
14
15 switch(op) {
16 case dr_LT: return i < 0;
17 case dr_LTEQ: return i <= 0;
18 case dr_EQ: return i == 0;
19 case dr_GTEQ: return i >= 0;
20 case dr_GT: return i > 0;
21 }
22 return 0;
23}
24
25}
26
27#ifdef TESTBIN
28int main(int argc, char **argv) {
29 if (argc != 3) { printf("Usage: %s <ver> <ver>\n", argv[0]); exit(1); }
30
31 printf("%d\n", versioncmp(argv[1], argv[2]));
32 return 0;
33}
34#endif
035
=== added file 'update_out/dpkg.c'
--- update_out/dpkg.c 1970-01-01 00:00:00 +0000
+++ update_out/dpkg.c 2015-06-09 16:34:27 +0000
@@ -0,0 +1,2013 @@
1#include <stdio.h>
2#include <stdlib.h>
3#include <string.h>
4#include <ctype.h>
5#include <errno.h>
6
7#include "dpkg.h"
8#include "memory.h"
9
10
11// enlarge this is britney has issues to parse packages
12#define SIZEOFHASHMAP 16
13
14#define insert_packagenamelist(x,y) insert_l_packagenamelist(x,y,__LINE__)
15
16static void free_dependency(dependency *dep);
17static void free_package(dpkg_package *pkg);
18static void free_collected_package(dpkg_collected_package *pkg);
19static dpkg_paragraph *read_paragraph( FILE *f );
20static dpkg_package *read_package( FILE *f );
21static collpackagelist **get_matching_low(collpackagelist **addto,
22 dpkg_packages *pkgs, dependency *dep, int line);
23static collpackagelist *get_matching(dpkg_packages *pkgs, deplist *depopts, int line);
24static deplist *read_dep_and(char *buf);
25static deplistlist *read_dep_andor(char *buf);
26static ownedpackagenamelist *read_packagenames(char *buf);
27static dpkg_sources *read_sources_file(char *filename, int n_arches);
28static dpkg_source *new_source(dpkg_sources *owner);
29static dpkg_source *read_source(FILE *f, dpkg_sources *owner);
30static deplist *read_deplist(char **buf, char sep, char end);
31static dependency *read_dependency(char **buf, char *end);
32static void add_virtualpackage(virtualpkgtbl *vpkgs, char *package,
33 char *version, dpkg_collected_package *cpkg);
34static void remove_virtualpackage(virtualpkgtbl *vpkgs, char *pkgname,
35 dpkg_collected_package *cpkg);
36static char *read_packagename(char **buf, char *end);
37static char *read_until_char(char **buf, char *end);
38static void add_package(dpkg_packages *pkgs, dpkg_package *pkg);
39static void remove_package(dpkg_packages *pkgs, dpkg_collected_package *pkg);
40static dpkg_source_note *copy_source_note(dpkg_source_note *srcn);
41
42#if 0
43static inline void *bm(size_t s, int n) { void *res = block_malloc(s); fprintf(stderr, "ALLOCED: %d %p %lu\n", n, res, s); return res; }
44static inline void bf(void *p, size_t s, int n) { block_free(p,s); fprintf(stderr, "FREED: %d %p %lu\n", n, p, s); }
45
46#define block_malloc(s) bm(s,__LINE__)
47#define block_free(p,s) bf(p,s,__LINE__)
48#endif
49
50#define block_malloc(s) block_malloc2(s, __LINE__)
51
52static char *priorities[] = {
53 "required",
54 "important",
55 "standard",
56 "optional",
57 "extra",
58 NULL
59};
60
61static char *dependency_title[] = {
62 "Pre-Depends", "Depends", "Recommends", "Suggests", NULL
63};
64
65static int dependency_counts[] = { 1, 1, 0, 0 };
66
67char *dependency_relation_sym[] = {"*", "<<", "<=", "=", ">=", ">>"};
68
69#define SMB_SIZE (1<<22)
70struct stringmemblock {
71 struct stringmemblock *next;
72 size_t last;
73 char mem[SMB_SIZE];
74};
75static struct stringmemblock *stringmemory = NULL;
76static int stringmemorycount = 0;
77static const unsigned long stringmemblocksizekib = (unsigned long) sizeof(struct stringmemblock) / 1024;
78
79char *my_strdup(char *foo) {
80 struct stringmemblock *which;
81 size_t len;
82
83 if (!foo) return NULL;
84
85 len = strlen(foo) + 1;
86
87 if (len > SMB_SIZE) return strdup(foo);
88
89 for (which = stringmemory; which; which = which->next) {
90 if (SMB_SIZE - which->last > len + 1) {
91 break;
92 }
93 }
94 if (!which) {
95 which = malloc(sizeof(struct stringmemblock));
96 if (!which) return NULL;
97 MDEBUG1_ONLY(fprintf(stderr,
98 "ALLOC: string memblock %d (%lu KiB, %lu KiB total)\n",
99 stringmemorycount, stringmemblocksizekib,
100 (stringmemorycount+1) * stringmemblocksizekib));
101 memset(which->mem, 0, SMB_SIZE);
102 which->last = 0;
103 which->next = stringmemory;
104 stringmemory = which;
105 stringmemorycount++;
106 }
107 strcpy(&which->mem[which->last], foo);
108 foo = &which->mem[which->last];
109 which->last += len;
110 return foo;
111}
112
113char *my_rep_strdup(char *foo) {
114 static char *repeats[1000] = {0};
115 int i;
116
117 for (i = 0; i < 1000; i++) {
118 if (repeats[i] == NULL) {
119 DEBUG_ONLY(fprintf(stderr, "REPEAT NR %d\n", i+1); )
120 return repeats[i] = my_strdup(foo);
121 }
122 if (strcmp(repeats[i], foo) == 0) return repeats[i];
123 }
124
125 return my_strdup(foo);
126}
127
128/* DIE **/
129
130static void die(char *orig_msg) {
131 char *msg = my_strdup(orig_msg);
132 if (*msg && msg[strlen(msg)-1] == ':') {
133 msg[strlen(msg)-1] = '\0';
134 perror(msg);
135 } else {
136 printf("%s\n", msg);
137 }
138 abort();
139}
140
141/*************************************************************************
142 * Dpkg Control/Packages/etc Operations
143 */
144
145static dpkg_paragraph *read_paragraph( FILE *f ) {
146 dpkg_paragraph *result;
147 static int line_size = 0;
148 static char *line = NULL;
149 char *pch;
150
151 dpkg_entry *c_entry = NULL;
152 char *c_value = NULL;
153
154 if (line == NULL) {
155 line_size = 10;
156 line = malloc(line_size);
157 if (line == NULL) die("read_paragraph alloc 0:");
158 }
159
160 result = block_malloc( sizeof(dpkg_paragraph) );
161 if (result == NULL) die("read_paragraph alloc 1:");
162
163 result->n_entries = 0;
164 result->entry = NULL;
165 result->n_allocated = 0;
166
167 while(fgets(line, line_size, f)) {
168 while (!feof(f) && *line && line[strlen(line)-1] != '\n') {
169 line = realloc(line, line_size * 2);
170 if (!line) die("read_paragraph realloc:");
171 fgets(line + strlen(line), line_size, f);
172 line_size *= 2;
173 }
174
175 if (line[0] == '\n') break;
176
177 if (isspace(line[0])) {
178 if (c_value == NULL)
179 die("read_paragraph early spaces");
180
181 if (c_entry == NULL) {
182 /* no need to bother */
183 } else {
184 /* extend the line */
185 c_value = realloc(c_value, strlen(c_value) + strlen(line) + 1);
186 if (c_value == NULL)
187 die("read_paragraph realloc c_value:");
188
189 strcat(c_value, line);
190 }
191 } else {
192 if (c_entry) {
193 c_entry->value = my_strdup(c_value);
194 c_value[0] = '\0';
195 free(c_value);
196 c_value = NULL;
197 } else if (c_value) {
198 free(c_value);
199 c_value = NULL;
200 }
201 pch = strchr(line, ':');
202 if (pch == NULL) {
203 fprintf(stderr, "the line was: \"%s\"\n", line);
204 die("read_paragraph: no colon");
205 }
206
207 *pch = '\0';
208 while(isspace(*++pch));
209
210 if (strcmp(line, "Description") == 0) {
211 c_value = strdup(pch);
212 c_entry = NULL;
213 } else {
214 assert(result->n_entries <= result->n_allocated);
215 if (result->n_entries >= result->n_allocated) {
216 result->n_allocated += 10;
217 result->entry = realloc( result->entry,
218 sizeof(dpkg_entry)
219 * result->n_allocated);
220 if (result->entry == NULL)
221 die("read_paragraph realloc entry:");
222 }
223
224 c_entry = &result->entry[result->n_entries++];
225 c_entry->name = my_rep_strdup(line);
226 c_value = strdup(pch);
227 }
228 }
229 }
230
231 if (c_entry) {
232 c_entry->value = my_strdup(c_value);
233 c_value[0] = '\0';
234 free(c_value);
235 c_value = NULL;
236 }
237
238 if (result->n_entries == 0) {
239 if (result->entry) free(result->entry);
240 block_free(result, sizeof(dpkg_paragraph));
241 return NULL;
242 } else {
243 result->entry = realloc(result->entry,
244 sizeof(result->entry[0]) * result->n_entries);
245 result->n_allocated = result->n_entries;
246 }
247
248 return result;
249}
250
251static void write_paragraph(FILE *f, dpkg_paragraph *p) {
252 int i;
253
254 for (i = 0; i < p->n_entries; i++) {
255 fprintf(f, "%s: %s", p->entry[i].name, p->entry[i].value);
256 }
257 fprintf(f, "\n");
258}
259
260
261static void free_paragraph(dpkg_paragraph *p) {
262 int i;
263
264 if (p == NULL) return;
265
266 for (i = 0; i < p->n_entries; i++) {
267 /* block_free(p->entry[i].name); */
268 /* block_free(p->entry[i].value); */
269 }
270 free(p->entry);
271 block_free(p, sizeof(dpkg_paragraph));
272}
273
274/*************************************************************************
275 * Basic Package Operations
276 */
277
278static dpkg_package *new_package(void) {
279 dpkg_package *result;
280
281 result = block_malloc(sizeof(dpkg_package));
282 if (result == NULL) die("new_package alloc:");
283
284 result->package = NULL;
285 result->version = NULL;
286
287 result->priority = 0;
288 result->arch_all = 0;
289
290 result->source = NULL;
291 result->source_ver = NULL;
292
293 result->depends[0] = NULL;
294 result->depends[1] = NULL;
295 result->depends[2] = NULL;
296 result->depends[3] = NULL;
297
298 result->conflicts = NULL;
299
300 result->provides = NULL;
301 result->details = NULL;
302
303 return result;
304}
305
306static dpkg_collected_package *new_collected_package(dpkg_package *pkg) {
307 dpkg_collected_package *result;
308
309 result = block_malloc(sizeof(dpkg_collected_package));
310 if (result == NULL) die("new_collected_package alloc:");
311
312 result->pkg = pkg;
313
314 result->installed = 0;
315 result->conflicted = 0;
316
317 result->installable = UNKNOWN;
318 result->mayaffect = NULL;
319
320 return result;
321}
322
323static void free_collected_package(dpkg_collected_package *cpkg) {
324 if (cpkg == NULL) return;
325 cpkg->pkg = NULL;
326 free_packagenamelist(cpkg->mayaffect);
327 cpkg->mayaffect = NULL;
328 block_free(cpkg, sizeof(dpkg_collected_package));
329}
330
331static void free_package(dpkg_package *pkg) {
332 int i;
333 if (pkg == NULL) return;
334
335 /* block_free(pkg->package);
336 * block_free(pkg->version);
337 * block_free(pkg->source);
338 * block_free(pkg->source_ver); */
339
340 for (i = 0; i < 4; i++)
341 free_deplistlist(pkg->depends[i]);
342
343 free_deplist(pkg->conflicts);
344 free_ownedpackagenamelist(pkg->provides);
345
346 free_paragraph(pkg->details);
347
348 block_free(pkg, sizeof(dpkg_package));
349}
350
351static dpkg_package *read_package( FILE *f ) {
352 dpkg_package *result;
353 dpkg_paragraph *para;
354 dpkg_entry *e;
355 int i;
356
357 para = read_paragraph(f);
358 if (para == NULL) return NULL;
359
360 result = new_package();
361 result->details = para;
362
363 for (e = &para->entry[0]; e < &para->entry[para->n_entries]; e++) {
364 if (strcasecmp("Package", e->name) == 0) {
365 result->package = my_strdup(e->value);
366 if (result->package == NULL)
367 die("read_package my_strdup:");
368 result->package[strlen(result->package)-1] = '\0';
369 }
370
371 if (strcasecmp("Version", e->name) == 0) {
372 result->version = my_strdup(e->value);
373 if (result->version == NULL)
374 die("read_package my_strdup:");
375 result->version[strlen(result->version)-1] = '\0';
376 }
377
378 if (strcasecmp("Priority", e->name) == 0) {
379 int i;
380 for (i = 0; priorities[i] != NULL; i++) {
381 if (strcasecmp(priorities[i], e->value))
382 break;
383 }
384 result->priority = i;
385 if (priorities[i] == NULL) {
386 die("read_package: unknown priority");
387 }
388 }
389
390 if (strcasecmp("Architecture", e->name) == 0) {
391 if (strncasecmp(e->value, "all", 3) == 0) {
392 if (!e->value[3] || isspace(e->value[3])) {
393 result->arch_all = 1;
394 }
395 }
396 }
397
398 for (i = 0; dependency_title[i] != NULL; i++) {
399 if (strcasecmp(dependency_title[i], e->name) == 0) {
400 result->depends[i] = read_dep_andor(e->value);
401 }
402 }
403
404 /*
405 * Support Breaks by treating them as Conflicts.
406 */
407 if (strcasecmp("Conflicts", e->name) == 0
408 || strcasecmp("Breaks", e->name) == 0) {
409 deplist *pkgs = read_dep_and(e->value);
410 if (result->conflicts == NULL)
411 result->conflicts = pkgs;
412 else {
413 // TODO(dato): create extend_deplist() in templates.h?
414 deplist *end = result->conflicts;
415 while (end->next)
416 end = end->next;
417 end->next = pkgs;
418 }
419 }
420
421 if (strcasecmp("Provides", e->name) == 0)
422 result->provides = read_packagenames(e->value);
423
424 if (strcasecmp("source", e->name) == 0) {
425 char *pch = e->value;
426
427 assert(result->source == NULL);
428 assert(result->source_ver == NULL);
429
430 result->source = my_strdup(read_packagename(&pch, "("));
431 if (result->source == NULL)
432 die("read_package: bad source header");
433
434 while(isspace(*pch)) pch++;
435 if (*pch == '(') {
436 pch++;
437 result->source_ver = my_strdup(read_until_char(&pch, ")"));
438 if (result->source_ver == NULL)
439 die("read_package: bad source version");
440 while(isspace(*pch)) pch++;
441 if (*pch != ')')
442 die("read_package: unterminated ver");
443 }
444 }
445 }
446
447 if (result->source == NULL) {
448 assert(result->source_ver == NULL);
449 result->source = my_strdup(result->package);
450 }
451 if (result->source_ver == NULL) {
452 result->source_ver = my_strdup(result->version);
453 }
454
455 return result;
456}
457
458static void freesize(void *p, size_t s) { (void)s; free(p); }
459
460LIST_IMPL(deplist, dependency*, free_dependency, block_malloc, block_free);
461LIST_IMPL(deplistlist, deplist*, free_deplist, block_malloc, block_free);
462
463LIST_IMPLX(packagenamelist, char*, KEEP(char*));
464
465LIST_IMPL(ownedpackagenamelist, char*, KEEP(char*), block_malloc, block_free);
466 /* ownedpackagenamelist stores the packagename in the string store */
467
468static int packagecmp(dpkg_package *l, dpkg_package *r) {
469 if (l->priority < r->priority) return -1;
470 if (l->priority > r->priority) return +1;
471 return strcmp(l->package, r->package);
472}
473
474/* container for owned pkgs */
475LIST_IMPL(ownedpackagelist, dpkg_package *, free_package,
476 block_malloc, block_free);
477
478/* container for existing pkgs */
479LIST_IMPL(packagelist, dpkg_package *, KEEP(dpkg_package *), block_malloc, block_free);
480
481LIST_IMPLX(collpackagelist, dpkg_collected_package *,
482 KEEP(dpkg_collected_package *))
483#define insert_collpackagelist(x,y) insert_l_collpackagelist(x,y,__LINE__)
484
485/*************************************************************************
486 * Operations on distributions (collections of packages)
487 */
488
489dpkg_packages *new_packages(char *arch) {
490 dpkg_packages *result;
491
492 result = block_malloc(sizeof(dpkg_packages));
493 if (result == NULL) die("new_packages alloc:");
494
495 result->arch = my_strdup(arch);
496 result->packages = new_packagetbl();
497 result->virtualpkgs = new_virtualpkgtbl();
498
499 return result;
500}
501
502static void add_package(dpkg_packages *pkgs, dpkg_package *pkg)
503{
504 ownedpackagenamelist *v;
505 dpkg_collected_package *cpkg;
506
507 if (lookup_packagetbl(pkgs->packages, pkg->package) != NULL)
508 return;
509
510 cpkg = new_collected_package(pkg);
511
512 add_packagetbl(pkgs->packages, cpkg->pkg->package, cpkg);
513
514 add_virtualpackage(pkgs->virtualpkgs, cpkg->pkg->package,
515 cpkg->pkg->version, cpkg);
516 for (v = cpkg->pkg->provides; v != NULL; v = v->next) {
517 add_virtualpackage(pkgs->virtualpkgs, v->value, NULL, cpkg);
518 }
519}
520
521static void remove_package(dpkg_packages *pkgs, dpkg_collected_package *cpkg) {
522 ownedpackagenamelist *v;
523 packagenamelist *aff;
524 dpkg_collected_package *p;
525
526 for (aff = cpkg->mayaffect; aff != NULL; aff = aff->next) {
527 p = lookup_packagetbl(pkgs->packages, aff->value);
528 if (p == NULL) continue;
529 p->installable = UNKNOWN;
530 }
531
532 p = remove_packagetbl(pkgs->packages, cpkg->pkg->package);
533 if (p != cpkg) return;
534
535 remove_virtualpackage(pkgs->virtualpkgs, cpkg->pkg->package, cpkg);
536 for (v = cpkg->pkg->provides; v != NULL; v = v->next) {
537 remove_virtualpackage(pkgs->virtualpkgs, v->value, cpkg);
538 }
539
540 free_collected_package(cpkg);
541}
542
543dpkg_packages *get_architecture(dpkg_sources *srcs, char *arch) {
544 int i, arch_index;
545 dpkg_packages *result;
546 sourcetbl_iter srci;
547 ownedpackagelist *p;
548
549 arch_index = -1;
550 for (i = 0; i < srcs->n_arches; i++) {
551 if (strcmp(srcs->archname[i], arch) == 0) {
552 arch_index = i;
553 break;
554 }
555 }
556 if (arch_index == -1) die("get_architecture: unknown arch");
557
558 result = new_packages(arch);
559
560 for (srci = first_sourcetbl(srcs->sources);
561 !done_sourcetbl(srci);
562 srci = next_sourcetbl(srci))
563 {
564 for (p = srci.v->packages[arch_index]; p != NULL; p = p->next) {
565 add_package(result, p->value);
566 }
567 }
568
569 return result;
570}
571
572void free_packages(dpkg_packages *pkgs) {
573 if (pkgs == NULL) return;
574 /* block_free(pkgs->arch); */
575 free_packagetbl(pkgs->packages);
576 free_virtualpkgtbl(pkgs->virtualpkgs);
577 block_free(pkgs, sizeof(dpkg_packages));
578}
579
580
581HASH_IMPL(packagetbl, char *, dpkg_collected_package *, SIZEOFHASHMAP, strhash, strcmp,
582 KEEP(char*),free_collected_package);
583HASH_IMPL(virtualpkgtbl, char *, virtualpkg *, SIZEOFHASHMAP, strhash, strcmp,
584 KEEP(char*), free_virtualpkg);
585
586/* dpkg_provision refers to memory allocated elsewhere */
587LIST_IMPL(virtualpkg, dpkg_provision, KEEP(dpkg_provision), block_malloc, block_free);
588
589static void remove_virtualpackage(virtualpkgtbl *vpkgs, char *pkgname,
590 dpkg_collected_package *cpkg)
591{
592 virtualpkg *list;
593 virtualpkg **where;
594 list = lookup_virtualpkgtbl(vpkgs, pkgname);
595 assert(list != NULL);
596
597 where = &list;
598 while((*where)->value.pkg != cpkg) {
599 where = &(*where)->next;
600 assert(*where != NULL);
601 }
602
603 delete_virtualpkg(where);
604
605 if (list == NULL) {
606 remove_virtualpkgtbl(vpkgs, pkgname);
607 } else {
608 replace_virtualpkgtbl(vpkgs, pkgname, list);
609 }
610}
611
612static void add_virtualpackage(virtualpkgtbl *vpkgs, char *package,
613 char *version, dpkg_collected_package *cpkg)
614{
615 dpkg_provision value;
616 virtualpkg *list, **addto;
617 int shouldreplace;
618
619 value.pkg = cpkg;
620 value.version = version;
621
622 list = lookup_virtualpkgtbl(vpkgs, package);
623 shouldreplace = (list != NULL);
624
625 addto = &list;
626 while (*addto != NULL
627 && packagecmp(cpkg->pkg, (*addto)->value.pkg->pkg) >= 0)
628 {
629 addto = &(*addto)->next;
630 }
631 insert_virtualpkg(addto, value);
632
633 if (shouldreplace) {
634 replace_virtualpkgtbl(vpkgs, package, list);
635 /* old list is included in new list, so we don't need to free */
636 } else {
637 add_virtualpkgtbl(vpkgs, package, list);
638 }
639}
640
641/*************************************************************************
642 * Parsing Helper Functions
643 */
644
645static ownedpackagenamelist *read_packagenames(char *buf) {
646 ownedpackagenamelist *result = NULL;
647 ownedpackagenamelist **addto = &result;
648
649 DEBUG_ONLY( char *strend = buf + strlen(buf); )
650
651 char *sub;
652
653 while ((sub = my_strdup(read_packagename(&buf, ",")))) {
654 insert_ownedpackagenamelist(addto, sub);
655 addto = &(*addto)->next;
656
657 while(isspace(*buf)) buf++;
658 if (*buf == ',') {
659 buf++;
660 continue;
661 }
662 if (*buf == '\0') {
663 break;
664 }
665
666 die("read_packagenames no/bad seperator");
667 }
668
669 DEBUG_ONLY( assert(buf <= strend); )
670
671 return result;
672}
673
674static char *read_until_char(char **buf, char *end) {
675 static char *result = NULL;
676 char *start;
677 DEBUG_ONLY( char *strend = *buf + strlen(*buf); )
678 int n;
679
680 while(isspace(**buf)) (*buf)++;
681
682 start = *buf;
683 while (**buf && !isspace(**buf) && strchr(end, **buf) == NULL) {
684 (*buf)++;
685 }
686
687 n = *buf - start;
688 if (n == 0) return NULL;
689
690 result = realloc(result, n + 1);
691 if (result == NULL) die("read_until_char alloc:");
692
693 strncpy(result, start, n);
694 result[n] = '\0';
695
696 while(isspace(**buf)) (*buf)++;
697
698 DEBUG_ONLY( assert(*buf <= strend); )
699
700 return result;
701}
702
703static char *read_packagename(char **buf, char *end) {
704 return read_until_char(buf, end);
705}
706
707static deplist *read_dep_and(char *buf) {
708 return read_deplist(&buf, ',', '\0');
709}
710
711static deplist *read_deplist(char **buf, char sep, char end) {
712 deplist *result = NULL;
713 deplist **addto = &result;
714
715 char separs[3] = { sep, end, '\0' };
716
717 DEBUG_ONLY( char *strend = *buf + strlen(*buf); )
718
719 dependency *sub;
720
721 while ((sub = read_dependency(buf, separs))) {
722 insert_deplist(addto, sub);
723 addto = &(*addto)->next;
724
725 while(isspace(**buf)) (*buf)++;
726 if (**buf == sep) {
727 (*buf)++;
728 continue;
729 }
730 if (**buf == '\0' || **buf == end) {
731 break;
732 }
733
734 die("read_deplist no/bad seperator");
735 }
736
737 DEBUG_ONLY( assert(*buf <= strend); )
738
739 return result;
740}
741
742static deplistlist *read_dep_andor(char *buf) {
743 deplistlist *result = NULL;
744 deplistlist **addto = &result;
745
746 deplist *sub;
747
748 DEBUG_ONLY( char *strend = buf + strlen(buf); )
749
750 while ((sub = read_deplist(&buf, '|', ','))) {
751 insert_deplistlist(addto, sub);
752 addto = &(*addto)->next;
753
754 if (*buf == ',') buf++;
755 }
756
757 DEBUG_ONLY( assert(buf <= strend); )
758
759 return result;
760}
761
762static dependency *read_dependency(char **buf, char *end) {
763 dependency *dep;
764 char *name;
765 char newend[10];
766 DEBUG_ONLY( char *strend = *buf + strlen(*buf); )
767
768 assert(strlen(end) <= 8);
769 newend[0] = '('; strcpy(newend + 1, end);
770
771 name = my_strdup(read_until_char(buf, newend));
772 if (name == NULL) return NULL;
773
774 dep = block_malloc(sizeof(dependency));
775 if (dep == NULL) die("read_dependency alloc 1:");
776
777 dep->package = name;
778
779 while(isspace(**buf)) (*buf)++;
780
781 if (**buf != '(') {
782 dep->op = dr_NOOP;
783 dep->version = NULL;
784 } else {
785 (*buf)++;
786 while(isspace(**buf)) (*buf)++;
787 /* << , <= , = , >= , >> */
788 if (**buf == '<') {
789 (*buf)++;
790 if (**buf == '<') {
791 dep->op = dr_LT;
792 (*buf)++;
793 } else if (**buf == '=') {
794 dep->op = dr_LTEQ;
795 (*buf)++;
796 } else {
797 /* The forms `<' and `>' were used to mean earlier/later or
798 * equal, rather than strictly earlier/later, so they should
799 * not appear in new packages (though `dpkg' still supports
800 * them).
801 */
802 dep->op = dr_LTEQ;
803 }
804 } else if (**buf == '>') {
805 (*buf)++;
806 if (**buf == '>') {
807 dep->op = dr_GT;
808 (*buf)++;
809 } else if (**buf == '=') {
810 dep->op = dr_GTEQ;
811 (*buf)++;
812 } else {
813 dep->op = dr_GTEQ;
814 }
815 } else if (**buf == '=') {
816 dep->op = dr_EQ;
817 (*buf)++;
818 if (**buf == '>') {
819 dep->op = dr_GTEQ;
820 (*buf)++;
821 } else if (**buf == '<') {
822 dep->op = dr_LTEQ;
823 (*buf)++;
824 }
825 } else {
826 /* treat it as an implicit = :( */
827 dep->op = dr_EQ;
828 /* would prefer to: die("read_dependency unknown version op"); */
829 }
830
831 while (isspace(**buf)) (*buf)++;
832 newend[0] = ')';
833 dep->version = my_strdup(read_until_char(buf, newend));
834 while (isspace(**buf)) (*buf)++;
835
836 if (dep->version == NULL) die("read_dependency: no version");
837 if (**buf != ')') die("read_dependency: unterminated version");
838 (*buf)++;
839 }
840
841 DEBUG_ONLY( assert(*buf <= strend); )
842
843 return dep;
844}
845
846static void free_dependency(dependency *dep) {
847 if (dep == NULL) return;
848 /* block_free(dep->package); */
849 /* if (dep->version) block_free(dep->version); */
850 block_free(dep, sizeof(dependency));
851}
852
853/*************************************************************************
854 * Installability Checking
855 */
856
857static collpackagelist **get_matching_low(collpackagelist **addto,
858 dpkg_packages *pkgs, dependency *dep, int line)
859{
860 virtualpkg *vpkg;
861 for (vpkg = lookup_virtualpkgtbl(pkgs->virtualpkgs, dep->package);
862 vpkg != NULL;
863 vpkg = vpkg->next)
864 {
865 int add;
866
867 add = 0;
868 if (dep->op == dr_NOOP) {
869 add = 1;
870 } else if (vpkg->value.version != NULL) {
871 if (cmpversions(vpkg->value.version, dep->op, dep->version)) {
872 add = 1;
873 }
874 }
875
876 if (add) {
877 insert_l_collpackagelist(addto, vpkg->value.pkg, line);
878 addto = &(*addto)->next;
879 }
880 }
881
882 return addto;
883}
884
885static collpackagelist *get_matching(dpkg_packages *pkgs, deplist *depopts, int line) {
886 collpackagelist *list = NULL;
887 collpackagelist **addto = &list;
888
889 for(; depopts != NULL; depopts = depopts->next) {
890 addto = get_matching_low(addto, pkgs, depopts->value, line);
891 }
892
893 return list;
894}
895
896typedef struct instonelist instonelist;
897struct instonelist {
898 collpackagelist *curX;
899 collpackagelist *instoneX;
900 int expandedX;
901 struct instonelist *nextX, *prevX, *cutoffX;
902};
903
904#define I1CUR(i1) ((i1)->curX)
905#define I1INSTONE(i1) ((i1)->instoneX)
906#define I1CUTOFF(i1) ((i1)->cutoffX)
907#define I1NEXT(i1) ((i1)->nextX) /* can be modified ! */
908#define I1PREV(i1) ((i1)->prevX)
909#define I1EXPANDED(i1) ((i1)->expandedX)
910
911static instonelist *insert_instonelist(instonelist *where, collpackagelist *instone);
912static void trim_instonelist_after(instonelist *first);
913static void free_instonelist(instonelist *l);
914
915static int max_instone_len = 0;
916static int cur_instone_len = 0;
917
918static instonelist *insert_instonelist(instonelist *old, collpackagelist *instone)
919{
920 instonelist *n = block_malloc(sizeof(instonelist));
921 if (n == NULL)
922 die("insert_instonelist alloc:");
923
924 n->curX = NULL;
925 n->instoneX = instone;
926 n->cutoffX = NULL;
927 n->nextX = (old ? old->nextX : NULL);
928 n->prevX = old;
929 n->expandedX = 0;
930
931 if (old) old->nextX = n;
932 if (n->nextX) n->nextX->prevX = n;
933
934 cur_instone_len++;
935 if (cur_instone_len > max_instone_len) max_instone_len = cur_instone_len;
936
937 return n;
938}
939
940static void trim_instonelist_after(instonelist *first) {
941 if (!first->nextX) return;
942 first->nextX->prevX = NULL;
943 free_instonelist(first->nextX);
944 first->nextX = NULL;
945}
946
947static void free_instonelist(instonelist *l) {
948 instonelist *p, *k;
949 if (!l) return;
950 for (p = l; p->nextX; p = p->nextX);
951 do {
952 k = p;
953 p = k->prevX;
954 free_collpackagelist(k->instoneX);
955 block_free(k, sizeof(instonelist));
956 cur_instone_len--;
957 } while (k != l);
958}
959
960static int caninstall(dpkg_packages *pkgs, dpkg_collected_package *cpkg) {
961 collpackagelist *conflicts;
962 collpackagelist *conf;
963 int okay;
964
965 if (cpkg->installed > 0) return 1;
966 if (cpkg->conflicted > 0) return 0;
967
968 conflicts = get_matching(pkgs, cpkg->pkg->conflicts, __LINE__);
969
970 okay = 1;
971 for (conf = conflicts; conf != NULL; conf = conf->next) {
972 if (conf->value->installed > 0) {
973 okay = 0;
974 break;
975 }
976 }
977 free_collpackagelist(conflicts);
978 return okay;
979}
980
981static int cur_install =0;
982static int max_install = 0;
983static int cur_conflicted = 0;
984static int max_conflicted = 0;
985
986static void install(dpkg_packages *pkgs, dpkg_collected_package *cpkg) {
987 if (cpkg->installed == 0) {
988 collpackagelist *conflicts = get_matching(pkgs, cpkg->pkg->conflicts, __LINE__);
989 collpackagelist *conf;
990 for (conf = conflicts; conf != NULL; conf = conf->next) {
991 if (conf->value == cpkg) continue;
992 assert(conf->value->installed == 0);
993 conf->value->conflicted++;
994 if (conf->value->conflicted == 1) {
995 cur_conflicted++;
996 if (max_conflicted < cur_conflicted)
997 max_conflicted = cur_conflicted;
998 }
999 }
1000 free_collpackagelist(conflicts);
1001 cur_install++;
1002 if (max_install < cur_install) max_install = cur_install;
1003 }
1004 assert(cpkg->conflicted == 0);
1005 cpkg->installed++;
1006}
1007
1008static void uninstall(dpkg_packages *pkgs, dpkg_collected_package *cpkg) {
1009 assert(cpkg->installed > 0);
1010 assert(cpkg->conflicted == 0);
1011 cpkg->installed--;
1012 if (cpkg->installed == 0) {
1013 collpackagelist *conflicts = get_matching(pkgs, cpkg->pkg->conflicts, __LINE__);
1014 collpackagelist *conf;
1015 for (conf = conflicts; conf != NULL; conf = conf->next) {
1016 if (conf->value == cpkg) continue;
1017 assert(conf->value->installed == 0);
1018 assert(conf->value->conflicted > 0);
1019 conf->value->conflicted--;
1020 if (conf->value->conflicted == 0) cur_conflicted--;
1021 }
1022 free_collpackagelist(conflicts);
1023 cur_install--;
1024 }
1025}
1026
1027satisfieddep *new_satisfieddep(void) {
1028 satisfieddep *sd = block_malloc(sizeof(satisfieddep));
1029 if (!sd) die("new_satisfieddep alloc:");
1030 return sd;
1031}
1032
1033void free_satisfieddep(satisfieddep *sd) {
1034 if (!sd) return;
1035 free_packagelist(sd->pkgs);
1036 block_free(sd, sizeof(satisfieddep));
1037}
1038
1039LIST_IMPL(satisfieddeplist, satisfieddep *, free_satisfieddep, block_malloc, block_free);
1040
1041packagelist *collpkglist2pkglist(collpackagelist *l) {
1042 packagelist *r = NULL;
1043 packagelist **addto = &r;
1044
1045 for (; l != NULL; l = l->next) {
1046 insert_packagelist(addto, l->value->pkg);
1047 addto = &(*addto)->next;
1048 }
1049
1050 return r;
1051}
1052
1053satisfieddeplist *checkunsatisfiabledeps(dpkg_packages *pkgs,
1054 deplistlist *deps) {
1055 satisfieddeplist *unsatisfiable = NULL;
1056 satisfieddeplist **addto = &unsatisfiable;
1057 satisfieddep *sd;
1058 collpackagelist *deppkgs;
1059
1060 for (; deps != NULL; deps = deps->next) {
1061 /* deplist *dep; */
1062 /* for (dep = deps->value; dep != NULL; dep = dep->next) { */
1063 sd = new_satisfieddep();
1064 /* sd->dep = dep->value; */
1065 sd->depl = deps->value;
1066
1067 deppkgs = NULL;
1068 /* get_matching_low(&deppkgs, pkgs, dep->value); */
1069 deppkgs = get_matching(pkgs, deps->value, __LINE__);
1070 sd->pkgs = collpkglist2pkglist(deppkgs);
1071 free_collpackagelist(deppkgs);
1072
1073 insert_satisfieddeplist(addto, sd);
1074 addto = &(*addto)->next;
1075 /* } */
1076 }
1077
1078 return unsatisfiable;
1079}
1080
1081int checkinstallable2(dpkg_packages *pkgs, char *pkgname) {
1082 dpkg_collected_package *cpkg = lookup_packagetbl(pkgs->packages, pkgname);
1083 collpackagelist *cpl = NULL;
1084
1085 if (cpkg == NULL) return 0;
1086
1087 insert_collpackagelist(&cpl, cpkg);
1088 /* cpl gets freed in checkinstallable :-/ */
1089 return checkinstallable(pkgs, cpl);
1090}
1091
1092static void debug_checkinstallable(FILE *out, instonelist *list,
1093 instonelist *last, instonelist *pointer)
1094{
1095 instonelist *l;
1096 fprintf(out, "Status:");
1097
1098 /* codes: | = multiple options here
1099 * @ = no options can satisfy this dep
1100 * + = dependencies that can be expanded have been
1101 * * = nothing selected yet
1102 * > = where pointer points
1103 * ^ = the cut point for where we are
1104 */
1105
1106 for (l = list; ; l = I1NEXT(l)) {
1107 fprintf(out, " ");
1108 if (l == pointer) fprintf(out, ">");
1109 if (l == I1CUTOFF(pointer)) fprintf(out, "^");
1110 if (I1INSTONE(l) == NULL) {
1111 fprintf(out, "@");
1112 } else {
1113 if (I1INSTONE(l)->next != NULL) {
1114 fprintf(out, "|");
1115 }
1116 if (I1EXPANDED(l)) {
1117 fprintf(out, "+");
1118 }
1119 if (I1CUR(l) == NULL) {
1120 fprintf(out, "*%s", I1INSTONE(l)->value->pkg->package);
1121 } else {
1122 fprintf(out, "%s", I1CUR(l)->value->pkg->package);
1123 }
1124 }
1125 if (l == last) break;
1126 }
1127 fprintf(out, " ###\n");
1128 fflush(out);
1129}
1130
1131int checkinstallable(dpkg_packages *pkgs, collpackagelist *instoneof) {
1132 /* We use pkg->installed, pkg->conflicted to note how many
1133 * times we've used this pkg to satisfy a dependency or installed
1134 * a package that conflicts with it.
1135 * Thus: pkg->installed == 0, or pkg->conflicted == 0
1136 *
1137 * We assume these are okay initially, aren't being played with
1138 * concurrently elsewhere, and make sure they're still okay when
1139 * we return.
1140 */
1141
1142 instonelist *list;
1143 instonelist *last;
1144
1145 instonelist *pointer;
1146
1147 unsigned long counter = 500000000;
1148
1149 int last_max_inst = max_install;
1150 int last_max_instone = max_instone_len;
1151 int last_max_conflict = max_conflicted;
1152
1153 {
1154 collpackagelist *cpkg;
1155 for (cpkg = instoneof; cpkg; cpkg = cpkg->next) {
1156 if (cpkg->value->installable == YES) {
1157 free_collpackagelist(instoneof);
1158 return 1;
1159 }
1160 }
1161 }
1162
1163 list = insert_instonelist(NULL, instoneof);
1164
1165 last = list;
1166 pointer = list;
1167
1168 while(--counter > 0 && pointer) {
1169 deplistlist *dep;
1170 dpkg_collected_package *instpkg; /* convenient alias */
1171 int i;
1172
1173#ifndef NDEBUG
1174 {
1175 instonelist *p;
1176 for (p = list; p != pointer; p = I1NEXT(p)) {
1177 assert(p != NULL);
1178 assert(I1CUR(p) != NULL);
1179 assert(I1CUR(p)->value != NULL);
1180 assert(I1CUR(p)->value->installed > 0);
1181 assert(I1CUR(p)->value->conflicted == 0);
1182 }
1183 if (I1NEXT(pointer) == NULL) {
1184 assert(pointer == last);
1185 } else {
1186 for (p = I1NEXT(pointer); p; p = I1NEXT(p)) {
1187 if (I1NEXT(p) == NULL) {
1188 assert(p == last);
1189 }
1190 assert(I1CUR(p) == NULL);
1191 }
1192 }
1193 }
1194#endif
1195
1196#ifdef DIAGNOSE
1197 debug_checkinstallable(stdout, list, last, pointer);
1198#endif
1199
1200 if (I1CUR(pointer) == NULL) {
1201 I1CUR(pointer) = I1INSTONE(pointer);
1202 /* try to choose an already installed package if there is one */
1203 while (I1CUR(pointer) != NULL) {
1204 if (I1CUR(pointer)->value->installed != 0) {
1205 break;
1206 }
1207 I1CUR(pointer) = I1CUR(pointer)->next;
1208 }
1209 if (I1CUR(pointer) == NULL) {
1210 I1CUR(pointer) = I1INSTONE(pointer);
1211 }
1212 assert(I1CUR(pointer) || !I1INSTONE(pointer));
1213
1214 I1CUTOFF(pointer) = last;
1215 } else {
1216 uninstall(pkgs, I1CUR(pointer)->value);
1217 trim_instonelist_after(I1CUTOFF(pointer));
1218 last = I1CUTOFF(pointer);
1219
1220 if (I1CUR(pointer)->value->installed > 0) {
1221 /* this dependency isn't the issue -- even doing
1222 * nothing to satisfy it (ie, using an already
1223 * installed package) doesn't do any good. So give up.
1224 */
1225 I1CUR(pointer) = NULL;
1226 } else {
1227 I1CUR(pointer) = I1CUR(pointer)->next;
1228 }
1229 }
1230
1231 while(I1CUR(pointer) && !caninstall(pkgs, I1CUR(pointer)->value)) {
1232 I1CUR(pointer) = I1CUR(pointer)->next;
1233 }
1234
1235 if (I1CUR(pointer) == NULL) {
1236 if (I1PREV(pointer) == NULL) break;
1237 pointer = I1PREV(pointer);
1238 continue;
1239 }
1240
1241 instpkg = I1CUR(pointer)->value;
1242
1243 install(pkgs, instpkg);
1244
1245 assert(instpkg->installed > 0);
1246 if (instpkg->installed == 1) {
1247 /* if it's been installed exactly once, then this must've been
1248 * the first time it was touched, so we need to look at the
1249 * dependencies. If it's the second or later, then we don't care
1250 * about them.
1251 */
1252
1253 /* if any of the deps can't be satisfied, don't move on */
1254 int bother = 1;
1255
1256 int expanded = I1EXPANDED(pointer);
1257
1258 for (i = 0; i < 4; i++) {
1259 if (!dependency_counts[i]) continue;
1260 for (dep = instpkg->pkg->depends[i];
1261 dep != NULL; dep = dep->next)
1262 {
1263 collpackagelist *thisdep = get_matching(pkgs, dep->value, __LINE__);
1264
1265 if (thisdep == NULL) {
1266 bother = 0;
1267
1268 } else if (thisdep != NULL && thisdep->next == NULL) {
1269 collpackagelist *x;
1270
1271 /* if there's only one way of fulfilling this dep,
1272 * do it "ASAP"
1273 */
1274
1275 /* optimisation: if thisdep == foo, but the parent
1276 * was foo|bar, then we already know "foo" is not going
1277 * to work in this combination, and we can skip it.
1278 *
1279 * This deals with cases like X deps: Y|bar, bar deps: Y
1280 * where bar is a virtual package; cf xlibs
1281 */
1282 for (x = I1INSTONE(pointer); x != I1CUR(pointer); x = x->next) {
1283 if (x->value == thisdep->value) {
1284 bother = 0;
1285 break;
1286 }
1287 }
1288
1289 if (I1INSTONE(pointer)->next == NULL) {
1290 /* the parent of this entry essentially depends
1291 * on this too, so we'll get it out of the way
1292 * ASAP, to reduce the degree of exponentiation
1293 * in bad cases.
1294 *
1295 * _However_ we only want to do this _once_ for
1296 * any particular node.
1297 */
1298 if (expanded) {
1299 /* thisdep isn't used! */
1300 free_collpackagelist(thisdep);
1301 } else {
1302 insert_instonelist(pointer, thisdep);
1303 I1EXPANDED(pointer) = 1;
1304 }
1305 } else {
1306 insert_instonelist(I1CUTOFF(pointer), thisdep);
1307 }
1308 if (I1NEXT(last)) last = I1NEXT(last);
1309 assert(!I1NEXT(last));
1310
1311 } else {
1312 /* otherwise it's a multi possibility dep, so do it
1313 * at the end
1314 */
1315
1316 last = insert_instonelist(last, thisdep);
1317 }
1318 }
1319 }
1320 if (!bother) {
1321 /* stay where we are, and try the next possibility */
1322 continue;
1323 }
1324 }
1325
1326 pointer = I1NEXT(pointer);
1327 }
1328
1329 if (max_install>last_max_inst || max_instone_len>last_max_instone || max_conflicted >last_max_conflict) {
1330 printf("MAX: i:%d / c:%d / i1:%d %s\n", max_install,max_conflicted,max_instone_len,instoneof->value->pkg->package);
1331 }
1332
1333 if (counter == 0) {
1334 fprintf(stderr, "AIEEE: counter overflow:");
1335 assert(pointer != NULL);
1336 if (I1CUR(pointer) == NULL || I1CUR(pointer)->value == NULL) {
1337 /* we're not guaranteed that pointer will make sense here */
1338 pointer = I1PREV(pointer);
1339 }
1340 for (; pointer != NULL; pointer = I1PREV(pointer)) {
1341 if (I1CUR(pointer) == NULL) {
1342 /* should only happen at pointer, so not here */
1343 fprintf(stderr, " >> eep, no packages at pointer <<");
1344 continue;
1345 }
1346 if (I1CUR(pointer)->value == NULL) {
1347 /* should never happen */
1348 fprintf(stderr, " >> eep, no package selected <<");
1349 continue;
1350 }
1351 fprintf(stderr, " %s%s",
1352 (I1INSTONE(pointer)->next == NULL ? "" : "|"),
1353 I1CUR(pointer)->value->pkg->package);
1354 uninstall(pkgs, I1CUR(pointer)->value);
1355 }
1356 fprintf(stderr, ".\n");
1357 free_instonelist(list);
1358 return 0;
1359 }
1360
1361 if (pointer == NULL) {
1362 dpkg_collected_package *cpkg = I1CUR(list)->value;
1363 assert(cpkg->installable != YES);
1364 cpkg->installable = YES;
1365 for (pointer = last; pointer != NULL; pointer = I1PREV(pointer)) {
1366 if (I1CUR(pointer)->value->installed == 1) {
1367 packagenamelist **p = &I1CUR(pointer)->value->mayaffect;
1368#if 0
1369 while ( *p && (*p)->value < cpkg->pkg->package ) {
1370 p = &(*p)->next;
1371 }
1372 if (*p == NULL || (*p)->value > cpkg->pkg->package)
1373#endif
1374 {
1375 insert_packagenamelist(p, cpkg->pkg->package);
1376 }
1377 }
1378 uninstall(pkgs, I1CUR(pointer)->value);
1379 }
1380 free_instonelist(list);
1381 return 1;
1382 } else {
1383 assert(I1CUR(list) == NULL);
1384 free_instonelist(list);
1385 return 0;
1386 }
1387}
1388
1389/******************/
1390
1391HASH_IMPL(sourcetbl, char *, dpkg_source *, SIZEOFHASHMAP, strhash, strcmp,
1392 KEEP(char*), free_source);
1393
1394static dpkg_sources *read_sources_file(char *filename, int n_arches) {
1395 FILE *f;
1396 dpkg_sources *result;
1397 dpkg_source *src;
1398 int i;
1399
1400 f = fopen(filename, "r");
1401 if (f == NULL && errno != ENOENT) {
1402 die("read_sources_file: couldn't open file:");
1403 }
1404
1405 result = block_malloc(sizeof(dpkg_sources));
1406 if (result == NULL) die("read_sources_file alloc 1:");
1407
1408 result->n_arches = n_arches;
1409 result->archname = block_malloc(sizeof(char*) * n_arches);
1410 if (result->archname == NULL) die("read_sources_file alloc 2:");
1411 for (i = 0; i < n_arches; i++) result->archname[i] = NULL;
1412 result->unclaimedpackages = block_malloc(sizeof(ownedpackagelist*)
1413 * n_arches);
1414 if (result->unclaimedpackages == NULL) die("read_sources_file alloc 3:");
1415 for (i = 0; i < n_arches; i++) result->unclaimedpackages[i] = NULL;
1416
1417 result->sources = new_sourcetbl();
1418
1419 if (f != NULL) {
1420 while ((src = read_source(f, result))) {
1421 dpkg_source *old = lookup_sourcetbl(result->sources, src->package);
1422 if (old == NULL) {
1423 add_sourcetbl(result->sources, src->package, src);
1424 } else {
1425 if (versioncmp(old->version, src->version) || 1) {
1426 int i;
1427 old = replace_sourcetbl(result->sources, src->package, src);
1428 for (i = 0; i < old->owner->n_arches; i++) {
1429 assert(old->packages[i] == NULL);
1430 }
1431 free_source(old);
1432 } else {
1433 int i;
1434 for (i = 0; i < src->owner->n_arches; i++) {
1435 assert(src->packages[i] == NULL);
1436 }
1437 free_source(src);
1438 }
1439 }
1440 }
1441 fclose(f);
1442 }
1443
1444 return result;
1445}
1446
1447void free_sources(dpkg_sources *s) {
1448 int i;
1449 if (s == NULL) return;
1450 free_sourcetbl(s->sources);
1451 for (i = 0; i < s->n_arches; i++) {
1452 /* block_free(s->archname[i]); */
1453 free_ownedpackagelist(s->unclaimedpackages[i]);
1454 }
1455 block_free(s->archname, s->n_arches * sizeof(char*));
1456 block_free(s->unclaimedpackages, s->n_arches * sizeof(ownedpackagelist*));
1457 block_free(s, sizeof(dpkg_sources));
1458}
1459
1460static dpkg_source *new_source(dpkg_sources *owner) {
1461 dpkg_source *result;
1462 int i;
1463
1464 result = block_malloc(sizeof(dpkg_source));
1465 if (result == NULL) die("new_source alloc 1:");
1466
1467 result->package = NULL;
1468 result->version = NULL;
1469 result->details = NULL;
1470 result->fake = 0;
1471
1472 result->owner = owner;
1473 result->packages = block_malloc(sizeof(packagelist*) * owner->n_arches);
1474 if (result->packages == NULL) die("new_source alloc 2:");
1475 for (i = 0; i < owner->n_arches; i++) {
1476 result->packages[i] = NULL;
1477 }
1478
1479 return result;
1480}
1481
1482static dpkg_source *read_source(FILE *f, dpkg_sources *owner) {
1483 dpkg_source *result;
1484 dpkg_paragraph *para;
1485 dpkg_entry *e;
1486
1487 para = read_paragraph(f);
1488 if (para == NULL) return NULL;
1489
1490 result = new_source(owner);
1491 result->details = para;
1492
1493 for (e = &para->entry[0]; e < &para->entry[para->n_entries]; e++) {
1494 if (strcmp("Package", e->name) == 0) {
1495 result->package = my_strdup(e->value);
1496 if (result->package == NULL)
1497 die("read_source strdup:");
1498 result->package[strlen(result->package)-1] = '\0';
1499 }
1500
1501 if (strcmp("Version", e->name) == 0) {
1502 result->version = my_strdup(e->value);
1503 if (result->version == NULL)
1504 die("read_source strdup:");
1505 result->version[strlen(result->version)-1] = '\0';
1506 }
1507 }
1508
1509 return result;
1510}
1511
1512void free_source(dpkg_source *s) {
1513 int i;
1514 if (s == NULL) return;
1515 assert(s->owner != NULL); /* shouldn't have allocated it */
1516 /* block_free(s->package); */
1517 /* block_free(s->version); */
1518 free_paragraph(s->details);
1519 for (i = 0; i < s->owner->n_arches; i++) {
1520 free_ownedpackagelist(s->packages[i]);
1521 }
1522 block_free(s->packages, s->owner->n_arches * sizeof(ownedpackagelist*));
1523 block_free(s, sizeof(dpkg_source));
1524}
1525
1526/******************************/
1527
1528dpkg_sources *read_directory(char *dir, int n_arches, char *archname[]) {
1529 char buf[1000];
1530 dpkg_sources *srcs;
1531 int i;
1532
1533 snprintf(buf, 1000, "%s/Sources", dir);
1534 srcs = read_sources_file(buf, n_arches);
1535
1536 for (i = 0; i < n_arches; i++) {
1537 FILE *f;
1538 dpkg_package *pkg;
1539
1540 srcs->archname[i] = my_strdup(archname[i]);
1541
1542 snprintf(buf, 1000, "%s/Packages_%s", dir, archname[i]);
1543 f = fopen(buf, "r");
1544 if (f == NULL && errno != ENOENT) die("load_dirctory fopen:");
1545 if (f != NULL) {
1546 while ((pkg = read_package(f))) {
1547 dpkg_source *src = lookup_sourcetbl(srcs->sources, pkg->source);
1548 if (src == NULL) {
1549 src = new_source(srcs);
1550 src->fake = 1;
1551 src->package = my_strdup(pkg->source);
1552 src->version = my_strdup(pkg->source_ver);
1553 add_sourcetbl(srcs->sources, src->package, src);
1554 }
1555 insert_ownedpackagelist(&src->packages[i], pkg);
1556 }
1557 fclose(f);
1558 }
1559 }
1560
1561 return srcs;
1562}
1563
1564void write_directory(char *dir, dpkg_sources *srcs) {
1565 FILE *src;
1566 FILE *archfile[100];
1567 char buf[1000];
1568 int i;
1569 sourcetbl_iter srciter;
1570
1571 snprintf(buf, 1000, "%s/Sources", dir);
1572 src = fopen(buf, "w");
1573 if (!src) die("write_directory: Couldn't open Sources file for output");
1574
1575 for (i = 0; i < srcs->n_arches; i++) {
1576 snprintf(buf, 1000, "%s/Packages_%s", dir, srcs->archname[i]);
1577 archfile[i] = fopen(buf, "w");
1578 }
1579
1580 for (srciter = first_sourcetbl(srcs->sources);
1581 !done_sourcetbl(srciter);
1582 srciter = next_sourcetbl(srciter))
1583 {
1584 ownedpackagelist *p;
1585 int i;
1586
1587 if (!srciter.v->fake)
1588 write_paragraph(src, srciter.v->details);
1589
1590 for (i = 0; i < srcs->n_arches; i++) {
1591 for (p = srciter.v->packages[i]; p != NULL; p = p->next) {
1592 write_paragraph(archfile[i], p->value->details);
1593 }
1594 }
1595 }
1596
1597 fclose(src);
1598 for (i = 0; i < srcs->n_arches; i++) {
1599 fclose(archfile[i]);
1600 }
1601}
1602
1603/*********************/
1604
1605HASH_IMPL(sourcenotetbl, char *, dpkg_source_note *, SIZEOFHASHMAP, strhash, strcmp,
1606 KEEP(char*), free_source_note);
1607
1608dpkg_source_note *new_source_note(dpkg_source *src, int n_arches) {
1609 dpkg_source_note *result = block_malloc(sizeof(dpkg_source_note));
1610 int i;
1611
1612 if (result == NULL) die("new_source_note alloc 1:");
1613 result->source = src;
1614 result->n_arches = n_arches;
1615 result->binaries = block_malloc(n_arches * sizeof(packagelist*));
1616 if (result->binaries == NULL) die("new_source_note alloc 2:");
1617 for (i = 0; i < n_arches; i++) {
1618 result->binaries[i] = NULL;
1619 }
1620 return result;
1621}
1622
1623void free_source_note(dpkg_source_note *srcn) {
1624 int i;
1625
1626 if (srcn == NULL) return;
1627
1628 if (srcn->binaries != NULL) {
1629 for (i = 0; i < srcn->n_arches; i++) {
1630 free_packagelist(srcn->binaries[i]);
1631 }
1632 block_free(srcn->binaries, sizeof(packagelist*) * srcn->n_arches);
1633 }
1634 block_free(srcn, sizeof(dpkg_source_note));
1635}
1636
1637#ifdef DEBUG
1638static int is_sources_note(dpkg_sources_note *srcsn) {
1639 int i;
1640
1641 assert(srcsn != NULL);
1642 assert(srcsn->magic == 0xa1eebabe);
1643 assert(srcsn->pkgs != NULL);
1644 for (i = 0; i < srcsn->n_arches; i++) {
1645 assert(srcsn->pkgs[i] != NULL && srcsn->archname[i] != NULL);
1646 assert(strcmp(srcsn->archname[i], srcsn->pkgs[i]->arch) == 0);
1647 }
1648
1649 return 1;
1650}
1651#endif
1652
1653dpkg_sources_note *new_sources_note(int n_arches, char **archname) {
1654 dpkg_sources_note *result = block_malloc(sizeof(dpkg_sources_note));
1655 int i;
1656
1657 if (result == NULL) die("new_sources_note alloc 1:");
1658 result->magic = 0xA1EEBABE;
1659 result->sources = new_sourcenotetbl();
1660 result->pkgs = block_malloc(n_arches * sizeof(dpkg_packages*));
1661 if (result->pkgs == NULL) die("new_sources_note alloc 2:");
1662 result->archname = block_malloc(n_arches * sizeof(char*));
1663 if (result->archname == NULL) die("new_sources_note alloc 3:");
1664
1665 result->n_arches = n_arches;
1666 for (i = 0; i < n_arches; i++) {
1667 result->archname[i] = my_strdup(archname[i]);
1668 result->pkgs[i] = new_packages(result->archname[i]);
1669 }
1670 result->undo = NULL;
1671 return result;
1672}
1673
1674void free_sources_note(dpkg_sources_note *srcsn) {
1675 int i;
1676 if (srcsn == NULL) return;
1677 assert(is_sources_note(srcsn));
1678 srcsn->magic = 0xBABEA1EE;
1679 free_sourcenotetbl(srcsn->sources);
1680 for (i = 0; i < srcsn->n_arches; i++) {
1681 free_packages(srcsn->pkgs[i]);
1682 /* block_free(srcsn->archname[i]); */
1683 }
1684 block_free(srcsn->pkgs, sizeof(dpkg_packages*) * srcsn->n_arches);
1685 block_free(srcsn->archname, sizeof(char*) * srcsn->n_arches);
1686 free_source_note_listlist(srcsn->undo);
1687 block_free(srcsn, sizeof(dpkg_sources_note));
1688}
1689
1690static void new_op(dpkg_sources_note *srcsn) {
1691 assert(is_sources_note(srcsn));
1692 insert_source_note_listlist(&srcsn->undo, NULL);
1693}
1694static void save_source_note(dpkg_sources_note *srcsn, dpkg_source_note *srcn) {
1695 source_note_list **where;
1696 assert(is_sources_note(srcsn));
1697 assert(srcsn->undo != NULL);
1698
1699 for (where = &srcsn->undo->value;
1700 *where != NULL;
1701 where = &(*where)->next)
1702 {
1703 if ((*where)->value->source == srcn->source)
1704 return; /* already saved */
1705 }
1706
1707 insert_source_note_list(where, copy_source_note(srcn));
1708}
1709static void save_empty_source_note(dpkg_sources_note *srcsn, dpkg_source *src) {
1710 dpkg_source_note *srcn;
1711 source_note_list **where;
1712
1713 for (where = &srcsn->undo->value;
1714 *where != NULL;
1715 where = &(*where)->next)
1716 {
1717 if ((*where)->value->source == src)
1718 return; /* already saved */
1719 }
1720
1721 srcn = block_malloc(sizeof(dpkg_source_note));
1722 if (srcn == NULL) die("save_empty_source_note alloc:");
1723 assert(is_sources_note(srcsn));
1724
1725 srcn->source = src;
1726 srcn->n_arches = 0;
1727 srcn->binaries = NULL;
1728
1729 insert_source_note_list(where, srcn);
1730}
1731
1732typedef enum { DO_ARCHALL = 0, SKIP_ARCHALL = 1 } do_this;
1733static void remove_binaries_by_arch(dpkg_sources_note *srcsn,
1734 dpkg_source_note *srcn, int archnum,
1735 do_this arch_all)
1736{
1737 packagelist *p;
1738 packagelist *leftovers = NULL, **addto = &leftovers;
1739 assert(is_sources_note(srcsn));
1740
1741 assert(arch_all == SKIP_ARCHALL || NULL == lookup_sourcenotetbl(srcsn->sources,srcn->source->package));
1742 /* if we're removing the entire binary, we should already have
1743 * removed the source. if we're removing just the binaries on this
1744 * arch (not arch:all) then we may be keeping the source
1745 *
1746 * really a logical XOR, I think. we don't rely on this assertion
1747 * here
1748 */
1749
1750 for (p = srcn->binaries[archnum]; p != NULL; p = p->next) {
1751 dpkg_collected_package *cpkg;
1752 if (arch_all == SKIP_ARCHALL && p->value->arch_all) {
1753 insert_packagelist(addto, p->value);
1754 addto = &(*addto)->next;
1755 continue;
1756 }
1757 cpkg = lookup_packagetbl(srcsn->pkgs[archnum]->packages,
1758 p->value->package);
1759 remove_package(srcsn->pkgs[archnum], cpkg);
1760 }
1761 free_packagelist(srcn->binaries[archnum]);
1762 srcn->binaries[archnum] = leftovers;
1763}
1764
1765typedef enum { NOTUNDOABLE = 0, UNDOABLE = 1 } undoable;
1766static void add_binaries_by_arch(dpkg_sources_note *srcsn,
1767 dpkg_source_note *srcn, dpkg_source *src,
1768 int archnum, undoable undoop, do_this arch_all)
1769{
1770 ownedpackagelist *p;
1771 const char *archname = srcsn->archname[archnum];
1772 int origarchnum = -1;
1773 int i;
1774
1775 assert(is_sources_note(srcsn));
1776 assert(srcn == lookup_sourcenotetbl(srcsn->sources,srcn->source->package));
1777 for (i = 0; i < src->owner->n_arches; i++) {
1778 if (strcmp(archname, src->owner->archname[i]) == 0) {
1779 origarchnum = i;
1780 break;
1781 }
1782 }
1783 if (origarchnum == -1) return; /* nothing to add, no biggie */
1784
1785 for (p = src->packages[origarchnum]; p != NULL; p = p->next) {
1786 dpkg_collected_package *cpkg;
1787
1788 if (arch_all == SKIP_ARCHALL && p->value->arch_all) continue;
1789
1790 if ((cpkg = lookup_packagetbl(srcsn->pkgs[archnum]->packages,
1791 p->value->package)))
1792 {
1793 dpkg_source_note *srcnB;
1794 packagelist **p;
1795
1796 if (!undoop) {
1797 printf("conflict w/o undo: binary %s, owned by %s, replaced by %s\n", cpkg->pkg->package, cpkg->pkg->source, src->package);
1798 fflush(stdout);
1799 }
1800 srcnB = lookup_sourcenotetbl(srcsn->sources, cpkg->pkg->source);
1801 assert(srcnB != NULL);
1802
1803 for (p = &srcnB->binaries[archnum]; *p != NULL; p = &(*p)->next) {
1804 if ((*p)->value == cpkg->pkg) break;
1805 }
1806 assert(*p != NULL); /* binary should be from source */
1807
1808 assert(undoop);
1809 save_source_note(srcsn, srcnB);
1810 remove_package(srcsn->pkgs[archnum], cpkg);
1811 remove_packagelist(p);
1812 }
1813
1814 add_package(srcsn->pkgs[archnum], p->value);
1815 insert_packagelist(&srcn->binaries[archnum], p->value);
1816 }
1817}
1818
1819void upgrade_source(dpkg_sources_note *srcsn, dpkg_source *src) {
1820 dpkg_source_note *srcn;
1821 int i;
1822
1823 new_op(srcsn);
1824
1825 assert(is_sources_note(srcsn));
1826 /* first, find the old source, if it exists */
1827 srcn = remove_sourcenotetbl(srcsn->sources, src->package);
1828 if (srcn != NULL) {
1829 save_source_note(srcsn, srcn);
1830 for (i = 0; i < srcn->n_arches; i++) {
1831 remove_binaries_by_arch(srcsn, srcn, i, DO_ARCHALL);
1832 }
1833 free_source_note(srcn);
1834 } else {
1835 save_empty_source_note(srcsn, src);
1836 }
1837
1838 /* then add the new one */
1839 srcn = new_source_note(src, srcsn->n_arches);
1840 add_sourcenotetbl(srcsn->sources, src->package, srcn);
1841 for (i = 0; i < srcsn->n_arches; i++) {
1842 add_binaries_by_arch(srcsn, srcn, src, i, UNDOABLE, DO_ARCHALL);
1843 }
1844 assert(is_sources_note(srcsn));
1845}
1846
1847void upgrade_arch(dpkg_sources_note *srcsn, dpkg_source *src, char *arch) {
1848 dpkg_source_note *srcn;
1849 int archnum = -1;
1850 int i;
1851
1852 assert(is_sources_note(srcsn));
1853 /* first, find the old source */
1854 srcn = lookup_sourcenotetbl(srcsn->sources, src->package);
1855
1856 assert(srcn != NULL);
1857 new_op(srcsn);
1858 save_source_note(srcsn, srcn);
1859
1860 /* then lookup the archnum */
1861 for (i = 0; i < srcsn->n_arches; i++) {
1862 if (strcmp(arch, srcsn->archname[i]) == 0) {
1863 archnum = i;
1864 break;
1865 }
1866 }
1867 if (archnum == -1) die("upgrade_arch: unknown arch");
1868
1869 /* then remove the old stuff and add the new */
1870 remove_binaries_by_arch(srcsn, srcn, archnum, SKIP_ARCHALL);
1871 add_binaries_by_arch(srcsn, srcn, src, archnum, UNDOABLE, SKIP_ARCHALL);
1872 assert(is_sources_note(srcsn));
1873}
1874
1875void remove_source(dpkg_sources_note *srcsn, char *name) {
1876 dpkg_source_note *srcn;
1877 int i;
1878
1879 assert(is_sources_note(srcsn));
1880 srcn = remove_sourcenotetbl(srcsn->sources, name);
1881 assert(srcn != NULL);
1882
1883 new_op(srcsn);
1884 save_source_note(srcsn, srcn);
1885 for (i = 0; i < srcn->n_arches; i++) {
1886 remove_binaries_by_arch(srcsn, srcn, i, DO_ARCHALL);
1887 }
1888 free_source_note(srcn);
1889 assert(is_sources_note(srcsn));
1890}
1891
1892int can_undo(dpkg_sources_note *srcsn) {
1893 assert(is_sources_note(srcsn));
1894 return srcsn->undo != NULL;
1895}
1896
1897void undo_change(dpkg_sources_note *srcsn) {
1898 dpkg_source_note *srcnO, *srcnC; /* old, current */
1899 source_note_list *srcnl;
1900 int i;
1901
1902 assert(is_sources_note(srcsn));
1903 assert(can_undo(srcsn));
1904
1905 srcnl = remove_source_note_listlist(&srcsn->undo);
1906
1907 while(srcnl) {
1908 srcnO = remove_source_note_list(&srcnl);
1909 assert(srcnO != NULL); /* can_undo() implies this is true... */
1910
1911 srcnC = remove_sourcenotetbl(srcsn->sources, srcnO->source->package);
1912 if (srcnC != NULL) {
1913 for (i = 0; i < srcnC->n_arches; i++) {
1914 remove_binaries_by_arch(srcsn, srcnC, i, DO_ARCHALL);
1915 }
1916 free_source_note(srcnC);
1917 assert(!lookup_sourcenotetbl(srcsn->sources,
1918 srcnO->source->package));
1919 }
1920
1921 if (srcnO->binaries == NULL) {
1922 /* no original source */
1923 assert(srcnC != NULL); /* some sort of no-op? freaky. */
1924 free_source_note(srcnO);
1925 } else {
1926 packagelist *p;
1927 /* original source */
1928 add_sourcenotetbl(srcsn->sources, srcnO->source->package, srcnO);
1929 for (i = 0; i < srcsn->n_arches; i++) {
1930 for (p = srcnO->binaries[i]; p != NULL; p = p->next) {
1931 add_package(srcsn->pkgs[i], p->value);
1932 }
1933 }
1934 }
1935 }
1936}
1937
1938LIST_IMPL(source_note_list, dpkg_source_note *, free_source_note,
1939 block_malloc, block_free);
1940LIST_IMPL(source_note_listlist, source_note_list *, free_source_note_list,
1941 block_malloc, block_free);
1942
1943void commit_changes(dpkg_sources_note *srcsn) {
1944 assert(is_sources_note(srcsn));
1945 free_source_note_listlist(srcsn->undo);
1946 srcsn->undo = NULL;
1947}
1948
1949dpkg_source_note *copy_source_note(dpkg_source_note *srcn) {
1950 dpkg_source_note *srcn2;
1951 packagelist *src, **dest;
1952 int i;
1953
1954 assert(srcn->binaries != NULL);
1955
1956 srcn2 = block_malloc(sizeof(dpkg_source_note));
1957 if (srcn2 == NULL) die("copy_source_note alloc:");
1958
1959 srcn2->source = srcn->source;
1960 srcn2->n_arches = srcn->n_arches;
1961 srcn2->binaries = block_malloc(sizeof(packagenamelist*) * srcn2->n_arches);
1962 if (srcn2->binaries == NULL) die("copy_source_note alloc:");
1963
1964 for (i = 0; i < srcn2->n_arches; i++) {
1965 dest = &(srcn2->binaries[i]);
1966 *dest = NULL;
1967 for (src = srcn->binaries[i]; src; src = src->next) {
1968 insert_packagelist(dest, src->value);
1969 dest = &((*dest)->next);
1970 }
1971 }
1972
1973 return srcn2;
1974}
1975
1976void write_notes(char *dir, dpkg_sources_note *srcsn) {
1977 FILE *src;
1978 FILE *archfile[100];
1979 char buf[1000];
1980 int i;
1981 sourcenotetbl_iter srciter;
1982
1983 assert(is_sources_note(srcsn));
1984 snprintf(buf, 1000, "%s/Sources", dir);
1985 src = fopen(buf, "w");
1986 for (i = 0; i < srcsn->n_arches; i++) {
1987 snprintf(buf, 1000, "%s/Packages_%s", dir, srcsn->archname[i]);
1988 archfile[i] = fopen(buf, "w");
1989 }
1990
1991 for (srciter = first_sourcenotetbl(srcsn->sources);
1992 !done_sourcenotetbl(srciter);
1993 srciter = next_sourcenotetbl(srciter))
1994 {
1995 packagelist *p;
1996 int i;
1997
1998 if (!srciter.v->source->fake)
1999 write_paragraph(src, srciter.v->source->details);
2000
2001 for (i = 0; i < srcsn->n_arches; i++) {
2002 for (p = srciter.v->binaries[i]; p != NULL; p = p->next) {
2003 write_paragraph(archfile[i], p->value->details);
2004 }
2005 }
2006 }
2007
2008 fclose(src);
2009 for (i = 0; i < srcsn->n_arches; i++) {
2010 fclose(archfile[i]);
2011 }
2012}
2013
02014
=== added file 'update_out/dpkg.h'
--- update_out/dpkg.h 1970-01-01 00:00:00 +0000
+++ update_out/dpkg.h 2015-06-09 16:34:27 +0000
@@ -0,0 +1,207 @@
1#ifndef DPKG_H
2#define DPKG_H
3
4#include "templates.h"
5#include "memory.h"
6
7#include <stdio.h>
8
9/**************************************************************************
10 * Coping with an rfc822-esque field
11 */
12
13typedef struct dpkg_entry dpkg_entry;
14struct dpkg_entry {
15 char *name;
16 char *value;
17};
18
19typedef struct dpkg_paragraph dpkg_paragraph;
20struct dpkg_paragraph {
21 int n_entries;
22 int n_allocated;
23 dpkg_entry *entry;
24};
25
26/**************************************************************************
27 * Coping with a package (or many pkgs) as an abstract entity
28 */
29
30typedef enum {dr_NOOP,dr_LT,dr_LTEQ,dr_EQ,dr_GTEQ,dr_GT} dependency_relation;
31extern char *dependency_relation_sym[];
32
33typedef struct dependency dependency;
34struct dependency {
35 char *package;
36 dependency_relation op;
37 char *version;
38};
39
40LIST(deplist, dependency*);
41LIST(deplistlist, deplist*);
42
43LIST(packagenamelist, char*);
44LIST(ownedpackagenamelist, char*);
45
46typedef struct dpkg_package dpkg_package;
47
48struct dpkg_package {
49 char *package;
50 char *version;
51
52 char *source;
53 char *source_ver;
54
55 int priority;
56
57 int arch_all;
58
59 deplistlist *depends[4];
60 deplist *conflicts;
61 ownedpackagenamelist *provides;
62
63 dpkg_paragraph *details;
64};
65
66LIST(packagelist, dpkg_package *);
67LIST(ownedpackagelist, dpkg_package *);
68
69typedef struct satisfieddep satisfieddep;
70
71struct satisfieddep {
72 /* dependency *dep; */
73 deplist *depl;
74 packagelist *pkgs;
75};
76
77LIST(satisfieddeplist, satisfieddep *);
78
79/**************************************************************************
80 * Coping with a source package (and collections thereof) as an abstract
81 * entity, owning a bunch of binary packages
82 */
83
84typedef struct dpkg_source dpkg_source;
85struct dpkg_source {
86 char *package;
87 char *version;
88
89 int fake;
90
91 struct dpkg_sources *owner;
92 ownedpackagelist **packages; /* one for each architecture */
93
94 dpkg_paragraph *details;
95};
96
97HASH(sourcetbl,char *,dpkg_source *);
98
99typedef struct dpkg_sources dpkg_sources;
100struct dpkg_sources {
101 int n_arches;
102 char **archname;
103 sourcetbl *sources;
104 ownedpackagelist **unclaimedpackages; /* one for each arch */
105};
106
107/**************************************************************************
108 */
109
110typedef struct dpkg_collected_package dpkg_collected_package;
111struct dpkg_collected_package {
112 dpkg_package *pkg;
113
114 int installed, conflicted;
115
116 enum { UNKNOWN, YES } installable;
117 packagenamelist *mayaffect;
118
119 /* on update, the installability_checked of each /mayaffect/ed package
120 * is cleared, and the mayaffect list is cleared.
121 *
122 * note that installable = NO couldn't be maintained over adding a package
123 * to testing. installable = YES can be, thanks to the mayaffect list
124 * (once a package is removed, everything it mayaffect must be set back
125 * to unknown, but everything else is okay)
126 */
127};
128
129LIST(collpackagelist, dpkg_collected_package *);
130
131/**************************************************************************
132 */
133
134typedef struct dpkg_provision dpkg_provision;
135struct dpkg_provision {
136 char *version;
137 dpkg_collected_package *pkg;
138};
139
140LIST(virtualpkg, dpkg_provision);
141
142HASH(virtualpkgtbl,char *,virtualpkg *);
143HASH(packagetbl,char *,dpkg_collected_package *);
144
145typedef struct dpkg_packages dpkg_packages;
146struct dpkg_packages {
147 char *arch;
148 packagetbl *packages;
149 virtualpkgtbl *virtualpkgs;
150};
151
152typedef struct dpkg_source_note dpkg_source_note;
153struct dpkg_source_note {
154 dpkg_source *source; /* unowned */
155 int n_arches;
156 packagelist **binaries; /* one for each arch */
157};
158HASH(sourcenotetbl, char *, dpkg_source_note *);
159
160LIST(source_note_list, dpkg_source_note *);
161 /* contains a copy of the previous source_note */
162LIST(source_note_listlist, source_note_list *);
163 /* contains a copy of all the source_notes modified by the last op */
164
165typedef struct dpkg_sources_note dpkg_sources_note;
166struct dpkg_sources_note {
167 unsigned long magic;
168 sourcenotetbl *sources;
169 int n_arches;
170 dpkg_packages **pkgs;
171 char **archname;
172
173 source_note_listlist *undo;
174};
175
176void free_packages(dpkg_packages *pkgs);
177void free_sources(dpkg_sources *s);
178
179dpkg_packages *get_architecture(dpkg_sources *srcs, char *arch);
180
181/* parsing things */
182int checkinstallable(dpkg_packages *pkgs, collpackagelist *instoneof);
183int checkinstallable2(dpkg_packages *pkgs, char *pkgname);
184satisfieddeplist *checkunsatisfiabledeps(dpkg_packages *pkgs,
185 deplistlist *deps);
186
187dpkg_sources *read_directory(char *dir, int n_arches, char *archname[]);
188void write_directory(char *dir, dpkg_sources *srcs);
189
190void free_source(dpkg_source *s);
191
192/* adding and deleting and stuff */
193dpkg_sources_note *new_sources_note(int n_arches, char **archname);
194void remove_source(dpkg_sources_note *srcsn, char *name);
195void upgrade_source(dpkg_sources_note *srcsn, dpkg_source *src);
196void upgrade_arch(dpkg_sources_note *srcsn, dpkg_source *src, char *arch);
197void write_notes(char *dir, dpkg_sources_note *srcsn);
198void free_sources_note(dpkg_sources_note *srcsn);
199void free_source_note(dpkg_source_note *srcn);
200void undo_change(dpkg_sources_note *srcsn);
201int can_undo(dpkg_sources_note *srcsn);
202void commit_changes(dpkg_sources_note *srcsn);
203
204int versioncmp(char *left, char *right);
205int cmpversions(char *left, int op, char *right);
206
207#endif
0208
=== added file 'update_out/freelist.c'
--- update_out/freelist.c 1970-01-01 00:00:00 +0000
+++ update_out/freelist.c 2015-06-09 16:34:27 +0000
@@ -0,0 +1,188 @@
1#include <stdio.h>
2#include <stdlib.h>
3#include "templates.h"
4
5typedef unsigned long ul;
6
7#define SIZE (sizeof(ul) * 8)
8#define ROUND_DOWN(x) ((x) & ~(SIZE-1))
9#define ROUND_UP(x) ROUND_DOWN((x) + (SIZE-1))
10#define NEXT_UP(x) ROUND_DOWN((x) + SIZE)
11#define NEXT_DOWN(x) ROUND_DOWN((x) - 1)
12
13#define SETBIT(s,p) \
14 assert( (bits[(s)/SIZE] & (p)) == (setp ? 0 : (p)) ); \
15 if (setp) bits[(s)/SIZE] |= (p); \
16 else bits[(s)/SIZE] &= ~(p)
17
18#define GETBIT(s) (bits[ROUND_DOWN(s)/SIZE] & (1ul << (NEXT_UP(s) - s - 1)))
19
20size_t count_free_bits_back(ul *bits, size_t s) {
21 size_t cnt = 0;
22 ul w = ROUND_DOWN(s) / SIZE;
23 size_t add = s % SIZE;
24 ul off = (~0ul) << (SIZE - add);
25 ul H, d;
26
27 while ((bits[w] & off) == 0) {
28 cnt += add;
29 add = SIZE;
30 off = ~0ul;
31 if (w == 0)
32 return cnt;
33 w--;
34 }
35
36 H = add;
37 add = 0;
38 while ((d = (H - add) / 2) > 0) {
39 ul offM = (off >> d) & off;
40 if (bits[w] & offM) {
41 off = offM;
42 H = H - d;
43 } else {
44 add = H - d;
45 }
46 }
47 cnt += add;
48 return cnt;
49}
50
51size_t count_free_bits_after(ul *bits, size_t s, size_t end) {
52 size_t cnt = 0;
53 ul w = ROUND_DOWN(s) / SIZE;
54 size_t add = SIZE - s % SIZE;
55 ul off = (~0ul) >> (SIZE - add);
56 ul H, d;
57
58 end /= SIZE;
59
60 while ((bits[w] & off) == 0) {
61 cnt += add;
62 add = SIZE;
63 off = ~0ul;
64 w++;
65 if (w == end)
66 return cnt;
67 }
68
69 H = add;
70 add = 0;
71 while ((d = (H - add) / 2) > 0) {
72 ul offM = off << d;
73 if (bits[w] & offM) {
74 off = offM;
75 H = H - d;
76 } else {
77 add = H - d;
78 }
79 }
80 cnt += add;
81 return cnt;
82}
83
84void find_long_freebits(ul *bits, size_t s, ul *start, size_t *size) {
85 ul clen = 0;
86 ul bstart = 0, blen = 0;
87 ul i, k;
88
89 for (i = 0; i < s; i++) {
90 if (bits[i] == 0) {
91 clen++;
92 } else {
93 if (clen > blen) {
94 bstart = i - clen;
95 blen = clen;
96 }
97 clen = 0;
98 }
99 }
100
101 if (blen == 0) return;
102
103 bstart *= SIZE; blen *= SIZE;
104 k = count_free_bits_back(bits, bstart);
105 bstart -= k; blen += k;
106
107 blen += count_free_bits_after(bits, bstart + blen, s*SIZE);
108
109 *start = bstart; *size = blen;
110}
111
112void mark_bits(ul *bits, ul s, size_t size, int setp) {
113 ul e = s+size;
114
115 ul rds = ROUND_DOWN(s);
116 ul nus = rds + SIZE;
117 ul rue = ROUND_UP(e);
118
119 ul patl = (~0UL) >> (s % SIZE);
120 ul patr = (~0UL) << (rue - e);
121
122 assert(size > 0);
123
124 /* bits[s1..e1] get touched, but bits[s1], bits[e1] only partially
125 *
126 * if s1 == e1, then bits[s1] get touched from [s%SIZE, e%SIZE)
127 * else
128 * bits[s1] gets touched from [s%SIZE, SIZE)
129 * bits[s2..e1) get reset completely
130 * bits[e1] gets touched from [0, e%SIZE)
131 */
132
133 if (nus >= e) {
134 /* ROUND_DOWN(s) <= s < e <= NEXT_UP(s) */
135 SETBIT(rds, patl & patr);
136 } else {
137 /* ROUND_DOWN(s) <= s < NEXT_UP(s) <= NEXT_DOWN(e) < e */
138 ul rde = ROUND_DOWN(e);
139
140 SETBIT(rds, patl);
141 SETBIT(rde, patr);
142 while (nus < rde) {
143 SETBIT(nus, ~0UL);
144 nus += SIZE;
145 }
146 }
147}
148
149void print_bits(ul *bits, ul s) {
150 ul i;
151 putchar(' ');
152 for (i = 0; i < s * SIZE; i++) {
153 putchar( GETBIT(i) ? '1' : '0' );
154 }
155}
156
157#ifdef TESTBIN
158
159#define X 2
160int main(void) {
161 ul memory[X];
162 ul l, r;
163 ul k = 5;
164
165 memset(memory, 0, sizeof(memory));
166 for (l = 0; l < X*SIZE; l += k) {
167 for (r = 1; l+(r*r) < X*SIZE; r++) {
168
169 printf("%lu %lu (%lu %lu", l, r*r,
170 (unsigned long) count_free_bits_back(memory, X*SIZE), (unsigned long) X*SIZE);
171 mark_bits(memory, l, r*r, 1);
172 printf("; %lu %lu %lu; %lu %lu %lu;): ",
173 (unsigned long) count_free_bits_back(memory, X*SIZE) + l + r*r,
174 (unsigned long) count_free_bits_after(memory, l + r*r, X*SIZE) + l + r*r,
175 (unsigned long) X*SIZE,
176 (unsigned long) count_free_bits_back(memory, l),
177 (unsigned long) count_free_bits_after(memory, 0, X*SIZE),
178 l);
179 print_bits(memory, X);
180 printf("\n");
181
182 mark_bits(memory, l, r*r, 0);
183 }
184 }
185
186 return 0;
187}
188#endif
0189
=== added file 'update_out/freelist.h'
--- update_out/freelist.h 1970-01-01 00:00:00 +0000
+++ update_out/freelist.h 2015-06-09 16:34:27 +0000
@@ -0,0 +1,14 @@
1
2#ifndef FREELIST_H
3#define FREELIST_H
4
5#include <stdlib.h>
6
7typedef unsigned long flb_t;
8
9void mark_bits(flb_t *bits, flb_t s, size_t size, int setp);
10size_t count_free_bits_back(flb_t *bits, size_t s);
11size_t count_free_bits_after(flb_t *bits, size_t s, size_t end);
12void find_long_freebits(flb_t *bits, flb_t s, flb_t *start, size_t *size);
13
14#endif
015
=== added file 'update_out/index.html'
--- update_out/index.html 1970-01-01 00:00:00 +0000
+++ update_out/index.html 2015-06-09 16:34:27 +0000
@@ -0,0 +1,18 @@
1<a href="README">README</a><br>
2<a href="Makefile">Makefile</a><br>
3<a href="assert.c">assert.c</a><br>
4<a href="britney-py.c">britney-py.c</a><br>
5<a href="checklib.c">checklib.c</a><br>
6<a href="dpkg.c">dpkg.c</a><br>
7<a href="freelist.c">freelist.c</a><br>
8<a href="memory.c">memory.c</a><br>
9<a href="memory2.c">memory2.c</a><br>
10<a href="memory3.c">memory3.c</a><br>
11<a href="dpkg-lib.cpp">dpkg-lib.cpp</a><br>
12<a href="dpkg.h">dpkg.h</a><br>
13<a href="freelist.h">freelist.h</a><br>
14<a href="memory.h">memory.h</a><br>
15<a href="templates.h">templates.h</a><br>
16<a href="check_out.py">check_out.py</a><br>
17<a href="check_uptodate.py">check_uptodate.py</a><br>
18<a href="update_out.py">update_out.py</a><br>
019
=== added file 'update_out/memory.c'
--- update_out/memory.c 1970-01-01 00:00:00 +0000
+++ update_out/memory.c 2015-06-09 16:34:27 +0000
@@ -0,0 +1,389 @@
1#include <stdio.h>
2#include <stdlib.h>
3
4#include "memory.h"
5#include "templates.h"
6#include "freelist.h"
7
8/**** THEORY
9 *
10
11So, we have blocks with a freelist
12
13 XXX............XXXXXXX..XXXXX.....XXXXXX......
14
15Within a block, we work with segments. A segment is...
16
17 ^..........|
18
19Every now and then we make sure we've got a decent sized segment.
20
21We have multiple blocks. They're kept ordered by the size of their
22current segment.
23
24 **********************************************/
25
26#define ALIGN 4
27
28#define FLBT_BITS (sizeof(flb_t)*8)
29#define MEMBLOCKSIZE (1 << 22)
30#define ALIGNEDSIZE(s) (((s) + ALIGN - 1) / ALIGN * ALIGN)
31
32struct memblock {
33 struct memblock *next;
34
35 size_t n_bytes; /* index of free char */
36 size_t size; /* size of block after char */
37
38 unsigned n_used_chunks; /* number of unfreed blocks */
39 size_t n_used_bytes; /* number of bytes actually used */
40 size_t n_productive_bytes; /* number of bytes used usefully */
41
42 flb_t free[MEMBLOCKSIZE/ALIGN/FLBT_BITS + 1];
43 unsigned char mem[MEMBLOCKSIZE];
44};
45typedef struct memblock memblock;
46
47static memblock *base = NULL;
48
49#ifdef MDEBUG1
50static int valid_memblock_mdebug1(struct memblock *mb) {
51 size_t cnt, i;
52 static int rarity = 0;
53
54 assert(mb->n_bytes + mb->size <= sizeof(mb->mem));
55
56 if (mb->n_used_chunks == 0) assert(mb->n_bytes == 0);
57 assert(((unsigned long)mb->mem + mb->n_bytes) % ALIGN == 0);
58
59 assert(mb->n_productive_bytes <= mb->n_used_bytes);
60 assert(mb->n_used_bytes + mb->size <= sizeof(mb->mem));
61
62#define TWO(k) (1ul << (k))
63#define CYCL(k) (~0ul / (1 + TWO(TWO(k))))
64
65 rarity++; rarity %= 25000;
66 if (rarity != 0) {
67 cnt = mb->n_used_bytes;
68 } else {
69 cnt = 0;
70 for (i = 0; i < sizeof(mb->mem)/ALIGN/FLBT_BITS+1; i++) {
71 unsigned long x = mb->free[i];
72 size_t s;
73 x = (x & CYCL(0)) + ((x >> TWO(0)) & CYCL(0));
74 x = (x & CYCL(1)) + ((x >> TWO(1)) & CYCL(1));
75 for (s = 2; (2u << s) <= FLBT_BITS; s++) {
76 x += x >> TWO(s);
77 x &= CYCL(s);
78 }
79 cnt += x * ALIGN;
80 }
81 }
82#undef TWO
83#undef CYCL
84
85 assert(cnt == mb->n_used_bytes);
86
87 return 1;
88}
89#endif
90
91#if MDEBUG3
92static int valid_memblock_mdebug3(struct memblock *mb) {
93 size_t offset, step, used;
94 unsigned chunk = 0;
95
96 offset = 0;
97 used = 0;
98 if ((unsigned long)mb->mem % ALIGN != 0)
99 offset = ALIGN - ((unsigned long)mb->mem % ALIGN);
100
101 while(offset < mb->n_bytes) {
102 step = *(size_t*)(mb->mem + offset);
103 assert(step % ALIGN == 0 || step % ALIGN == 1);
104 if (step % ALIGN == 1) step--; /* freed */
105 else used += step;
106 assert(step > 0);
107 offset += step;
108 chunk++;
109 }
110
111 assert(used == mb->n_used_bytes);
112
113 return 1;
114}
115#endif
116
117inline static int valid_memblock(struct memblock *mb) {
118 (void)mb;
The diff has been truncated for viewing.

Subscribers

People subscribed via source and target branches

to all changes: