Merge lp:~naesten/bzr-fastimport/import-tars into lp:~bzr/bzr-fastimport/fastimport.dev

[Samuel Bronson]

[Samuel Bronson (naesten)]

Hey, Ian, this is the script that generated the fast-import streams that led me to most of the bugs I've reported in bzr-fastimport so far.

[Ian Clatworthy (ian-clatworthy)]

I'm happy to include this but I think we need to explicitly check the licensing of the original code to confirm it's OK for us to redistribute it.

We probably need something like exporters/import-tars.README (and exporters/import-tars.LICENSE?) acknowledging the original author and your tweaks to it. Perhaps explicitly mention yourself as the maintainer of the script as well, if that's the case.

review: Needs Information

[Samuel Bronson (naesten)]

> I'm happy to include this but I think we need to explicitly check the
> licensing of the original code to confirm it's OK for us to redistribute it.
>
> We probably need something like exporters/import-tars.README (and exporters
> /import-tars.LICENSE?) acknowledging the original author and your tweaks to
> it. Perhaps explicitly mention yourself as the maintainer of the script as
> well, if that's the case.

Well, the most specific license info for that file is in git's top-level COPYING, as far as I can tell, which says "GPL v2 only -- not v2.1, not v3, *v2*" (paraphrased).

The contributers shown by

naesten@hydrogen:~/hacking/git% ../tig/tig v1.6.3.3 -- contrib/fast-import/import-tars.perl

(the ../tig/ is because I added something to tig to make the dates narrower) are:

2009-06-17 14:49 Johannes Schindelin + import-tars: support symlinks
2009-03-20 10:57 Giuseppe Bilotta * import-tars: separate author from commit~
2007-05-16 12:43 Junio C Hamano M Merge branch 'maint'
2007-05-16 17:22 Johannes Schindelin │`* import-tars: Use the "Link indicator" ~
2007-05-10 14:48 Junio C Hamano M │ Merge branch 'maint'
2007-05-07 21:13 Shawn O. Pearce │`* Properly handle '0' filenames in impor~
2007-05-02 13:24 Shawn O. Pearce M │ Merge branch 'gfi-maint' into gfi-mast~
2007-05-01 23:42 Johannes Schindelin │`* Teach import-tars about GNU tar's @Lon~
2007-04-29 01:52 Junio C Hamano M │ Merge branch 'maint'
2007-04-29 00:31 Junio C Hamano │`* Fix import-tars fix.
2007-04-29 01:59 Johannes Schindelin │ * import-tars: be nice to wrong director~
2007-04-24 13:51 Uwe Kleine-König * │ fix importing of subversion tars
2007-02-14 17:03 Michael Loeffler *' Use gunzip -c over gzcat in import-tars~
2007-02-12 15:17 Michael Loeffler * import-tars: brown paper bag fix for fil~
2007-02-08 15:26 Shawn O. Pearce * tar archive frontend for fast-import.

And I'd be happy to try and fix any bugs people may find in the script.

Oh, btw, I'm going to my grandpa's house for a week (... say, ten minutes ago?), leaving my computer behind, so I probably won't be able to do much except maybe talk until I'm back.

[Ian Clatworthy (ian-clatworthy)]

Where is this up too now?

[Samuel Bronson (naesten)]

On Mon, Nov 23, 2009 at 2:08 AM, Ian Clatworthy
<email address hidden> wrote:
> Where is this up too now?

Uh, mostly forgotten, sorry :-(. Maybe I'll look at it over christmas break...

Unmerged revisions

201. By Samuel Bronson on 2009-08-11

Adapt import-tars.perl to output fast-import data to stdout.

(The version from git's contrib/ directory fed it directly to 'git
fast-import'!)

Also make the script executable.

200. By Samuel Bronson on 2009-08-11

Import git's contrib/fast-import/import-tars.perl (git version 1.6.3.3).

=== added file 'exporters/import-tars.perl'
--- exporters/import-tars.perl	1970-01-01 00:00:00 +0000
+++ exporters/import-tars.perl	2009-08-11 03:35:05 +0000
@@ -0,0 +1,144 @@
+#!/usr/bin/perl
+
+## tar archive frontend for git-fast-import
+##
+## For example:
+##
+##  mkdir project; cd project; git init
+##  perl import-tars.perl *.tar.bz2 | git fast-import
+##  git whatchanged import-tars
+##
+
+use strict;
+die "usage: import-tars *.tar.{gz,bz2,Z}\n" unless @ARGV;
+
+my $branch_name = 'import-tars';
+my $branch_ref = "refs/heads/$branch_name";
+my $author_name = $ENV{'GIT_AUTHOR_NAME'} || 'T Ar Creator';
+my $author_email = $ENV{'GIT_AUTHOR_EMAIL'} || '[e-mail]';
+my $committer_name = $ENV{'GIT_COMMITTER_NAME'} || `git config --get user.name`;
+my $committer_email = $ENV{'GIT_COMMITTER_EMAIL'} || `git config --get user.email`;
+
+chomp($committer_name, $committer_email);
+
+*FI = *stdout;
+
+foreach my $tar_file (@ARGV)
+{
+	my $commit_time = time;
+	$tar_file =~ m,([^/]+)$,;
+	my $tar_name = $1;
+
+	if ($tar_name =~ s/\.(tar\.gz|tgz)$//) {
+		open(I, '-|', 'gunzip', '-c', $tar_file)
+			or die "Unable to gunzip -c $tar_file: $!\n";
+	} elsif ($tar_name =~ s/\.(tar\.bz2|tbz2)$//) {
+		open(I, '-|', 'bunzip2', '-c', $tar_file)
+			or die "Unable to bunzip2 -c $tar_file: $!\n";
+	} elsif ($tar_name =~ s/\.tar\.Z$//) {
+		open(I, '-|', 'uncompress', '-c', $tar_file)
+			or die "Unable to uncompress -c $tar_file: $!\n";
+	} elsif ($tar_name =~ s/\.tar$//) {
+		open(I, $tar_file) or die "Unable to open $tar_file: $!\n";
+	} else {
+		die "Unrecognized compression format: $tar_file\n";
+	}
+
+	my $author_time = 0;
+	my $next_mark = 1;
+	my $have_top_dir = 1;
+	my ($top_dir, %files);
+
+	while (read(I, $_, 512) == 512) {
+		my ($name, $mode, $uid, $gid, $size, $mtime,
+			$chksum, $typeflag, $linkname, $magic,
+			$version, $uname, $gname, $devmajor, $devminor,
+			$prefix) = unpack 'Z100 Z8 Z8 Z8 Z12 Z12
+			Z8 Z1 Z100 Z6
+			Z2 Z32 Z32 Z8 Z8 Z*', $_;
+		last unless length($name);
+		if ($name eq '././@LongLink') {
+			# GNU tar extension
+			if (read(I, $_, 512) != 512) {
+				die ('Short archive');
+			}
+			$name = unpack 'Z257', $_;
+			next unless $name;
+
+			my $dummy;
+			if (read(I, $_, 512) != 512) {
+				die ('Short archive');
+			}
+			($dummy, $mode, $uid, $gid, $size, $mtime,
+			$chksum, $typeflag, $linkname, $magic,
+			$version, $uname, $gname, $devmajor, $devminor,
+			$prefix) = unpack 'Z100 Z8 Z8 Z8 Z12 Z12
+			Z8 Z1 Z100 Z6
+			Z2 Z32 Z32 Z8 Z8 Z*', $_;
+		}
+		next if $name =~ m{/\z};
+		$mode = oct $mode;
+		$size = oct $size;
+		$mtime = oct $mtime;
+		next if $typeflag == 5; # directory
+
+		print FI "blob\n", "mark :$next_mark\n";
+		if ($typeflag == 2) { # symbolic link
+			print FI "data ", length($linkname), "\n", $linkname;
+			$mode = 0120000;
+		} else {
+			print FI "data $size\n";
+			while ($size > 0 && read(I, $_, 512) == 512) {
+				print FI substr($_, 0, $size);
+				$size -= 512;
+			}
+		}
+		print FI "\n";
+
+		my $path;
+		if ($prefix) {
+			$path = "$prefix/$name";
+		} else {
+			$path = "$name";
+		}
+		$files{$path} = [$next_mark++, $mode];
+
+		$author_time = $mtime if $mtime > $author_time;
+		$path =~ m,^([^/]+)/,;
+		$top_dir = $1 unless $top_dir;
+		$have_top_dir = 0 if $top_dir ne $1;
+	}
+
+	print FI <<EOF;
+commit $branch_ref
+author $author_name <$author_email> $author_time +0000
+committer $committer_name <$committer_email> $commit_time +0000
+data <<END_OF_COMMIT_MESSAGE
+Imported from $tar_file.
+END_OF_COMMIT_MESSAGE
+
+deleteall
+EOF
+
+	foreach my $path (keys %files)
+	{
+		my ($mark, $mode) = @{$files{$path}};
+		$path =~ s,^([^/]+)/,, if $have_top_dir;
+		$mode = $mode & 0111 ? 0755 : 0644 unless $mode == 0120000;
+		printf FI "M %o :%i %s\n", $mode, $mark, $path;
+	}
+	print FI "\n";
+
+	print FI <<EOF;
+tag $tar_name
+from $branch_ref
+tagger $author_name <$author_email> $author_time +0000
+data <<END_OF_TAG_MESSAGE
+Package $tar_name
+END_OF_TAG_MESSAGE
+
+EOF
+
+	close I;
+}
+close FI;