lp:~cfuhrman/+junk/netbsd-othersrc-trunk

Created by Christopher M. Fuhrman on 2012-07-27 and last modified on 2012-07-27
Get this branch:
bzr branch lp:~cfuhrman/+junk/netbsd-othersrc-trunk
Only Christopher M. Fuhrman can upload to this branch. If you are Christopher M. Fuhrman please log in for upload directions.

Related bugs

Related blueprints

Branch information

Owner:
Christopher M. Fuhrman
Status:
Development

Recent revisions

1922. By agc on 2012-05-07

Break out the routines (from tnftp) to calculate and draw a progress bar.
The code was originally written by Luke Mewburn - all mangling and bugs
courtesy of agc.

A quick (hypothetical) example of how to use this progress bar:

           progress_t prog;
           uint64_t total;
           uint64_t done;

           total = 15768554;
           progress_init(&prog, "test update bar", total);
           for (done = 0 ; done < total ; done += (total / 20) + ((random() % 20) * 1024)) {
                   progress_draw(&prog, done);
                   sleep(1);
           }
           progress_complete(&prog, done);

1921. By riz on 2012-04-25

Allow the bootstrap to install xbase.tgz, since that will unfortunately
be needed for many binary packages. One of these days, this script
should be cleaned up a bit, too.

1920. By riz on 2012-04-20

Example scripts and supporting files for creating Amazon EC2 AMIs
(Amazon Machine Images) running NetBSD. Requires an Amazon EC2 account.
Please test and extend these scripts!

The original scripts were written by Jean-Yves Migeon, heavily modified

From the README:

Notes on using these scripts for creating EC2 AMIs
--------------------------------------------------

Here are some assumptions made:

- You have the EC2 API tools already set up. If you don't already, the
  easiest way to do so is to install the misc/ec2-api-tools package
  from pkgsrc. An easy way to check if they're set up properly is
  if you can run "ec2-describe-instances" without an error.

- You are running this from a NetBSD system. It's certainly not impossible
  to update this for running on another platform, but it'll be necessary
  to adapt to using a NetBSD tools build for (at least) makefs.

- Your default EC2 profile allows ssh to your running instances from
  the location you're running this script!

- The most-used (and tested!) script (create_ec2_ami.sh) assumes that there
  are NetBSD 6.0 AMIs available on EC2 already, for bootstrapping. The
  public images in account 101367081206 (an account owned by me, Jeff Rizzo,
  a NetBSD developer) are available for use, and are searched in addition
  to the user's own AMIs. You can also specify a particular AMI to use
  for the bootstrapping. There is another script (create-new.sh) which
  is less well tested, and which may currently be broken, which allows
  you to use a Linux AMI to create a NetBSD one. You will need to
  create a 'bootstrap' image with an ext2 file system (see build_ec2_img.sh,
  specifically the "-b" flag).

CREATING AN AMAZON MACHINE IMAGE (AMI)
--------------------------------------

Once you've verified the Amazon EC2 command-line tools work (see above),
the basic workflow is as follows:

- Edit the set_ec2_env.sh script to have the location of your certs/keys.
  This file is read by several of the scripts.

- Either build sets with build.sh, or identify where the release you want to
  base your AMI on lives. You can access built sets via http, ftp, or on
  the local fs.

- Build an image for upload, using the build_ec2_img.sh script. This
  image will contain everything *except* the kernel (the kernel will be
  on a separate ext2fs partition, due to EC2 boot requirements), and will
  be resized to fit the volume created in the AMI creation script.

  An example of doing this:

  ./build_ec2_img.sh -d http://ftp.netbsd.org/pub/NetBSD/NetBSD-6.0_BETA/amd64/binary/sets

  ...this will create an image file in the current directory.

- Create the AMI from the above image plus a kernel. You have to
  specify which region to create the AMI in:

  ./create_ec2_ami.sh -i ./NetBSD-AMI.img.xz -a x86_64 -d "AMI description goes here" -k http://ftp.netbsd.org/pub/NetBSD/NetBSD-6.0_BETA/amd64/binary/kernel/netbsd-XEN3_DOMU.gz -v "6.0_BETA-or-whatever-version-string-you-want" us-east-1

The process can take anywhere from 10-60 minutes, depending on which region
you choose and how fast your connection is for uploading. After a successful
run, be sure to test your AMI!

1919. By agc on 2012-03-10

don't link with libthreshold any more

run simple tests in place without requiring lib installation

1918. By agc on 2012-03-10

restore correct operation for odd-sized files

1917. By agc on 2012-03-09

+ don't allocate space if we're passed an in-memory array as either
a source or a destination of split/combine

+ fix up the split_vec/combine_vec combinatino so that they both
work a bit better now - still some work needed here, though

+ free the internal s4 structure as part of the combine/combine_vec
operation, rather than doing it in ssss_end(). We know at that stage
that we have allocated it, and the principle of freeing memory in
the same place in the calling stack as it was allocated still applies

+ rip out parts of the initial comment which aren't really accurate
any more - i'm not sure there's that much of the original code left

1916. By agc on 2012-03-08

Update the ssss code:

+ add the ability to encode split shares as 16bit or 24-bit values,
which greatly simplifies input and output, and allows constant-sized
output for given inputs. This also gives us the ability to use larger
primes, although space is wasted with these, and it's not immediately
obvious what advantage would accrue. But it's there, just in case.

+ we advertise being able to do 255 (SSSS_MAX_SHARES) shares - so
complete the inverse table for GF(16) arithmetic modulo P for 256
values

+ use SSSS_MAX_SHARES (255) internally, rather than the previous
internal (and much smaller) limits

+ simplify the I/O in this library - it was a remnant from 1993

+ remove private structs and definitions from the header file - they're
not part of the public interface, and shouldn't be exposed

+ add ssss_split_vec() and ssss_combine_vec() which give input and
oputput to split and combine via struct iovec

+ update the manual page

1915. By skrll on 2012-02-23

Typo

1914. By agc on 2012-02-21

Numerous changes to ssss(1) and libssss(3):

+ don't use the threshold headers any more - just use our own
header

+ "ssss" is the only algorithm supported by ssss(1)

+ add back (seamless) file I/O functionality, in the case that a
memory mapping, or subsequent memory allocation, fails

+ don't store information in the ssss share header which could be used
as part of an attack (specifically the threshold and total number of
shares).

+ size the extra memory required on a much better estimate of the size
needed, rather than a constant size

+ get rid of unneeded files, now that everything has been cleaned up

1913. By agc on 2012-02-20

minor cosmetic changes to make this more readable - no functional change

Branch metadata

Branch format:
Branch format 7
Repository format:
Bazaar repository format 2a (needs bzr 1.16 or later)
This branch contains Public information 
Everyone can see this information.