Merge lp:~jontai/openvista-gtm-integration/bug363080 into lp:openvista-gtm-integration

[Jon Tai]

[jeff.apple (jeff-apple)]

Looks good.
The only corner case would be if the newline conversion converts within a string for a global value. I'm guessing that never happens with this data, the underlying tools may not support it anyway, it isn't a bad thing anyway, etc.

review: Approve

[Jon Tai (jontai)]

> Looks good.
> The only corner case would be if the newline conversion converts within a
> string for a global value. I'm guessing that never happens with this data, the
> underlying tools may not support it anyway, it isn't a bad thing anyway, etc.

In ZWR format, a newline within a global is escaped with $C(...). I'm not sure what the other formats do, but if a newline shows up unescaped, it's going to confuse %RI/%GI anyway. So I don't think we're doing anything wrong here.

I'm going to commit.

=== modified file 'packages/rpm/openvista-utils.spec'
--- packages/rpm/openvista-utils.spec	2009-04-18 01:54:22 +0000
+++ packages/rpm/openvista-utils.spec	2009-04-20 08:35:31 +0000
@@ -51,6 +51,7 @@
 install scripts/usr/bin/ovtied %{buildroot}%{_bindir}/
 install scripts/usr/bin/ovbackup %{buildroot}%{_bindir}/
 install scripts/usr/bin/ovpurgejournals %{buildroot}%{_bindir}/
+install scripts/usr/bin/ovimport %{buildroot}%{_bindir}/
 
 install -d %{buildroot}%{_sysconfdir}/rc.d/init.d/
 install scripts/etc/init.d/openvista-databases %{buildroot}%{_sysconfdir}/rc.d/init.d/
@@ -81,6 +82,7 @@
 %{_bindir}/ovtied
 %{_bindir}/ovbackup
 %{_bindir}/ovpurgejournals
+%{_bindir}/ovimport
 %{_sysconfdir}/rc.d/init.d/openvista-databases
 %{_sysconfdir}/rc.d/init.d/openvista
 %{_sysconfdir}/bash_completion.d/openvista
 
=== added file 'scripts/usr/bin/ovimport'
--- scripts/usr/bin/ovimport	1970-01-01 00:00:00 +0000
+++ scripts/usr/bin/ovimport	2009-04-20 08:35:23 +0000
@@ -0,0 +1,274 @@
+#!/bin/bash
+
+# This script imports routines and globals into a new OpenVista instance.
+
+
+# Copyright (C) 2009 Medsphere Systems Corporation
+#
+# This program is free software; you can redistribute it and/or modify it solely
+# under the terms of the GNU Affero General Public License version 3 as published
+# by the Free Software Foundation.
+# 
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Affero General Public License
+# for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses>.
+# 
+# You can contact Medsphere Systems Corporation headquarters at 1917 Palomar 
+# Oaks Way, Suite 200, Carlsbad, CA 92008 or at [e-mail].
+
+
+# common functions
+. /usr/lib/openvista/functions
+
+# parse arguments
+while getopts hqfr:g: option; do
+    case $option in
+        h)
+            echo "Usage: ovimport [-q] [-f] [-r ROUTINES] [-g GLOBALS] INSTANCE_NAME"
+            echo "   or  ovimport -h"
+            echo
+            echo "  -q  quiet; suppress status messages"
+            echo "  -f  force; import routines and globals even if target instance is not empty"
+            echo "  -r  path to routine export file, archive, or directory"
+            echo "  -g  path to global export file or archive"
+            echo "  -h  display this help text"
+            exit 0
+            ;;
+        q)
+            skip_status=true
+            ;;
+        f)
+            force=true
+            ;;
+        r)
+            routines=$OPTARG
+            ;;
+        g)
+            globals=$OPTARG
+            ;;
+    esac
+done
+shift `expr $OPTIND - 1`
+
+: ${skip_status:=false}
+: ${force:=false}
+real_routines=$routines
+real_globals=$globals
+
+# check command syntax
+if [ -z "$1" ]; then
+    echo "ovimport: missing operand" >&2
+    echo "Try 'ovimport -h' for more information" >&2
+    exit 2
+fi
+
+# check permissions
+in_groups || {
+    echo "ovimport: permission denied" >&2 
+    exit 1
+}
+
+# check for for valid arguments (requires correct permissions to do; must happen after permission checks)
+set_gtm_env "$1" || {
+    echo "ovimport: $instance: Not an OpenVista instance" >&2
+    exit 3
+}
+
+if [ -z "$routines" ] && [ -z "$globals" ]; then
+    echo "ovimport: No routines or globals specified; please use -r ROUTINES, -g GLOBALS, or both" >&2
+    exit 3
+fi
+
+if [ "0x0000000000000001" != `dse dump -fileheader 2>&1 | grep -F 'Current transaction' | awk '{ print $3 }'` ]; then
+    $force || {
+        echo "ovimport: $instance: OpenVista instance is not new; use -f to force"
+        exit 1
+    }
+fi
+
+if [ `find "$root/$instance/routines" -type f -name '*.m' | wc -l` -ne 0 ]; then
+    $force || {
+        echo "ovimport: $instance: OpenVista instance is not new; use -f to force"
+        exit 1
+    }
+fi
+
+# we can't use a subshell to isolate the umask call because $tempdir wouldn't
+# survive, so save/restore the umask instead
+old_umask=`umask`
+umask 077
+tempdir=`mktemp -d -p "$root/$instance/tmp" .ovimport.XXXXXXXXXX` || 
+exit_with_error "ovimport" "Unable to create temporary directory"
+umask "$old_umask"
+        
+# the EXIT handler may or may not run if we're interrupted or killed depending
+# on the shell used (from what I've read, it does run on bash, but not on
+# dash); use $exit to exit with the correct status in both cases
+trap '[ -z "$exit" ] && rm -rf "$tempdir"; exit ${exit:-0}' EXIT
+
+trap 'print_status "ovimport" "Caught SIGINT, exiting" "$skip_status"; rm -rf "$tempdir"; exit ${exit:=130}' INT
+trap 'print_status "ovimport" "Caught SIGTERM, exiting" "$skip_status"; rm -rf "$tempdir"; exit ${exit:=143}' TERM
+
+if [ -n "$routines" ]; then
+    if [ -f "$routines" ]; then
+        # FIXME: handle compressed files
+
+        routines_format="RO"
+
+        if grep -E $'\r$' "$routines" > /dev/null; then
+            routines_convert_newlines=true
+        fi
+    elif [ -d "$routines" ]; then
+        if [ `find "$routines" -type f -name '*.m' | wc -l` -ne 0 ]; then
+            routines_format="M"
+        else
+            echo "ovimport: $routines: Directory does not contain .m files" >&2
+            exit 1
+        fi
+    else
+        echo "ovimport: $routines: No such file or directory" >&2
+        exit 3
+    fi
+fi
+
+if [ -n "$globals" ]; then
+    if [ -f "$globals" ]; then
+        # FIXME: handle compressed files
+
+        if awk '{ if (NR == 3) { print $0; exit 0 } }' "$globals" | grep -F '=' > /dev/null; then
+            globals_format=ZWR
+        else
+            globals_format=GO
+        fi
+
+        if grep -E $'\r$' "$globals" > /dev/null; then
+            globals_convert_newlines=true
+        fi
+    else
+        echo "ovimport: $globals: No such file" >&2
+        exit 3
+    fi
+fi
+
+if [ -n "$routines" ]; then
+    print_status "ovimport" "Starting load of $routines into $instance" "$skip_status"
+
+    print_status "ovimport" "Routines are in $routines_format format" "$skip_status"
+
+    if ${routines_convert_newlines:-false}; then
+        print_status "ovimport" "Converting file to UNIX-style line endings" "$skip_status"
+
+        real_routines=$tempdir/ovimport.ro
+
+        {
+            cp "$routines" "$real_routines" &&
+            perl -pi -we 's/\r\n/\n/' "$real_routines"
+        } || exit_with_error "ovimport" "Unable to convert file to UNIX-style line endings"
+    fi
+
+    case $routines_format in
+        RO)
+            print_status "ovimport" "Loading routines using ^%RI" "$skip_status"
+
+            # convert $real_routines into an absolute path, since %RI may run
+            # with a different cwd than this script
+            dirname=`dirname "$real_routines"`
+            basename=`basename "$real_routines"`
+            real_routines=`cd "$dirname"; pwd`/"$basename"
+
+            # redirect output to a temporary file because we need to both log
+            # the output and inspect it to determine success 
+            (
+                umask 007
+                mumps -run ^%RI > "$tempdir/ovimport.ri" 2>&1 <<EOF
+N
+$real_routines
+$root/$instance/routines/
+EOF
+            )
+
+            cat "$tempdir/ovimport.ri" \
+              | grep -v '^$' \
+              | logger -p user.info -t "ovimport[$$]"
+
+            grep '^Restored [0-9]\{1,\} lines in [0-9]\{1,\} routines' "$tempdir/ovimport.ri" > /dev/null ||
+            exit_with_error "ovimport" "Unable to import routines"
+            ;;
+        M)
+            print_status "ovimport" "Copying routines" "$skip_status"
+
+            # use install to copy routines instead of cp because install allows
+            # setting the permissions at the same time
+            find "$real_routines" -type f -name '*.m' -exec install -m 660 {} "$root/$instance/routines/" \; ||
+            exit_with_error "ovimport" "Unable to import routines"
+            ;;
+    esac
+
+    print_status "ovimport" "Compiling routines" "$skip_status"
+
+    (
+        umask 007
+        cd "$root/$instance/objects" &&
+        find "$root/$instance/routines" -type f -name '*.m' -exec mumps {} \; 2> /dev/null
+    ) || exit_with_error "ovimport" "Unable to compile routines"
+
+    print_status "ovimport" "Finished loading $routines into $instance" "$skip_status"
+fi
+
+if [ -n "$globals" ]; then
+    print_status "ovimport" "Starting load of $globals into $instance" "$skip_status"
+
+    print_status "ovimport" "Globals are in $globals_format format" "$skip_status"
+
+    if ${globals_convert_newlines:-false}; then
+        print_status "ovimport" "Converting file to UNIX-style line endings" "$skip_status"
+
+        real_globals=$tempdir/ovimport.go
+
+        {
+            cp "$globals" "$real_globals" &&
+            perl -pi -we 's/\r\n/\n/' "$real_globals"
+        } || exit_with_error "ovimport" "Unable to convert file to UNIX-style line endings"
+    fi
+
+    # disabling journaling while "mupip load" runs saves about a minute.  this
+    # is pretty safe to do, since we check that the database is empty before we
+    # load, so even if the system crashes and the database is destroyed, we can
+    # just create a new one and run ovimport again.  if -f was given, do not
+    # disable journaling because the database may have other data in it worth
+    # protecting
+    $force || {
+        print_status "ovimport" "Temporarily disabling journaling" "$skip_status"
+        mupip set -journal="OFF" -region DEFAULT 2>&1 \
+          | grep -v '^$' \
+          | logger -p user.info -t "ovimport[$$]"
+    }
+
+    print_status "ovimport" "Loading globals using 'mupip load'" "$skip_status"
+
+    mupip load -format="$globals_format" "$real_globals" 2>&1 \
+      | grep -v '^$' \
+      | logger -p user.info -t "ovimport[$$]"
+    retval=${PIPESTATUS[0]}
+    [ $retval -eq 0 ] || {
+        exit_with_error "ovimport" "Unable to import globals"
+    }
+
+    $force || {
+        print_status "ovimport" "Re-enabling journaling" "$skip_status"
+        mupip set -journal="ON,BEFORE_IMAGES" -region DEFAULT 2>&1 \
+          | grep -v '^$' \
+          | logger -p user.info -t "ovimport[$$]"
+        retval=${PIPESTATUS[0]}
+        [ $retval -eq 0 ] || {
+            exit_with_error "ovimport" "Unable to re-enable journaling"
+        }
+    }
+
+    print_status "ovimport" "Finished loading $globals into $instance" "$skip_status"
+fi
+
 
=== modified file 'scripts/usr/sbin/ovinstanceadd'
--- scripts/usr/sbin/ovinstanceadd	2009-04-18 04:42:59 +0000
+++ scripts/usr/sbin/ovinstanceadd	2009-04-20 05:53:54 +0000
@@ -239,7 +239,7 @@
   | logger -p user.info -t "ovinstanceadd[$$]"
 retval=${PIPESTATUS[0]}
 [ $retval -eq 0 ] || {
-    echo "ovinstanceadd: Unable to enable journaling for region DEFAULT" >&2
+    echo "ovinstanceadd: Unable to enable journaling" >&2
     exit 1
 }
 
 
=== modified file 'scripts/usr/sbin/ovrestore'
--- scripts/usr/sbin/ovrestore	2009-04-18 04:42:59 +0000
+++ scripts/usr/sbin/ovrestore	2009-04-20 08:35:23 +0000
@@ -188,12 +188,14 @@
     rsync -a --delete --exclude 'lost+found' "$tempdir/routines/" "$root/$instance/routines/"
 } || exit_with_error "ovrestore" "Unable to restore routines"
 
+print_status "ovrestore" "Compiling routines" "$skip_status"
+
 (
     umask 007
     find "$root/$instance/objects" -type f -name '*.o' -exec rm -f {} \; &&
     cd "$root/$instance/objects" &&
     find "$root/$instance/routines" -type f -name '*.m' -exec mumps {} \; 2> /dev/null
-) || exit_with_error "ovrestore" "Unable to recompile routines"
+) || exit_with_error "ovrestore" "Unable to compile routines"
 
 # if we're not applying journals, shut off journaling on the old database so
 # that there's a clean break, then move the journal file out of the way