Merge lp:~maddevelopers/mg5amcnlo/fix_new_gridpack into lp:~maddevelopers/mg5amcnlo/upgrade_pythia_compatibility

[Johan Alwall]

Changed the way gridpacks work: Set granularity to 1 (so randomly select channels only if they should generate less than 1 event), but allowing channels to run down to a single iteration. This removes all old problems with the gridpacks, while giving even faster event generation.

[lp:~maddevelopers/mg5amcnlo/fix_new_gridpack]

353. By Johan Alwall on 2012-02-01

Remove all compilations in gridrun mode, since make might not be implemented on nodes (and compilation is supposed to have been done before)

[Johan Alwall (johan-alwall)]

Hello Olivier,

Can you check that I correctly removed the compilation for gridruns in the latest revision? There should be no compilations during a gridrun, since everything should be compiled in advance.

Thanks,
Johan

[lp:~maddevelopers/mg5amcnlo/fix_new_gridpack]

354. By Johan Alwall on 2012-02-01

Fixed sum_html to make sure it works for all cases of results.dat files.

[Olivier Mattelaer (olivier-mattelaer)]

> Can you check that I correctly removed the compilation for gridruns in the
> latest revision? There should be no compilations during a gridrun, since
> everything should be compiled in advance.

Yes that's fine.

I've looked at the modification off the code, I think this is (almost) perfect.
I've only one not trivial comment and two quite stupid one.
I'm going to test it further (run test and make some generation) before the final approval.
But the results that you send by email make me quite confortable.

Stupid comment #1:
in dsample.f you modify the line:
 - 1892 c JA 8/17/2011 allow minimum 3 iterations instead of 5
 + 1667c JA 8/17/2011 allow minimum itmin iterations instead of 5
it should be better to update the date. (I said that the comment was stupid)

a second in the same style:
in dsample.f at line 1717
c Calculate chi2 for last three events (ja 03/11)
three should be replace by itsum

a less stupid one
in madevent_interface.py, you add:
3816 try:
3817 os.remove(pjoin(self.me_dir,'RunWeb'))
3818 except:
3819 pass

I don't agree with that, those lines should be added to the file gridrun and not here.
(I added those lines to that file in the 1.4.0 version in fact)

[Olivier Mattelaer (olivier-mattelaer)]

Ok all tests passes,

but i still have one comment:

 - 86 err_goal = 1.5*nreq ! extra factor to ensure works
 + 86 err_goal = 1.2*nreq ! extra factor to ensure works

are you sure that 1.2 is always enough?
shouldn't we put this ratio as an input in the run_card?
This will allow CMS to play with that for hard scenario.

What do you think?

In any case, I approve it right now.

review: Approve

[lp:~maddevelopers/mg5amcnlo/fix_new_gridpack]

355. By Johan Alwall on 2012-02-03

Improved comments as requested by Olivier

356. By Johan Alwall on 2012-02-03

Removed the RunWeb removal since Olivier said its in the wrong place

[Johan Alwall (johan-alwall)]

Thanks Olivier for your review!

> Stupid comment #1:
> in dsample.f you modify the line:
> - 1892 c JA 8/17/2011 allow minimum 3 iterations instead of 5
> + 1667c JA 8/17/2011 allow minimum itmin iterations instead of 5
> it should be better to update the date. (I said that the comment was stupid)

I have updated the comments to make them clearer and up-to-date.

> a less stupid one
> in madevent_interface.py, you add:
> 3816 try:
> 3817 os.remove(pjoin(self.me_dir,'RunWeb'))
> 3818 except:
> 3819 pass

Ok I removed those lines from madevent_interface.py.

> but i still have one comment:
>
> - 86 err_goal = 1.5*nreq ! extra factor to ensure works
> + 86 err_goal = 1.2*nreq ! extra factor to ensure works
>
> are you sure that 1.2 is always enough?

This is what we use in the regular refine, so there is no reason it wouldn't work here (and it makes a big difference in run time). We'll send this version to Alexis for testing, and if we hear from him that there are problems getting the events, then we can change it. For now, I'd prefer not to add yet another flag in the run_card.

> In any case, I approve it right now.

Thanks!

[Johan Alwall (johan-alwall)]

Since there has been no reaction from Tim or Rikkert, I'll perform the merge to v. 1.4 now. If you guys have comments, let us know before tomorrow and we can make the changes in v. 1.4.

Thanks,
Johan

=== modified file 'Template/Source/dsample.f'
--- Template/Source/dsample.f	2011-08-19 00:06:00 +0000
+++ Template/Source/dsample.f	2012-02-03 01:48:18 +0000
@@ -1,4 +1,4 @@
-      subroutine sample_full(ndim,ncall,itmax,dsig,ninvar,nconfigs)
+      subroutine sample_full(ndim,ncall,itmax,itmin,dsig,ninvar,nconfigs)
 c**************************************************************************
 c     Driver for sample which does complete integration
 c     This is done in double precision, and should be told the
@@ -6,7 +6,8 @@
 c     Arguments:
 c     ndim       Number of dimensions for integral(number or random #'s/point)
 c     ncall      Number of times to evaluate the function/iteration
-c     itmax      Number of iterations
+c     itmax      Max number of iterations
+c     itmin      Min number of iterations
 c     ninvar     Number of invarients to keep grids on (s,t,u, s',t' etc)
 c     nconfigs   Number of different pole configurations 
 c     dsig       Function to be integrated
@@ -16,7 +17,7 @@
 c
 c Arguments
 c
-      integer ndim,ncall,itmax,ninvar,nconfigs
+      integer ndim,ncall,itmax,itmin,ninvar,nconfigs
       external         dsig
       double precision dsig
 c
@@ -24,7 +25,7 @@
 c
       double precision x(maxinvar),wgt,p(4*maxdim/3+14)
       double precision tdem, chi2, dum
-      integer ievent,kevent,nwrite,iter,nun,luntmp
+      integer ievent,kevent,nwrite,iter,nun,luntmp,itsum
       integer jmax,i,j,ipole
       integer itmax_adjust
 c
@@ -51,8 +52,8 @@
       common /to_accuracy/accur
 
       double precision twgt, maxwgt,swgt(maxevents)
-      integer                             lun, nw
-      common/to_unwgt/twgt, maxwgt, swgt, lun, nw
+      integer                             lun, nw, itminx
+      common/to_unwgt/twgt, maxwgt, swgt, lun, nw, itminx
 
       integer nzoom
       double precision  tx(1:3,maxinvar)
@@ -100,6 +101,7 @@
       kevent = 0
       nzoom = 0
       xzoomfact = 1d0
+      itminx = itmin
       if (nsteps .lt. 1) nsteps=1
       nwrite = itmax*ncall/nsteps
 c      open(unit=66,file='.sample_warn',status='unknown')
@@ -140,7 +142,7 @@
                wgt=0d0
             endif
             if (nzoom .le. 0) then
-               call sample_put_point(wgt,x(1),iter,ipole) !Store result
+               call sample_put_point(wgt,x(1),iter,ipole,itmin) !Store result
             else
                nzoom = nzoom -1
                ievent=ievent-1
@@ -165,7 +167,9 @@
          i=i+1
       enddo
       cur_it = i
-      i = cur_it - 3
+c     Use the last 3 iterations or cur_it-1 if cur_it-1 >= itmin but < 3
+      itsum = min(max(itmin,cur_it-1),3)
+      i = cur_it - itsum
       if (i .gt. 0) then
       tmean = 0d0
       tsigma = 0d0
@@ -183,13 +187,14 @@
       nun = neventswritten
 
       chi2 = 0d0
-      do i = cur_it-3,cur_it-1
+      do i = cur_it-itsum,cur_it-1
          chi2 = chi2+(xmean(i)-tmean)**2/xsigma(i)**2
       enddo
       chi2 = chi2/2d0   !Since using only last 3, n-1=2
       write(*,'(a)') '-----------------------------------------------------'
       write(*,'(a)') '---------------------------'
-      write(*,'(a,e12.4)') ' Results Last 3 iters:  Integral = ',tmean
+      write(*,'(a,i3,a,e12.4)') ' Results Last ',itsum,
+     $     ' iters:  Integral = ',tmean
       write(*,'(25x,a,e12.4)') 'Std dev = ',tsigma
       write(*,'(17x,a,f12.4)') 'Chi**2 per DoF. =',chi2
       write(*,'(a)') '-----------------------------------------------------'
@@ -198,14 +203,14 @@
       if (nun .lt. 0) nun=-nun   !Case when wrote maximun number allowed
       if (chi2 .gt. 1) tsigma=tsigma*sqrt(chi2)
       if (icor .eq. 0) then
-         write(66,'(3e12.5,2i9,i5,i9,e10.3)')tmean,tsigma,0.0,kevent,nw,
-     &     cur_it-1,nun, nun/max(tmean,1d-99)
+         write(66,'(3e12.5,2i9,i5,i9,e10.3,e12.5)')tmean,tsigma,0.0,
+     &     kevent, nw, cur_it-1, nun, nun/max(tmean,1d-99), twgt
       else
-         write(66,'(3e12.5,2i9,i5,i9,e10.3)')tmean,0.0,tsigma,kevent,nw,
-     &     cur_it-1,nun, nun/max(tmean,1d-99)
+         write(66,'(3e12.5,2i9,i5,i9,e10.3,e12.5)')tmean,0.0,tsigma,
+     &     kevent, nw, cur_it-1, nun, nun/max(tmean,1d-99), twgt
       endif
 c      do i=1,cur_it-1
-      do i=cur_it-3,cur_it-1
+      do i=cur_it-itsum,cur_it-1
          write(66,'(i4,4e15.5)') i,xmean(i),xsigma(i),xeff(i),xwmax(i)
       enddo
       close(66)
@@ -297,7 +302,7 @@
                wgt=0d0
             endif
             if (nzoom .le. 0) then
-               call sample_put_point(wgt,x(1),iter,ipole) !Store result
+               call sample_put_point(wgt,x(1),iter,ipole,itmin) !Store result
             else
                nzoom = nzoom -1
                ievent=ievent-1
@@ -314,7 +319,9 @@
          i=i+1
       enddo
       cur_it = i
-      i = cur_it - 3
+c     Use the last 3 iterations or cur_it-1 if cur_it-1 >= itmin
+      itsum = min(max(itmin,cur_it-1),3)
+      i = cur_it - itsum
       if (i .gt. 0) then
       tmean = 0d0
       tsigma = 0d0
@@ -334,13 +341,13 @@
       nun = neventswritten
 
       chi2 = 0d0
-      do i = cur_it-3,cur_it-1
+      do i = cur_it-itsum,cur_it-1
          chi2 = chi2+(xmean(i)-tmean)**2/xsigma(i)**2
       enddo
       chi2 = chi2/2d0   !Since using only last 3, n-1=2
       write(*,'(a)') '-----------------------------------------------------'
       write(*,'(a)') '---------------------------'
-      write(*,'(a,e12.4)') ' Results Last 3 iters:  Integral = ',tmean
+      write(*,'(a,i3,a,e12.4)') ' Results Last ',itsum,' iters:  Integral = ',tmean
       write(*,'(25x,a,e12.4)') 'Std dev = ',tsigma
       write(*,'(17x,a,f12.4)') 'Chi**2 per DoF. =',chi2
       write(*,'(a)') '-----------------------------------------------------'
@@ -349,14 +356,14 @@
       if (nun .lt. 0) nun=-nun   !Case when wrote maximun number allowed
       if (chi2 .gt. 1) tsigma=tsigma*sqrt(chi2)
       if (icor .eq. 0) then
-         write(66,'(3e12.5,2i9,i5,i9,e10.3)')tmean,tsigma,0.0,kevent,nw,
-     &     cur_it-1,nun, nun/max(tmean,1d-99)
+         write(66,'(3e12.5,2i9,i5,i9,e10.3,e12.5)')tmean,tsigma,0.0,
+     &     kevent, nw, cur_it-1, nun, nun/max(tmean,1d-99), twgt
       else
-         write(66,'(3e12.5,2i9,i5,i9,e10.3)')tmean,0.0,tsigma,kevent,nw,
-     &     cur_it-1,nun, nun/max(tmean,1d-99)
+         write(66,'(3e12.5,2i9,i5,i9,e10.3,e12.5)')tmean,0.0,tsigma,
+     &     kevent, nw, cur_it-1, nun, nun/max(tmean,1d-99), twgt
       endif
 c      do i=1,cur_it-1
-      do i=cur_it-3,cur_it-1
+      do i=cur_it-itsum,cur_it-1
          write(66,'(i4,4e15.5)') i,xmean(i),xsigma(i),xeff(i),xwmax(i)
       enddo
       close(66)
@@ -371,237 +378,6 @@
 
       end
 
-      subroutine sample_fullx(ndim,ncall,itmax,dsig,ninvar,nconfigs)
-c**************************************************************************
-c     Driver for sample which does complete integration
-c     This is done in double precision, and should be told the
-c     number of possible phasespace choices.
-c     Arguments:
-c     ndim       Number of dimensions for integral(number or random #'s/point)
-c     ncall      Number of times to evaluate the function/iteration
-c     itmax      Number of iterations
-c     ninvar     Number of invarients to keep grids on (s,t,u, s',t' etc)
-c     nconfigs   Number of different pole configurations 
-c     dsig       Function to be integrated
-c**************************************************************************
-      implicit none
-      include 'genps.inc'
-c
-c Arguments
-c
-      integer ndim,ncall,itmax,ninvar,nconfigs
-      external         dsig
-      double precision dsig
-c
-c Local
-c
-      double precision x(maxinvar),wgt,p(4*maxdim/3+14)
-      double precision tdem, chi2
-      integer ievent,kevent,nwrite,iter,nun
-      integer jmax,i,j,ipole
-c
-c     External
-c
-      integer  n_unwgted
-      external n_unwgted
-c
-c Global
-c
-      integer                                      nsteps
-      character*40          result_file,where_file
-      common /sample_status/result_file,where_file,nsteps
-      double precision fx
-      common /to_fx/   fx
-
-      integer           mincfig, maxcfig
-      common/to_configs/mincfig, maxcfig
-
-      double precision     xmean(99),xsigma(99),xwmax(99),xeff(99)
-      common/to_iterations/xmean,    xsigma,    xwmax,    xeff
-
-      double precision    accur
-      common /to_accuracy/accur
-
-      double precision twgt, maxwgt,swgt(maxevents)
-      integer                             lun, nw
-      common/to_unwgt/twgt, maxwgt, swgt, lun, nw
-
-      integer nzoom
-      double precision  tx(1:3,maxinvar)
-      common/to_xpoints/tx, nzoom
-
-      double precision xzoomfact
-      common/to_zoom/  xzoomfact
-
-      double precision tmean, tsigma
-      integer             dim, events, itm, kn, cur_it, invar, configs
-      common /sample_common/
-     .     tmean, tsigma, dim, events, itm, kn, cur_it, invar, configs
-
-      integer              icor
-      common/to_correlated/icor
-c
-c     External
-c
-      logical pass_point
-c
-c     Data
-c
-c      data result_file,where_file,nsteps/'SAMPLE','WHERE.AMI',100/
-c      data accur/-1d0/
-c      data mincfig /1/
-c      data maxcfig /1/
-c      data twgt/-1d0/              !Dont write out events
-c      data lun/27/                 !Unit number for events
-c      data maxwgt/0d0/
-c      data nw/0/                   !Number of events written
-      
-
-c-----
-c Begin Code
-c-----
-      ievent = 0
-      kevent = 0
-      nzoom = 0
-      xzoomfact = 1d0
-      if (nsteps .lt. 1) nsteps=1
-      nwrite = itmax*ncall/nsteps
-c      open(unit=66,file='.sample_warn',status='unknown')
-c      write(66,*) 'Warnings from sample run.',itmax,ncall
-c      close(66)
-      call sample_init(ndim,ncall,itmax,ninvar,nconfigs)
-      call graph_init
-      do i=1,itmax
-         xmean(i)=0d0
-         xsigma(i)=0d0
-      enddo
-c      mincfig=1
-c      maxcfig=nconfigs
-      wgt = 0d0
-c
-c     Main Integration Loop
-c
-      iter = 1
-      do while(iter .le. itmax)
-c
-c     Get integration point
-c
-         call sample_get_config(wgt,iter,ipole)
-         if (iter .le. itmax) then
-            ievent=ievent+1
-            call x_to_f_arg(ndim,ipole,mincfig,maxcfig,ninvar,wgt,x,p)
-            if (pass_point(p)) then
-               xzoomfact = 1d0
-               fx = dsig(p,wgt,0) !Evaluate function
-               if (xzoomfact .gt. 0d0) then
-                  wgt = wgt*fx*xzoomfact
-               else
-                  wgt = -xzoomfact
-               endif
-               if (wgt .gt. 0d0) call graph_point(p,wgt) !Update graphs
-            else
-               fx =0d0
-               wgt=0d0
-            endif
-            if (nzoom .le. 0) then
-               call sample_put_point(wgt,x(1),iter,ipole) !Store result
-            else
-               nzoom = nzoom -1
-               ievent=ievent-1
-            endif
-         endif
-         if (wgt .gt. 0d0) kevent=kevent+1    
-c
-c     Write out progress/histograms
-c
-         if (kevent .ge. nwrite) then
-            nwrite = nwrite+ncall*itmax/nsteps
-            nwrite = min(nwrite,ncall*itmax)
-c            open(unit=22,file=where_file,status='old',
-c     &           access='append',err=99)
-c            write(22,'(2i15)') ievent,kevent
-c            close(22)
-            call graph_store
-         endif
- 99   enddo
-c
-c     All done
-c
-c      open(unit=66,file='.sample_warn',status='old',access='append')      
-c      write(66,*) 'Finished sample',ievent,kevent,wgt
-c      if (wgt .ne. -1) then
-c         do j=1,5
-c            jmax = min(ndim,4*j)
-c            write(66,'(4e19.12)') (x(i),i=(j-1)*4+1,jmax)
-c         enddo
-c      endif
-c      close(66)
-
-      open(unit=66,file='results.dat',status='unknown')
-      i=1
-      do while(xmean(i) .ne. 0 .and. i .lt. cur_it)
-         i=i+1
-      enddo
-      cur_it = i
-      i = cur_it - 3
-      if (i .gt. 0) then
-      tmean = 0d0
-      tsigma = 0d0
-      tdem = 0d0
-      do while (xmean(i) .ne. 0 .and. i .lt. cur_it)
-         tmean = tmean+xmean(i)*xmean(i)**2/xsigma(i)**2
-         tdem = tdem+xmean(i)**2/xsigma(i)**2
-         tsigma = tsigma + xmean(i)**2/ xsigma(i)**2
-         i=i+1
-      enddo
-c      tmean = tmean/dble(i-1)
-c      tsigma= sqrt(tsigma)/dble(i-1)
-      tmean = tmean/tsigma
-c      tsigma= sqrt(tsigma)/dble(3)
-      tsigma= tmean/sqrt(tsigma)
-      nun = n_unwgted()
-
-      chi2 = 0d0
-      do i = cur_it-3,cur_it-1
-         chi2 = chi2+(xmean(i)-tmean)**2/xsigma(i)**2
-      enddo
-      chi2 = chi2/2d0   !Since using only last 3, n-1=2
-c      tsigma = tsigma*sqrt(chi2)
-c      write(*,*) "chi2 / dof=", chi2
-      write(*,'(a)') '-----------------------------------------------------'
-      write(*,'(a)') '---------------------------'
-      write(*,'(a,e12.4)') ' Results Last 3 iters:  Integral = ',tmean
-      write(*,'(25x,a,e12.4)') 'Std dev = ',tsigma
-      write(*,'(17x,a,f12.4)') 'Chi**2 per DoF. =',chi2
-      write(*,'(a)') '-----------------------------------------------------'
-      write(*,'(a)') '---------------------------'
-
-      if (nun .lt. 0) nun=-nun   !Case when wrote maximun number allowed
-      if (chi2 .gt. 1) tsigma=tsigma*sqrt(chi2)
-      if (icor .eq. 0) then
-         write(66,'(3e12.5,2i9,i5,i9,e10.3)')tmean,tsigma,0.0,kevent,nw,
-     &     cur_it-1,nun, nun/max(tmean,1d-99)
-      else
-         write(66,'(3e12.5,2i9,i5,i9,e10.3)')tmean,0.0,tsigma,kevent,nw,
-     &     cur_it-1,nun, nun/max(tmean,1d-99)
-      endif
-c      do i=1,cur_it-1
-      do i=cur_it-3,cur_it-1
-         write(66,'(i4,4e15.5)') i,xmean(i),xsigma(i),xeff(i),xwmax(i)
-      enddo
-      close(66)
-      else
-         open(unit=66,file='results.dat',status='unknown')
-         write(66,'(3e12.5,2i9,i5,i9,e10.3)')0,0,0.0,kevent,nw,
-     &     1,0, 0
-         write(66,'(i4,4e15.5)') 1,0,0,0,0
-         close(66)
-
-      endif
-      
-      end
-
-
       subroutine sample_writehtm()
 c***********************************************************************
 c     Writes out results of run in html format
@@ -1336,10 +1112,10 @@
       endif
       end
 
-      subroutine sample_result(mean, sigma)
+      subroutine sample_result(mean, sigma, itmin)
       implicit none
       double precision mean, sigma
-      integer i,cur_it
+      integer i,cur_it,itmin,itsum
       double precision tsigma,tmean,tsig,tdem
 
       double precision     xmean(99),xsigma(99),xwmax(99),xeff(99)
@@ -1351,7 +1127,9 @@
          i=i+1
       enddo
       cur_it = i
-      i = cur_it - 3
+c     Use the last 3 iterations or cur_it-1 if cur_it-1 >= itmin
+      itsum = min(max(itmin,cur_it-1),3)
+      i = cur_it - itsum
       tmean = 0d0
       tsigma = 0d0
       if (i .gt. 0) then
@@ -1393,7 +1171,7 @@
 c
 c     Local
 c
-      integer i, j, k, knt, non_zero, nun
+      integer i, j, k, knt, non_zero, nun,itsum
       double precision vol,xnmin,xnmax,tot,xdum,tmp1,chi2tmp
       double precision rc, dr, xo, xn, x(maxinvar), dum(ng-1)
       save vol,knt
@@ -1447,8 +1225,8 @@
       common /to_error/reliable
 
       double precision twgt, maxwgt,swgt(maxevents)
-      integer                             lun, nw
-      common/to_unwgt/twgt, maxwgt, swgt, lun, nw
+      integer                             lun, nw, itmin
+      common/to_unwgt/twgt, maxwgt, swgt, lun, nw, itmin
 
 
       real*8             wmax                 !This is redundant
@@ -1889,9 +1667,9 @@
             xdum=dsig(0,0,2)
 c
 c     Add test to see if we have achieved desired accuracy 
-c     JA 8/17/2011 allow minimum 3 iterations instead of 5
+c     Allow minimum itmin iterations instead of 5
 c
-            if (tsigma .gt. 0d0 .and. cur_it .gt. 3 .and. accur .gt. 0d0) then
+            if (tsigma .gt. 0d0 .and. cur_it .gt. itmin .and. accur .gt. 0d0) then
 
                xmean = tmean/tsigma
                xchi2 = (chi2/xmean/xmean-tsigma)/dble(cur_it-2)               
@@ -1930,7 +1708,7 @@
 c
 c New check to see if we need to keep integrating this one or not.
 c
-            if (cur_it .gt. 3 .and. accur .lt. 0d0) then  !Check luminocity
+            if (cur_it .gt. itmin .and. accur .lt. 0d0) then  !Check luminocity
 c
 c             Lets get the actual number instead 
 c             tjs 5/22/2007
@@ -1941,16 +1719,18 @@
                nun = neventswritten
 c               tmp1 = tmean / tsigma
 c               chi2tmp = (chi2/tmp1/tmp1-tsigma)/dble(cur_it-2)
-c     Calculate chi2 for last three events (ja 03/11)
+c     Calculate chi2 for last few iterations (ja 03/11)
                tmeant = 0d0
                tsigmat = 0d0
-               do i=cur_it-3,cur_it-1
+c     Use the last 3 iterations or cur_it-1 if cur_it-1 >= itmin but < 3
+               itsum = min(max(itmin,cur_it-1),3)
+               do i=cur_it-itsum,cur_it-1
                   tmeant = tmeant+ymean(i)*ymean(i)**2/ysigma(i)**2
                   tsigmat = tsigmat + ymean(i)**2/ ysigma(i)**2
                enddo
                tmeant = tmeant/tsigmat
                chi2tmp = 0d0
-               do i = cur_it-3,cur_it-1
+               do i = cur_it-itsum,cur_it-1
                   chi2tmp = chi2tmp+(ymean(i)-tmeant)**2/ysigma(i)**2
                enddo
                chi2tmp = chi2tmp/2d0  !Since using only last 3, n-1=2

=== modified file 'Template/Source/gen_ximprove.f'
--- Template/Source/gen_ximprove.f	2012-01-23 12:33:28 +0000
+++ Template/Source/gen_ximprove.f	2012-02-03 01:48:18 +0000
@@ -19,8 +19,8 @@
 c
 c     global
 c
-      integer max_np
-      common/max_np/max_np
+      integer max_np,min_iter
+      common/max_np/max_np,min_iter
 
 c
 c     local
@@ -62,6 +62,7 @@
      &        ', or number events (>1), max processes per job',
      &        ', and whether to split channels (T/F)'
          read(*,*) err_goal, max_np, split_channels
+         min_iter=3
          parallel = .false.
          if (err_goal .lt. 1) then
             write(*,'(a,f8.2,a)') 'Running for accuracy of ',
@@ -79,11 +80,13 @@
       else
          gen_events=.true.
          split_channels=.false.
+c        Allow all the way down to a single iteration for gridruns
+         min_iter=1
          call get_integer(npara,param,value," gevents "  ,nreq  ,2000   )
-         err_goal = 1.5*nreq ! extra factor to ensure works
+         err_goal = 1.2*nreq ! extra factor to ensure works
          call get_integer(npara,param,value," gseed "  ,iseed  ,4321   )
          call get_integer(npara,param,value," ngran "  ,ngran  , -1)
-         if (ngran.eq.-1) ngran = int(sqrt(real(nreq)))
+         if (ngran.eq.-1) ngran = 1
          write(*,*) "Running on Grid to generate ",nreq," additional events"
          write(*,*) "   with granularity equal to ",ngran
 c
@@ -229,8 +232,8 @@
 c
 c     global
 c
-      integer max_np
-      common/max_np/max_np
+      integer max_np,min_iter
+      common/max_np/max_np,min_iter
 c
 c     Arguments
 c
@@ -329,7 +332,7 @@
          write(26,20) 'rm -f $k'
 c         write(26,20) 'rm -f moffset.dat'
 
-         write(26,'(5x,a,2i8,a)') 'echo "',npoints,max_iter,
+         write(26,'(5x,a,3i8,a)') 'echo "',npoints,max_iter,min_iter,
      $        '" >& input_sg.txt' 
          write(26,'(5x,a,f8.3,a)') 'echo "',max(elimit/ysec,0.001d0),
      $        '" >> input_sg.txt'
@@ -476,8 +479,8 @@
 c
 c     global
 c
-      integer max_np
-      common/max_np/max_np
+      integer max_np,min_iter
+      common/max_np/max_np,min_iter
 c      integer    max_np     !now set in run_config.inc
 c      parameter (max_np = 5)  !number of channels/job
 
@@ -654,7 +657,7 @@
          write(26,20) 'if [[ ! -e ftn25 ]]; then'
 
 
-         write(26,'(9x,a,2i8,a)') 'echo "',npoints,max_iter,
+         write(26,'(9x,a,3i8,a)') 'echo "',npoints,max_iter,min_iter,
      $        '" >& input_sg.txt' 
 c
 c     tjs 8/7/2007-JA 8/17/11 Allow stop when have enough luminocity
@@ -677,7 +680,7 @@
 
          write(26,25) 'rm -f $k'
 
-         write(26,'(9x,a,2i8,a)') 'echo "',npoints,max_iter,
+         write(26,'(9x,a,3i8,a)') 'echo "',npoints,max_iter,min_iter,
      $        '" >& input_sg.txt' 
 c
 c tjs 8/7/2007-JA 8/17/11    Change to request luminocity not accuracy
@@ -740,8 +743,8 @@
 c
 c   global
 c
-      integer max_np
-      common/max_np/max_np
+      integer max_np,min_iter
+      common/max_np/max_np,min_iter
 c
 c     Arguments
 c
@@ -787,7 +790,7 @@
                write(27,*) xtot*ngran/xsec(i)/goal_lum
             endif
             npoints = goal_lum * xsec(i) / xtot
-            if (npoints .lt. min_gevents_wu) npoints = min_gevents_wu
+            if (npoints .lt. ngran) npoints = ngran
             np = np+1
             if (np .gt. max_np) then
                if (fopened) then
@@ -822,8 +825,8 @@
             write(26,20) 'if [[ ! -e ftn25 ]]; then'
 
 
-            write(26,'(9x,a,2i8,a)') 'echo "',max(npoints,min_events),
-     $           max_iter,'" >& input_sg.txt' 
+            write(26,'(9x,a,3i8,a)') 'echo "',max(npoints,min_events),
+     $           max_iter,min_iter,'" >& input_sg.txt' 
 c
 c     tjs 8/7/2007  Allow stop when have enough events
 c
@@ -845,8 +848,8 @@
 
             write(26,25) 'rm -f $k'
             
-            write(26,'(9x,a,2i8,a)') 'echo "',max(npoints,min_events),
-     $           max_iter,'" >& input_sg.txt' 
+            write(26,'(9x,a,3i8,a)') 'echo "',max(npoints,min_events),
+     $           max_iter,min_iter,'" >& input_sg.txt' 
 c
 c tjs 8/7/2007    Change to request events not accuracy
 c

=== modified file 'Template/SubProcesses/unwgt.f'
--- Template/SubProcesses/unwgt.f	2011-10-03 08:39:00 +0000
+++ Template/SubProcesses/unwgt.f	2012-02-03 01:48:18 +0000
@@ -27,8 +27,8 @@
 C     GLOBAL
 C
       double precision twgt, maxwgt,swgt(maxevents)
-      integer                             lun, nw
-      common/to_unwgt/twgt, maxwgt, swgt, lun, nw
+      integer                             lun, nw, itmin
+      common/to_unwgt/twgt, maxwgt, swgt, lun, nw, itmin
       integer nzoom
       double precision  tx(1:3,maxinvar)
       common/to_xpoints/tx, nzoom
@@ -126,8 +126,8 @@
 C     GLOBAL
 C
       double precision twgt, maxwgt,swgt(maxevents)
-      integer                             lun, nw
-      common/to_unwgt/twgt, maxwgt, swgt, lun, nw
+      integer                             lun, nw, itmin
+      common/to_unwgt/twgt, maxwgt, swgt, lun, nw, itmin
 
 c-----
 c  Begin Code
@@ -163,8 +163,8 @@
 C     GLOBAL
 C
       double precision twgt, maxwgt,swgt(maxevents)
-      integer                             lun, nw
-      common/to_unwgt/twgt, maxwgt, swgt, lun, nw
+      integer                             lun, nw, itmin
+      common/to_unwgt/twgt, maxwgt, swgt, lun, nw, itmin
 
       double precision    matrix
       common/to_maxmatrix/matrix
@@ -245,8 +245,8 @@
 C     GLOBAL
 C
       double precision twgt, maxwgt,swgt(maxevents)
-      integer                             lun, nw
-      common/to_unwgt/twgt, maxwgt, swgt, lun, nw
+      integer                             lun, nw, itmin
+      common/to_unwgt/twgt, maxwgt, swgt, lun, nw, itmin
 
       integer                   neventswritten
       common /to_eventswritten/ neventswritten
@@ -269,7 +269,7 @@
 c     First scale all of the events to the total cross section
 c
       if (nw .le. 0) return
-      call sample_result(xsec,xerr)
+      call sample_result(xsec,xerr,itmin)
       if (xsec .le. 0) return   !Fix by TS 12/3/2010
       xtot=0
       call dsort(nw, swgt)
@@ -403,8 +403,8 @@
 C     GLOBAL
 C
       double precision twgt, maxwgt,swgt(maxevents)
-      integer                             lun, nw
-      common/to_unwgt/twgt, maxwgt, swgt, lun, nw
+      integer                             lun, nw, itmin
+      common/to_unwgt/twgt, maxwgt, swgt, lun, nw, itmin
 
       integer              IPROC 
       DOUBLE PRECISION PD(0:MAXPROC)
@@ -596,8 +596,8 @@
 C     GLOBAL
 C
       double precision twgt, maxwgt,swgt(maxevents)
-      integer                             lun, nw
-      common/to_unwgt/twgt, maxwgt, swgt, lun, nw
+      integer                             lun, nw, itmin
+      common/to_unwgt/twgt, maxwgt, swgt, lun, nw, itmin
 c-----
 c  Begin Code
 c-----

=== modified file 'UpdateNotes.txt'
--- UpdateNotes.txt	2012-02-01 19:52:37 +0000
+++ UpdateNotes.txt	2012-02-03 01:48:18 +0000
@@ -1,6 +1,6 @@
 Update notes for MadGraph 5 (in reverse time order)
 
-1.4.b10 (19/01/12): OM: New Interface to controll the madevent run.
+1.4.b12 (31/01/12): OM: New Interface to controll the madevent run.
                        This one is accessible by 
                         1) (from madevent output) ./bin/madevent
                         2) (from MG5 command line) launch [MADEVENT_PATH] -i
@@ -82,7 +82,13 @@
                   OM: Improve the parralel suite and change the release script to run
                       some of the parralel suite. This ensures more stability of the 
                       code for the future release.
-                  
+                  JA: Changed the way gridpacks work: Set granularity
+                       to 1 (so randomly select channels only if they
+                       should generate less than 1 event), but
+                       allowing channels to run down to a single
+                       iteration. This removes all old problems with
+                       the gridpacks, while giving even faster event
+                       generation.
 
                   Thanks to Johan Alwall, Sho Iwamoto for all the important 
                   testing/bug reports.

=== modified file 'madgraph/VERSION'
--- madgraph/VERSION	2012-02-01 19:53:45 +0000
+++ madgraph/VERSION	2012-02-03 01:48:18 +0000
@@ -1,3 +1,3 @@
-version = 1.4.0.beta_11
-date = 2012-01-30
+version = 1.4.0.beta_12
+date = 2012-01-31
 

=== modified file 'madgraph/interface/madevent_interface.py'
--- madgraph/interface/madevent_interface.py	2012-02-02 22:07:18 +0000
+++ madgraph/interface/madevent_interface.py	2012-02-03 01:48:18 +0000
@@ -3795,6 +3795,7 @@
         
         MadEventCmd.__init__(self, me_dir, *completekey, **stdin)
         self.run_mode = 0
+        self.random = seed
         self.configuration['automatic_html_opening'] = False
         # Now it's time to run!
         if me_dir and nb_event and seed:
@@ -3827,17 +3828,14 @@
         self.exec_cmd('store_events')
         self.exec_cmd('pythia --no_default -f')
 
-
-
-
     def refine4grid(self, nb_event):
-        """Advanced commands: launch survey for the current process """
+        """Special refine for gridpack run."""
         self.nb_refine += 1
         
         precision = nb_event
 
         # initialize / remove lhapdf mode
-        self.configure_directory()
+        # self.configure_directory() # All this has been done before
         self.cluster_mode = 0 # force single machine
         
         self.update_status('Refine results to %s' % precision, level=None)
@@ -3858,13 +3856,15 @@
                     os.remove(pjoin(Pdir, match))
             
             devnull = os.open(os.devnull, os.O_RDWR)
+            logfile = pjoin(Pdir, 'gen_ximprove.log')
             proc = subprocess.Popen([pjoin(bindir, 'gen_ximprove')],
                                     stdin=subprocess.PIPE,
+                                    stdout=open(logfile,'w'),
                                     cwd=Pdir)
             proc.communicate('%s 1 F\n' % (precision))
 
             if os.path.exists(pjoin(Pdir, 'ajob1')):
-                misc.compile(['madevent'], cwd=Pdir)
+                # misc.compile(['madevent'], cwd=Pdir) # Done before
                 #
                 os.system("chmod +x %s/ajob*" % Pdir)
                 alljobs = glob.glob(pjoin(Pdir,'ajob*'))

=== modified file 'madgraph/iolibs/template_files/madevent_driver.f'
--- madgraph/iolibs/template_files/madevent_driver.f	2011-11-13 10:33:10 +0000
+++ madgraph/iolibs/template_files/madevent_driver.f	2012-02-03 01:48:18 +0000
@@ -11,12 +11,12 @@
       include 'genps.inc'
       include 'maxconfigs.inc'
       include 'nexternal.inc'
-      INTEGER    ITMAX,   NCALL
+      INTEGER    ITMAX, ITMIN, NCALL
 C
 C     LOCAL
 C
-      integer i,ninvar,nconfigs,j,l,l1,l2,ndim
-      double precision dsig,tot,mean,sigma
+      integer i,ninvar,nconfigs,j,l,l1,l2,ndim,idum
+      double precision dsig,tot,mean,sigma,xdum
       integer npoints,lunsud
       double precision x,y,jac,s1,s2,xmin
       external dsig
@@ -78,8 +78,18 @@
      $     read(buf(l1+1:l2-1),*,err=11) ngroup
  11   print *,'Process in group number ',ngroup
 
+c     Read weight from results.dat if present
+      open (unit=lun+1,file='results.dat',status='unknown',err=13)
+      read (lun+1,'(a130)',err=12,end=12) buf
+      close (lun+1)
+      read(buf,'(3e12.5,2i9,i5,i9,e10.3,e12.5)',err=13) xdum,xdum,xdum,
+     $     idum,idum,idum,idum,xdum,twgt
+      goto 14
+ 12   close (lun+1)
+ 13   twgt = -2d0               !determine wgt after first iteration
+ 14   continue
       lun = 27
-      twgt = -2d0            !determine wgt after first iteration
+
       open(unit=lun,status='scratch')
       nsteps=2
       param_card_name = '%(param_card_name)s'
@@ -126,7 +136,7 @@
 c     Get user input
 c
       write(*,*) "getting user params"
-      call get_user_params(ncall,itmax,mincfig)
+      call get_user_params(ncall,itmax,itmin,mincfig)
       maxcfig=mincfig
       minvar(1,1) = 0              !This tells it to map things invarients
       write(*,*) 'Attempting mappinvarients',nconfigs,nexternal
@@ -144,8 +154,8 @@
             minvar(ndim,j) = ninvar
          endif
       enddo
-      write(*,*) "about to integrate ", ndim,ncall,itmax,ninvar,nconfigs
-      call sample_full(ndim,ncall,itmax,dsig,ninvar,nconfigs)
+      write(*,*) "about to integrate ", ndim,ncall,itmax,itmin,ninvar,nconfigs
+      call sample_full(ndim,ncall,itmax,itmin,dsig,ninvar,nconfigs)
 c
 c     Now write out events to permanent file
 c
@@ -170,7 +180,7 @@
 c     $B$ get_user_params $B$ ! tag for MadWeight
 c     change this routine to read the input in a file
 c
-      subroutine get_user_params(ncall,itmax,iconfig)
+      subroutine get_user_params(ncall,itmax,itmin,iconfig)
 c**********************************************************************
 c     Routine to get user specified parameters for run
 c**********************************************************************
@@ -183,7 +193,7 @@
 c
 c     Arguments
 c
-      integer ncall,itmax,iconfig
+      integer ncall,itmax,itmin,iconfig
 c
 c     Local
 c
@@ -206,9 +216,9 @@
 c-----
 c  Begin Code
 c-----
-      write(*,'(a)') 'Enter number of events and iterations: '
-      read(*,*) ncall,itmax
-      write(*,*) 'Number of events and iterations ',ncall,itmax
+      write(*,'(a)') 'Enter number of events and max and min iterations: '
+      read(*,*) ncall,itmax,itmin
+      write(*,*) 'Number of events and iterations ',ncall,itmax,itmin
       write(*,'(a)') 'Enter desired fractional accuracy: '
       read(*,*) accur
       write(*,*) 'Desired fractional accuracy: ',accur

=== modified file 'madgraph/iolibs/template_files/madevent_run_config.inc'
--- madgraph/iolibs/template_files/madevent_run_config.inc	2011-11-14 09:27:10 +0000
+++ madgraph/iolibs/template_files/madevent_run_config.inc	2012-02-03 01:48:18 +0000
@@ -38,8 +38,3 @@
 c
 	double precision trunc_max
         parameter (trunc_max=0.01)
-c
-c	The following are used for grid type running
-c
-      integer      min_gevents_wu
-      parameter   (min_gevents_wu=200)   !Minumum # unweighted events to generate from channel

=== modified file 'madgraph/iolibs/template_files/madevent_symmetry.f'
--- madgraph/iolibs/template_files/madevent_symmetry.f	2012-01-29 23:45:37 +0000
+++ madgraph/iolibs/template_files/madevent_symmetry.f	2012-02-03 01:48:18 +0000
@@ -127,8 +127,8 @@
 
       open (unit=26, file = 'input_app.txt', status='unknown',
      $     err=99)
-      write(26,*) npoints,itmax,
-     &     '     !Number of events and iterations'      
+      write(26,*) npoints,itmax,3,
+     &     '     !Number of events and max and min iterations'      
       write(26,'(f8.4,a)') acc, '    !Accuracy'
       write(26,*) ' 2       !Grid Adjustment 0=none, 2=adjust'
       write(26,*) ' 1       !Suppress Amplitude 1=yes'

=== modified file 'madgraph/various/sum_html.py'
--- madgraph/various/sum_html.py	2012-01-12 13:07:49 +0000
+++ madgraph/various/sum_html.py	2012-02-03 01:48:18 +0000
@@ -50,7 +50,7 @@
             if i == 1:
                 data = [float(d) for d in line.split()]
                 self.xsec, self.xerru, self.xerrc, self.nevents, self.nw,\
-                         self.maxit, self.nunwgt, self.luminosity = data
+                         self.maxit, self.nunwgt, self.luminosity = data[:8]
                 if self.mfactor > 1:
                     self.luminosity /= self.mfactor
                     #self.ysec_iter.append(self.xsec)