Extracting data from numpy array

Question

I have two numpy array data

   grid_code=[1,5,8,7,8,3,40,20....]
   data= Gridcode    X     Y    LINKCODE
            1        ..    ..    0
            1        ..    ..    0
            8        ..    ..    100
            8        ..    ..    100
            10       ..    ..    200
            10       ..    ..    200
            8        ..    ..    111
            8        ..    ..    111

I have write code like this

  for i in grid_code:
     new_list=numpy.where(data[:,0]==i)[0]
     mask_list=data[new_list,:]

I want the output like this in separate file :

    Grid code    x    Y
       1        ..   ..
       1        ..   ..

    Grid code     X      Y
       5          ..     ..
       5          ..     ..

    Grid Code     X      Y        ** This is for the one unique link code (eg.for this time 100)
       8          ..     ..
       8          ..     ..

    Grid Code      X      Y        
       7          ..     ..
       7          ..     ..

    Grid Code      X      Y        ** this is for other link code ( this time 111)
       8          ..     ..
       8          ..     ..

I need to extract data according to the grid code. I face problem when grid code is repeated. When grid code is repeated I need to extract according to the First LINKCODE in the array.

I have tried like this now with this sample data

grid_code=np.array([6,1,9,6,1,6])

data=np.array([[1,50,40,100],
[1,40,20,100],
[6,50,40,5],
[6,50,20,5],
[9,60,90,10],
[9,90,00,10],
[6,100,100,101],
[6,50,90,101],
[6,101,10,101],
[1,11,11,11],
[1,10,10,11],
[6,200,200,102],
[6,200,200,102]])

new=[]
unique=[]
for i in grid_code:
  new_list=numpy.where(data[:,0]==i)[0]
  mask_list=data[new_list,:]
  unique_mask=numpy.unique(mask_list[:,3]).tolist()
  if len(unique_mask)>1:
      unique.append([i])
      new_unique=np.array(unique)
      nq=new_unique[np.where(new_unique[:,0]==i)[0],:]
      if len(nq)>=1:
          b=len(nq)
          a=b-1
          for j in range(a,b):
              p_list=np.where(mask_list[:,3]==unique_mask[j])[0]
              n_list=mask_list[p_list,:]
              print n_list

  else:
      print mask_list

Please have a look on this code and suggest if there is any efficient way of getting the same output.

Answer 1

If you're after a quick, clean and possibly not exaggeratedly efficient solution, here it is my proposal

# ev is for everything, let's start with an empty dict
ev = {}

# we identify and label all our stuff
for row in data:

    # unpack a row of the data matrix
    idx1, x, y, idx2 = row

    # the new value for key=idx1 is {} (empty dict) if first time we see
    # idx1, otherwise we keep the old value
    ev[idx1] = ev.get(idx1,{})

    # the new value for key=idx2 of the dictionary that is the value
    # for key=idx1 is the old/default value (a list/the null list)
    # plus a tuple (x,y)
    ev[idx1][idx2] =  ev[idx1].get(idx2,[]).append((x,y))

# we output our carefully labeled collection of x,y data
for idx1 in keys(ev):
    for idx2, xys in ev[idx1]:

        # the choice of the filename is subjective, isn;t it?
        f = open("file%d_%d.out" % (idx1,idx2), "w")

        for x, y in xys:
            f.write("%d %g %g" % (idx1, x, y))

        f.close()