A typical representation of the block grid system for KIVA-3 is shown in
Fig. 2. This represents a tensor-product grid with single-index cell
numbering. Cell index increases in an x-y-z order sweeping x-y planes
one at a time in increasing z-direction. As we have mentioned, an active
cell is represented by its i4 vertex leading the loops to include ghost
cells if they are not sorted out. Notice that the real physical domain
is surrounded by ghost cells in all directions. The boundary vertices
on the right, back, and top are real vertices, but they represent
cell volumes that are not part of the physical domain. There are flags
(F for cells, FV for vertices) assoicated with each cell or vertex
to indicate if they are real or not. The flags take values of 1.0 or 0.0.
Based on the flag values one can sort out the elements on such a
grid in different groups. Elements with correspond to the
ghost cells on the left, front, and bottom, whereas elements with
correspond to the active cells, elements with
correspond to
ghost cells on the right, back, and top, and elements with
**F>0**
correspond to real vertices as distinguished in Fig. 2. The storage of
array elements according to such sorting in KIVA-3 is illustrated in
Fig. 3.

Since the access pattern for vertices and cells on the boundaries need to be known in advance for communication requirements, one needs to further sort the groups within themselves separating those on the left, right, and so on. One would also have to assume that the boundary shared between neighbor processors has the same grid points on both sides to assure proper communication. The suggested sorting could be either done in the pre-processor or KIVA-3 program itself.

**Figure 2:** Grid Structure as shown in x-y planes.

**Figure 3:** Storage sorted by F and FV to maintain the shortest possible
vector lengths.

Tue Jun 6 10:52:43 EDT 1995