Background

I built a grid fill utility for Autodesk Maya, after learning that the option was not available by default.

I had tried out 3D modeling using Autodesk Maya at Gnomon Online, and came to appreciate having a few MEL/Python scripts on hand, such as a hard edge script for beveling edges, or a script to generate leaves along the vertices of a surface, in order to make repetitive modeling tasks easier to complete. After blocking out some cylindrical props in 3D (bottles, table legs, lamps … et cetera), I wanted to find out if a grid fill function, similar to that found in Blender, was available for use in Maya.

I was previously aware of this document (with credits to Pixar) which provided some modeling tips when working with the Catmull-Clark subdivision surface modifer. The document mentions that placing a high valence vertex, or a vertex connected to more than 4 adjacent edges, should be avoided in case a problem occurs in which “wavy” surfaces are produced with the modifier:

Instead, circular shapes and cylinder caps should be modeled using quad-based topology:

I eventually found that Maya did not have this functionality built in (as of yet). It seemed to me that a 3D artist who needed to split a large number of n-gon faces into quads, would have to go with one of the following options to replicate Blender’s grid fill functionality:

1. Find an existing script to extend Maya’s capabilities and fill the n-gon faces
2. Find a colleague to write the script, or write the script themselves
3. Simply complete the task by hand

Normally, a 3D artist could use the Multi-Cut Tool to manually cut edges into an n-gon face.

Research

I looked for existing grid fill scripts for Maya, and found a Quad Fill Hole script published by Gabriel Nadeau on the ArtStation marketplace. From the demo and documentation, I saw that the script enabled the quad fill operation by default, and provided alternate actions such as the standard fill hole operation (in case the quad fill operation could not be fulfilled).

Additionally, the script provided the option to add an inset around the hole, which made future operations on the outer edge (edge loop selection, beveling) easier to complete. The width and direction of the inset could also be adjusted.

Grid Fill in Blender

I tested out the grid fill option in Blender, and found that adjusting the span changed the number of divisions along the grid fill. Additionally, adjusting the offset changed the starting position and angle of the grid fill:

Maya Python API

I bookmarked the official homepage for the Maya API, as well as the following sites:

• Maya: MEL & API by Bryan Ewert https://danielfaust.github.io/ewertb/maya.html

I opened the Script Editor to test out a few key commands in MEL (Maya Embedded Language) and in Python, to list the currently selected object(s) in the scene:

MEL

ls -selection;

Python

from maya import cmds
cmds.ls(selection=True)

With the Maya devkit for Windows, I navigated to the devkit\pythonScripts subdirectory to try out the example scripts included with the devkit. For example, the createNodeUI.py script, which relies on a createNode.ui file in the same directory, opens a PySide dialog when executed, and allows a user to create a named node at the scene origin:

Implementation

I decided to implement the core grid fill functionality first, before introducing a GUI later down the line.

First, I created a cylinder primitive with an even number of edges and completed the grid fill process by hand. I kept Maya’s Script Editor open and took note of the MEL commands printed to the history pane per operation.

Note that the MEL commands listed in the table below should be interpreted as Python-like format strings (variable names between { and } should be replaced with object names):

Operation	MEL	Additional Notes
Click on a polygon edge	select {objNode}.e[{id}];	objNode is the cylinder transform node id is the edge component ID
Double click on edge to select edge loop	select -add {objNode}.e[{id}];	Add to existing selection
Fill hole	polyCloseBorder;
Select Multi-Cut tool	MultiCutTool;
Create new edge	polySplit -ep {e1} 1 -ep {e2} 1	e1 is the first edge component ID e2 is the second edge component ID

Core Functionality

From testing out the edge loop selection on cylinder primitives, I found that the edge/vertex component indices were returned in ascending order, and happened to be consecutive:

from maya import cmds
sl = cmds.ls(selection=True, flatten=True)
# Result: [objNode.e[0], objNode.e[1]. objNode.e[2] ... ]

v = cmds.polyListComponentConversion(sl, toVertex=True)
cmds.select(v)
slv = cmds.ls(selection=True, flatten=True)
# Result: [objNode.vtx[0], objNode.vtx[1]. objNode.vtx[2] ... ]

Since the edge/vertex indices were consecutive, I assumed that the vertex at the start of the list (lowest index), followed by the vertex in the middle of the list (number of selected edges divided by 2), would make up the endpoints of an edge that bisected/roughly bisected the n-gon face:

sledge = cmds.polyEvaluate(sl[0], edgeComponent=True)

i = 0
j = sledge // 2

cmds.polyConnectComponents(f'{objNode}.vtx[{i}]', f'{objNode}.vtx[{j}]')

Adding the Offset

Adding some number, or the offset, to both the starting endpoint i and opposite endpoint j, would rotate the edge by that number:

def getIdx(idx, n):
    ret = idx
    if idx < 0:
        ret = (n + idx) % n # wrap around
    elif idx >= n:
        ret = idx % n # wrap around
    return ret

i = getIdx(0 + offset, sledge)
j = getIdx((sledge // 2) + offset, sledge)

To create the edges on either side of the bisecting edge, I saw two ways to get the adjacent endpoints:

1. Increase the starting index i by one, and decrease the opposite index j by one
2. Decrease the starting index i by one, and increase the opposite index j by one

i = getIdx(i + 1, sledge)
j = getIdx(j - 1, sledge)

cmds.polyConnectComponents(f'{objNode}.vtx[{i}]', f'{objNode}.vtx[{j}]')

Determining the Span

I needed to find a cutoff point for creating edges on either side of the bisecting edge, as adding/subtracting from the starting and opposite indices i and j would lead to an out of bounds error (where the upper bound was equal to the total number of selected edges).

I found that if I wanted a roughly equal number of rows and columns in the grid, I could set the upper limit for the number of rows to the number of selected edges in the edge loop, divided by four. The same upper limit could also be applied to the columns.

The number of corner vertices, or the vertices between the rows and columns, would need to be subtracted from the number of selected edges to maintain the final quad topology of the cylinder cap.

The existing edges (rows) would be subdivided to provide vertices for future polySplit / polyConnectComponents operations to connect to (columns). In the following image, the new edges (columns) are labeled in blue, and the corner vertices (to exclude) are labeled in red:

With these rules in mind, I set the span, as well as the rows (# of rows) and cols (# of columns) to the following:

span = sledge // 4
rows = span if span % 2 == 1 else span - 1
cols = sledge // 2 - rows - 2

Iterating over Rows

I subdivided the existing edges (rows) by the number of columns using polySubdivideEdge, and looped over the number of rows to connect the starting and opposite vertices using polyConnectComponents:

rowBeginEdge = cmds.polyEvaluate(sl[0], edge=True)

i = getIdx(i - ((span - 1) // 2), sledge)
j = getIdx(j + ((span - 1) // 2), sledge)
k = rowBeginEdge
ii = i # save begin index

p = 0
while p < rows:
    # create edge
    cmds.polyConnectComponents(f'{objNode}.vtx[{i}]', f'{objNode}.vtx[{j}]')
    # subdivide edge
    cmds.select(f'{objNode}.e[{k}]')
    cmds.polySubdivideEdge(divisions=cols)
    # increment
    p += 1
    if p != rows:
        i = getIdx(i + 1, sledge)
        j = getIdx(j - 1, sledge)
        k += cols + 1

Iterating over Columns

With the intermediate vertices in place, I created the edges (columns) perpendicular to the initial bisecting edge. For the starting and opposite column vertices, I offset the outermost row vertices by two to exclude the corners:

colBeginEdge = cmds.polyEvaluate(sl[0], edge=True)

i = getIdx(i + 2, sledge)
j = getIdx(ii - 2, sledge)
k = rowBeginEdge + 1

q = 0
while q < cols:
    d = k
    # loop from j to i
    orig = j; dest = d
    for _ in range(rows):
        # create edge
        cmds.polyConnectComponents(f'{objNode}.vtx[{orig}]', f'{objNode}.vtx[{dest}]')
        d += cols + 1
        orig = dest; dest = d
    # create the last edge
    cmds.polyConnectComponents(f'{objNode}.vtx[{orig}]', f'{objNode}.vtx[{i}]')
    # increment
    q += 1 
    if q != cols:
        i = getIdx(i + 1, sledge)
        j = getIdx(j - 1, sledge)
        k += 1

Adjust Edge Flow

Finally, I edited the edge flow of the newly created edges, to even out the distribution:

Validation

I revisited the start of the script to add a few safeguards. To exit the script early, given no selection at all:

errStr = "Please select an edge loop."
...
if not sl:
    cmds.error("Nothing selected. " + errStr)

To check that the selected object corresponded to a polygon mesh:

if not cmds.objectType(sl[0], isType="mesh"):
    typeErrStr = f"Type \"{cmds.objectType(sl[0])}\" selected. "
    cmds.error(typeErrStr + errStr)

And to check for an even number of selected edges:

if (sledge % 2 == 1):
    cmds.warning("Please select an even number of edges.")

Adding the Inset

For the inset, I relied on the polyExtrudeEdge command - I matched the Local Translate Z, Offset, and Divisions arguments to those passed to polyExtrudeEdge:

if (doInset):
    cmds.polyExtrudeEdge(*sl, offset=yInset, localTranslateZ=zInset, divisions=divInset)
    sl = cmds.ls(selection=True, flatten=True) # reselect edge loop

Alternate Actions

For the alternate n-gon fill, I simply called the polyCloseBorder command:

cmds.polyCloseBorder()

If both grid fill and n-gon options are ignored, then the script simply does not fill the hole.

User Interface

Once again, I relied on Qt Designer to setup the user interface.

From the example code in the devkit, I made sure that my ui file and Python file were placed in the same scripts subdirectory, under my project folder. Then, I employed the QUiLoader class to create the grid fill tool UI at run-time, using the information stored in the ui file:

def initUI(self):

    """Initialize the UI"""

    # load the QT Designer File
    loader = QUiLoader()
    ws = cmds.workspace(q=True, rd=True)
    file = QFile(ws + "/scripts/gridfill.ui")
    file.open(QFile.ReadOnly)

    self.ui = loader.load(file, parentWidget=self)
    file.close()

I also added the grid fill tool to a custom shelf for quick access outside of the Script Editor:

Future Work

The return value from an ls command (selected edge loop) is currently a list of selected edges ordered by component ID ascending. And so, a selected edge loop with non-consecutive IDs will break the script:

A future version of the grid fill tool should reorder the list of selected edges by adjacency, with possible help from polySelectConstraint and its optional propagate argument to grow an existing selection. Then, it should map the index of each edge in the revised list of edges, to its underlying edge component ID:

mapEdges = dict()

# TODO: get the edges ordered by adjacency, and not by component ID
for idx, val in enumerate(orderedEdges):
    mapEdges[idx] = getComponentId(val)