Background

I built a grid fill utility for Autodesk Maya, after learning that the option was not available by default.

I had recently tried out 3D modeling with Autodesk Maya over at Gnomon Online, and came to appreciate having a few MEL/Python scripts on hand, such as a hard edge script for beveling edges, or a script to generate leaves along the vertices of a surface, in order to make repetitive modeling tasks much easier to complete. After blocking out some cylindrical props in 3D (bottles, table legs, lamps … et cetera), I wanted to find out if a grid fill utility, similar to that found in Blender, was available for use in Maya.

I was previously aware of this document (with credits to Pixar) which provided some modeling tips when working with the Catmull-Clark subdivision surface modifer. As per the document, placing a high valence vertex, or a vertex connected to more than 4 adjacent edges, should be avoided in case a problem occurs in which a circular shape produces “wavy” surfaces with the modifier applied:

Instead, circular shapes and cylinder caps should be modeled using quad-based topology:

… a process which was normally done by hand, as I eventually found that Maya did not have this functionality built in (as of yet). It seemed to me that a 3D artist who needed to (hypothetically) fill a large number of n-gon faces with quads, would have to go with the following options to replicate Blender’s grid fill operator:

1. Find an existing script somewhere on the Internet in order to extend Maya’s capabilities and fill the n-gon faces
2. Find someone, such as a colleague, to write a script to fill the n-gon faces
3. Resort to writing the script themselves

If all else failed, then a 3D artist was to fall back on something such as the Multi-Cut Tool, to manually cut edges into each n-gon face.

Research

I looked for existing grid fill scripts for Maya, and found a Quad Fill Hole script published by Gabriel Nadeau on the ArtStation marketplace. From the demo and documentation, I saw that the script enabled the quad fill operation by default, and provided alternate actions such as the standard fill hole operation - in case the quad fill operation could not be fulfilled.

Additionally, the script provided the option to add an inset around the hole, to support additional operations such as edge loop selection and bevels. If enabled, then the width and direction of the inset could also be adjusted.

Grid Fill in Blender

I tested out the grid fill option in Blender, and found that adjusting the span changed the number of divisions along the grid fill. And, adjusting the offset changed the starting position and angle of the grid fill.

Maya Python API

I bookmarked the official homepage for the Maya API, as well as the following sites:

• Maya: MEL & API by Bryan Ewert https://danielfaust.github.io/ewertb/maya.html

I eventually opened the Script Editor to test out a few key commands in MEL (Maya Embedded Language) and in Python, to list the currently selected object(s) in the scene:

MEL

ls -selection;

Python

from maya import cmds
cmds.ls(selection=True)

With the Maya devkit for Windows, I navigated to the devkit\pythonScripts subdirectory to try out the example scripts included with the devkit. For example, the createNodeUI.py script, which relies on a createNode.ui file in the same directory, opens a PySide dialog when executed, and allows a user to create a named node at the scene origin:

Implementation

I decided to implement the core grid fill functionality first, before introducing a GUI later down the line.

I first created a cylinder primitive with an even number of edges, and completed the grid fill process by hand. I kept Maya’s Script Editor open and took note of the MEL commands printed to the history pane per operation.

Note that the MEL commands listed in the table below should be interpreted as Python-like formatted string literals, and are invalid unless the variable names denoted between { and } are replaced with actual strings:

Operation	MEL	Additional Notes
Click on a polygon edge	select {objNode}.e[{id}];	objNode is the cylinder transform node id is the edge component ID
Double click on edge to select edge loop	select -add {objNode}.e[{id}];	Add to existing selection
Fill hole	polyCloseBorder;
Select Multi-Cut tool	MultiCutTool;
Create new edge	polySplit -ep {e1} 1 -ep {e2} 1	e1 is the first edge component ID e2 is the second edge component ID

From the MEL commands listed in the history pane, I was able to get the documentation for the select, polyCloseBorder, and polySplit Python wrapper functions.

Core Functionality

From testing out the edge loop selection on cylinder primitives, I found that the edge/vertex component indices were returned in ascending order, and happened to be consecutive:

from maya import cmds
sl = cmds.ls(selection=True, flatten=True)
# Result: [objNode.e[0], objNode.e[1]. objNode.e[2] ... ]

v = cmds.polyListComponentConversion(sl, toVertex=True)
cmds.select(v)
slv = cmds.ls(selection=True, flatten=True)
# Result: [objNode.vtx[0], objNode.vtx[1]. objNode.vtx[2] ... ]

Since the edge/vertex indices were consecutive, I assumed that selecting the vertex at the start of the list (lowest index), followed by the vertex in the middle of the list (number of selected edges divided by 2), would translate to the endpoints of an edge that bisected/roughly bisected the inner face:

sledge = cmds.polyEvaluate(sl[0], edgeComponent=True)

i = 0
j = sledge // 2

cmds.polyConnectComponents(f'{objNode}.vtx[{i}]', f'{objNode}.vtx[{j}]')

Adding the Offset

Adding some number, or the offset, to both the starting endpoint i and opposite endpoint j, would rotate the edge by that number. I needed the modulo operator to help resolve to the actual indices, in case the adjusted indices wrapped to the opposite end of the list of selected vertices:

def getIdx(idx, n):
    ret = idx
    if idx < 0:
        ret = (n + idx) % n
    elif idx >= n:
        ret = idx % n
    return ret

i = getIdx(0 + offset, sledge)
j = getIdx((sledge // 2) + offset, sledge)

To create the edges on either side, I saw that there were two ways to get the adjacent endpoints:

1. Increase the starting index i by 1, and decrease the opposite index j by 1
2. Decrease the starting index i by 1, and increase the opposite index j by 1

For example, to implement the first option and to add a parallel edge:

i = getIdx(i + 1, sledge)
j = getIdx(j - 1, sledge)

cmds.polyConnectComponents(f'{objNode}.vtx[{i}]', f'{objNode}.vtx[{j}]')

Determining the Span

I needed to find a stopping point for creating the edges on either side, as adding or subtracting from the starting and opposite indices i and j would eventually lead to an out of bounds error (where the upper bound was equal to the total number of selected edges).

I eventually found that if I wanted a roughly equal number of “rows” and “columns” in the grid, I could set the upper limit for the number of existing edges, or “rows”, to the number of selected edges in the edge loop, divided by 4. This number would correspond to the span value from earlier experiments with the Blender grid fill operator.

Meanwhile, the number of perpendicular edges, or “columns”, would be related to the number of remaining, unconnected vertices. The corner vertices, or the vertices between the “rows” and “columns”, would need to be excluded to maintain the final quad topology of the cylinder cap.

The existing edges would need to be subdivided to provide vertices for future polySplit / polyConnectComponents operations to connect to. In short, the next few steps would need to create new edges/vertices along certain guidelines (labeled in blue), while excluding the corner vertices (labeled in red):

With these limitations in mind, I set the span, as well as the rows (# of rows) and cols (# of columns) to the following:

span = sledge // 4
rows = span if span % 2 == 1 else span - 1
cols = sledge // 2 - rows - 2

Iterating over Rows

I subdivided the parallel edges by the number of “columns”, while iterating over the number of “rows” to connect the starting and opposite vertices. To select the parallel edges, I needed the total number of edges in the polygon mesh at the very beginning of the grid fill operation, to determine the starting index:

rowBeginEdge = cmds.polyEvaluate(sl[0], edge=True)

i = getIdx(i - ((span - 1) // 2), sledge)
j = getIdx(j + ((span - 1) // 2), sledge)
k = rowBeginEdge
ii = i # save begin index

p = 0
while p < rows:
    # create edge
    cmds.polyConnectComponents(f'{objNode}.vtx[{i}]', f'{objNode}.vtx[{j}]')
    # subdivide edge
    cmds.select(f'{objNode}.e[{k}]')
    cmds.polySubdivideEdge(divisions=cols)
    # increment
    p += 1
    if p != rows:
        i = getIdx(i + 1, sledge)
        j = getIdx(j - 1, sledge)
        k += cols + 1

Iterating over Columns

With the intermediate vertices in place, I created the edges perpendicular to the starting edge. For the starting and opposite “column” vertices, I offset the outermost “row” vertices by 2, in order to exclude the corners:

colBeginEdge = cmds.polyEvaluate(sl[0], edge=True)

i = getIdx(i + 2, sledge)
j = getIdx(ii - 2, sledge)
k = rowBeginEdge + 1

q = 0
while q < cols:
    d = k
    # loop from j to i
    orig = j; dest = d
    for _ in range(rows):
        # create edge
        cmds.polyConnectComponents(f'{objNode}.vtx[{orig}]', f'{objNode}.vtx[{dest}]')
        d += cols + 1
        orig = dest; dest = d
    # create the last edge
    cmds.polyConnectComponents(f'{objNode}.vtx[{orig}]', f'{objNode}.vtx[{i}]')
    # increment
    q += 1 
    if q != cols:
        i = getIdx(i + 1, sledge)
        j = getIdx(j - 1, sledge)
        k += 1

Adjust Edge Flow

Finally, I edited the edge flow of the newly created inner edges, to even out the distribution:

Validation

I revisited the beginning of the script in order to add a few safeguards. To exit the script early, given no selection at all:

errStr = "Please select an edge loop."
...
if not sl:
    cmds.error("Nothing selected. " + errStr)

To check that the selected object corresponded to a polygon mesh:

if not cmds.objectType(sl[0], isType="mesh"):
    typeErrStr = f"Type \"{cmds.objectType(sl[0])}\" selected. "
    cmds.error(typeErrStr + errStr)

And to check for an even number of selected edges:

if (sledge % 2 == 1):
    cmds.warning("Please select an even number of edges.")

Adding the Inset

For the inset, I relied on the polyExtrudeEdge command - I matched the Local Translate Z, Offset, and Divisions arguments to those passed to polyExtrudeEdge:

if (doInset):
    cmds.polyExtrudeEdge(*sl, offset=yInset, localTranslateZ=zInset, divisions=divInset)
    sl = cmds.ls(selection=True, flatten=True) # reselect edge loop

Alternate Actions

For the alternate n-gon fill, I simply called the polyCloseBorder command:

cmds.polyCloseBorder()

And if both grid fill and n-gon options are ignored, then the script simply does nothing.

User Interface

Once again, I relied on Qt Designer to setup the user interface.

From the example code in the devkit, I made sure that my ui file and Python file were placed in the same scripts subdirectory, under my project folder. Then, I employed the QUiLoader class to dynamically create the grid fill tool UI at run-time, using the information stored in the ui file:

def initUI(self):

    """Initialize the UI"""

    # load the QT Designer File
    loader = QUiLoader()
    ws = cmds.workspace(q=True, rd=True)
    file = QFile(ws + "/scripts/gridfill.ui")
    file.open(QFile.ReadOnly)

    self.ui = loader.load(file, parentWidget=self)
    file.close()

I also added a shortcut to the grid fill tool to a custom shelf, to access the tool outside of the Script Editor:

Future Work

The default value returned from an ls command (applied to a selected edge loop), is a list of selected edges ordered by component ID ascending. This ordering currently breaks the script when given an edge loop with non-consecutive IDs:

The next version of the grid fill tool should reorder the list of selected edges by adjacency, with possible help from polySelectConstraint and its optional propagate argument to grow an existing selection. Then, it should map the index of each edge in the revised list of edges, to its underlying edge component ID:

mapEdges = dict()

# TODO: get the edges ordered by adjacency, and not by component ID
for idx, val in enumerate(orderedEdges):
    mapEdges[idx] = getComponentId(val)