Fanout
From Fmepedia
| Table of contents |
Definition
A fanout is a way within FME to split output data based upon the value of an attribute. The data is divided at the time of writing, rather than within the workspace itself (so like an on-the-fly AttributeFilter).
Above: A fanout. Data is divided up on the basis of an elevation attribute.
There are two different kinds of fanout: feature type fanout and dataset fanout.
Feature Type Fanout
A feature type fanout writes a single dataset but divides the data up into a number of layers/themes/feature types/objects/classes in that dataset.
Above: A feature type fanout. Data is divided up into a number of layers on the basis of an elevation attribute.
Dataset Fanout
A dataset fanout divides the data up and writes a different dataset for each division.
Above: A dataset fanout. Data is divided up into a number of datasets on the basis of an elevation attribute.
Combination Fanouts
There's no problem with setting both a feature type fanout AND a dataset fanout in the same workspace. The dataset fanout will produce a number of output datasets, each of which has a number of layers created by the feature type fanout.
Multi-Attribute Fanouts
Each fanout setting only allows a user to select a single attribute to fan out by. However, multi-attribute fanouts can be created by merging attributes together using the Concatenator transformer.
- For more information see an example of this technique.
File vs Folder Datasets
The behaviour of fanouts changes based on whether you are working with a file-based format or a directory-based format. For instance, a DWG writer (file-based) will create new layers with a feature type fanout, and new drawings with a dataset fanout. A SHAPE writer (directory-based) will create new shapefiles on a feature-type fanout, and new directories on a dataset fanout.
Example
The difference between a dataset and feature type fanout is best explained through an example.
Given:
- A point dataset (e.g. trees in a city)
- A polygon dataset (e.g. a division of a city into neighbourhoods)
The challenge is to produce a DXF file of the points, grouped into layers per neighbourhood.
The polygons have an attribute containing the neighbourhood code. After doing a simple PointOnAreaOverlayer this attribute has been transferred to the points. On the output feature type, we've set the fanout based upon this attribute. This will group the points by neighbourhood and create a separate layer for each group. This is a feature type fanout.
A second challenge is to produce one DXF file per neighbourhood. After disabling the feature type fanout, we move to the dataset (in the workspace panel) and again use the neighbourhood attribute. This then generates a separate DXF file per neighbourhood. This is a dataset fanout.
Batch Processing
When combined with input aggregation features (swizzlers and merge filters) you can often get away with doing all of your translations in a single workspace, in a sort of pseudo-batch process.
Above: "Batch" processing with fanouts. For each dataset that enters the workspace, a similarly named dataset is output.
- Mark will tell you to use Batch Deploy, but personally, I think it makes it easier to manage if I can just run the workspace.
- Easier to manage perhaps - but a Dataset fanout carries overheads in terms of data being written to disk. Because there is no guarantee that data will arrive in the correct Dataset order FME has to cache it ALL to disk, then separate it out. Will take much longer than batch deploy where we can be sure the data arrives in the correct order for writing.
