ML.NET - Dynamically build pipeline chain

#1

Using code taken from this example.

This code work:

var pipeline = mlContext.Transforms.CopyColumns(outputColumnName: "Label", inputColumnName: "FareAmount")
            .Append(mlContext.Transforms.Categorical.OneHotEncoding(outputColumnName: "VendorIdEncoded", inputColumnName: "VendorId"))
            .Append(mlContext.Transforms.Categorical.OneHotEncoding(outputColumnName: "RateCodeEncoded", inputColumnName: "RateCode"))
            .Append(mlContext.Transforms.Categorical.OneHotEncoding(outputColumnName: "PaymentTypeEncoded", inputColumnName: "PaymentType"))
            .Append(mlContext.Transforms.Concatenate("Features", "VendorIdEncoded", "RateCodeEncoded", "PassengerCount", "TripTime", "TripDistance", "PaymentTypeEncoded"))
            .Append(mlContext.Regression.Trainers.FastTree());

But I would like to Append dynamically at runtime like so:

var pipeline = mlContext.Transforms.CopyColumns(outputColumnName: "Label", inputColumnName: "FareAmount");
pipeline.Append(mlContext.Transforms.Categorical.OneHotEncoding(outputColumnName: "VendorIdEncoded", inputColumnName: "VendorId"));
pipeline.Append(mlContext.Transforms.Categorical.OneHotEncoding(outputColumnName: "RateCodeEncoded", inputColumnName: "RateCode"));
pipeline.Append(mlContext.Transforms.Categorical.OneHotEncoding(outputColumnName: "PaymentTypeEncoded", inputColumnName: "PaymentType"));
pipeline.Append(mlContext.Transforms.Concatenate("Features", "VendorIdEncoded", "RateCodeEncoded", "PassengerCount", "TripTime", "TripDistance", "PaymentTypeEncoded"));
pipeline.Append(mlContext.Regression.Trainers.FastTree());

None of the commands after the first have an effect. How can I dynamically chain Appends?

When trying:

var pipeline = mlContext.Transforms.CopyColumns(outputColumnName: "Label", inputColumnName: "FareAmount");
pipeline = pipeline.Append(mlContext.Transforms.Categorical.OneHotEncoding(outputColumnName: "VendorIdEncoded", inputColumnName: "VendorId"));
pipeline = pipeline.Append(mlContext.Transforms.Categorical.OneHotEncoding(outputColumnName: "RateCodeEncoded", inputColumnName: "RateCode"));
pipeline = pipeline.Append(mlContext.Transforms.Categorical.OneHotEncoding(outputColumnName: "PaymentTypeEncoded", inputColumnName: "PaymentType"));
pipeline = pipeline.Append(mlContext.Transforms.Concatenate("Features", "VendorIdEncoded", "RateCodeEncoded", "PassengerCount", "TripTime", "TripDistance", "PaymentTypeEncoded"));
pipeline = pipeline.Append(mlContext.Regression.Trainers.FastTree());

I get “Cannot implicitly convert type ‘Microsoft.ML.Data.EstimatorChain’ to ‘Microsoft.ML.Transforms.ColumnCopyingEstimator’” errors

#2

You’d be better served asking that question on GitHub or StackOverflow.

#3

Thank Paulo. I actually did post on StackOverflow and was able to get it solved
Here is the link if anyone has the same question https://stackoverflow.com/questions/55831053/dynamically-build-pipeline-chain/55831398

1 Like
.NET Foundation Website | Blog | Projects | Code of Conduct