The WeGO-Torch project is in early access state with a few known usage issues:
- WeGO-Torch cannot support RCNN models (with control-flow) because:
- There is dynamic shape issue in such models (shape of the tensors in the model may change during runtime when different images are provided as model's input, such as RCNN models), to deploy them in WeGO. Some modifications must be performed manually to remove this constraint.
- Such models usually accept
Tensor as input type and it's not supported by WeGO's compile API. On the other hand, using
Tensor as input type means the float model itself is batch-sensitive and the quantized models through tracing are different when different batch size are used during torchscript tracing phase. To deploy these models in WeGO:
Tensor, Tensor, ...(when the number of inputs is known) as input type in the original float model.
- The batch size used for inference in WeGO must be the same as the one used in export phase during quantization.
- WeGO-Torch currently only covers a subset of operators that data center DPUs can support, which means WeGO-Torch will dispatch some operators into CPU for execution even if these operators can be supported by data center DPUs.