FAQs#
1. Can I use Arthur without using the Python SDK?
Yes! The Arthur platform is API-first. You can use our REST API to onboard models, send predictions, and query metrics and insights.
2. Does Arthur need a copy of my model?
Arthur doesn’t generally need access to your actual model, but only captures the inputs to the model and predictions it makes. This means that you can even use Arthur to monitor models you have no access to, such as models hosted by third-party services.
In order to enable explainability, Arthur does need access to your model.
When enabling this feature,
you will need to provide access to the model’s predict function.
3. What if my data is proprietary? Can I still use Arthur?
Yes! Arthur offers on-premises installation for customers with data security requirements. By integrating Arthur into your business’s on-premises stack, you can be confident that all security requirements are met while still getting the benefits of the computation and analytics Arthur provides.
4. What if I don’t have ground truth labels for my data? Or what if I will have the ground truth labels in the future, but they are not available yet?
You don’t need ground truth labels to log your model’s inferences with Arthur.
If your ground truth labels become available after your model’s inferences, whether seconds later or years later, Arthur can link these new ground truth values to your model’s past predictions, linking the new values by ID to their corresponding inferences already in the Arthur system.
In the meantime, Arthur’s data drift metrics can offer leading indicators of model underperformance to keep you covered if your ground truth labels are delayed or never become available.
5. I got an error using the SDK. What do I do?
If the error message says “an internal exception occurred, please report to Arthur” that means there was a problem
on our side. Please email the Arthur support team at support@arthur.ai to let us know what happened.
Otherwise, the error message should provide helpful instructions for how to resolve the issue. If you don’t find the error message actionable, please let Arthur know so that we can improve it.
6. Do I have to type my credentials in every time I use the SDK?
No! Instead of manually entering them you can specify an ARTHUR_ENDPOINT_URL and ARTHUR_API_KEY
environment variable to be used to create the ArthurAI connection object.
7. What are streaming and batch models?
Streaming and batch models are two model types with different patterns of ingesting data to send to Arthur.
A streaming model processes data as a stream of individual inferences: data is logged with Arthur directly as individual inferences when the data flows into the model.
A batch model processes data as a sequence of grouped inferences, which are usually grouped over time: data is logged with Arthur as a group of inferences as the model processes the batch.
8. Which drift metric should I use?
Population stability index (PSI) is typically a good default drift metric to use.
There are some cases where one wants a drift metric with a certain property, e.g. using a drift metric with the unit nats for interpretability, or using a drift metric bounded between 0 and 1 so that drift values don’t increase arbitrarily for outliers. In these cases, other metrics may be preferable to PSI.
For a review of the data drift metrics Arthur offers and their properties, see the data drift section of our glossary. Furthermore, see our blog post for an overview of data how Arthur automates the choice of thresholding for drift metrics.
9. Do I need to configure my attribute ranges in order to onboard my model?
No - none of Arthur’s model performance metrics or monitoring capabilities are impacted by the range property of an
attribute. So, if you observe the attribute looks off when you are reviewing your model during onboarding, you should feel fine
going ahead and saving your model to the platform.
The range property of an attribute - the min/max values of numerical attributes - is only a measurement taken by
Arthur for making decisions on how to format plots in the online dashboard.