OrionStats Data Analytics Expert
Level III — end-to-end advanced analytical solutions
The Expert certificate is OrionStats' most demanding credential. It confirms that a candidate can independently solve a substantial, unfamiliar business problem end-to-end: choose a methodology, validate data, engineer features, train and correctly evaluate a predictive model (including gradient boosting and time-series forecasting where appropriate), avoid leakage, build a working Streamlit application and an executive dashboard, recommend business action — and defend every decision in a live technical review.
A working application alone is not sufficient. The technical defense is mandatory.
Intended candidate
- Senior analysts and analytics engineers who already design and deliver complete analytical products.
- Candidates should be fluent in advanced SQL, Python (pandas, scikit-learn, XGBoost), statistics, BI modeling and Streamlit.
Examination blueprint
Domain weights of the written examination and the competencies assessed.
Difficulty distribution: level 3: 30% · level 4: 45% · level 5: 25%
Practical assessment
Scenario. Orion Retail's buying team over-stocks slow items and runs out of fast movers. The COO wants a tool that forecasts product-level demand for the next 3 months, flags stock-out risk, and recommends order quantities — and wants to know how much this could improve margin.
Data. retail (orders, order_items, products, inventory_monthly, campaigns, calendar).
You must deliver:
- Problem definition & methodology note: target, granularity, horizon, evaluation metric(s) and why.
- Data validation & EDA with documented checks and business patterns (seasonality, campaigns, ABC/XYZ classes).
- Feature engineering (lags, rolling stats, calendar, campaign flags, price) with explicit leakage controls.
- Models: a naive seasonal baseline, a regression model and an XGBoost model, evaluated with time-based backtesting (rolling origin). Report MAE/MAPE/WAPE by ABC class and compare to baseline.
- Streamlit application that loads the data, lets the user pick a product/category, shows history + forecast with intervals or error bands, stock-out risk and a recommended order quantity with editable service-level assumption. Must run with
streamlit run app.pyand a requirements.txt. - Executive dashboard (Power BI or Tableau) summarising forecast accuracy and expected inventory impact.
- Executive memo (≤ 3 pages): findings, recommended actions, financial impact estimate with assumptions and sensitivity, limitations and monitoring plan.
- Technical defense (live): explain choices, leakage controls, error analysis and what you would do next.
Time allowed: 14 days + scheduled defense.
Scoring rubric
| Criterion | Weight | Excellent | Proficient | Developing | Insufficient |
|---|---|---|---|---|---|
| Methodology & problem framing | 10% | Target, horizon and metric choices justified against business use. | Sound with minor gaps. | Partly inappropriate. | Absent. |
| Data validation & EDA | 10% | Thorough checks, reconciled totals, insightful patterns. | Adequate. | Superficial. | Absent. |
| Feature engineering & leakage controlCRITICAL | 15% | Rich, justified features; no leakage; explicitly tested. | Good features, no leakage. | Weak features or minor leakage risk. | Leakage present. |
| Modeling & evaluationCRITICAL | 20% | Correct backtesting, baseline comparison, error by segment, honest limitations. | Correct evaluation, fewer insights. | Questionable evaluation. | Invalid evaluation. |
| Streamlit applicationCRITICAL | 15% | Runs cleanly, well structured, error handling, useful interactivity. | Runs with minor issues. | Runs partially. | Does not run. |
| Executive dashboard | 10% | Decision-focused and accurate. | Clear. | Cluttered or unclear. | Missing. |
| Business reasoning & memo | 10% | Quantified impact with assumptions and sensitivity; actionable. | Reasonable. | Generic. | Missing. |
| Technical defenseCRITICAL | 10% | Confident, accurate, self-critical. | Mostly accurate. | Gaps in understanding. | Cannot explain own work. |
Levels score 100 / 75 / 40 / 0% of each criterion's weight. Critical criteria must be at least “Proficient”, whatever the total.
Passing rules
- Written exam: at least 75% overall and at least 60% in every blueprint domain.
- Practical: at least 75% on the rubric with every critical criterion at least “Proficient”.
- Live technical defense judged satisfactory by the examiners.
- Critical failure conditions: Data leakage that invalidates evaluation; Evaluation on training data or wrong validation scheme for time series; Application does not run; Candidate cannot explain or defend their own code/model in the defense; Plagiarism or undisclosed assistance.
- Examiner review: Two examiners review the practical; Live 45-minute technical defense is required; Issuance requires explicit examiner approval.
- Prerequisites: none (Level I is recommended preparation, not required).
- Retakes: up to 2 candidacies per year; 60 days after an unsuccessful decision.
- Validity: 36 months, then renewal.
Preparation pathway
- Study the modules listed in the blueprint and complete their module exams.
- Complete related projects in the Project Lab.
- Take the timed mock exam until you are consistently above the pass mark in every domain.
- Open a candidacy, sit the official written exam, and submit the practical.