AI Prediction Accuracy Ranking: Where AI Excels and Struggles
2026-03-16
Our site uses a LightGBM machine learning model to predict visibility, but accuracy varies dramatically by site. IOP achieves R²=0.82 (highly accurate), while Yonaguni sits at R²≈0.05 (virtually unpredictable). Let's explore why through data.
0.82
Best R² (IOP)
0.00
Worst R² (Osezaki Bay)
0.824
Global model R²
45
Features
What is R² (coefficient of determination)?
R² ranges from 0 to 1, where 1 means perfect prediction. R²=0.82 means the model explains 82% of visibility variation. R²=0.05 means only 5% — essentially random.
AI Prediction Accuracy by Site
| # | Site | R² | Grade | Model | Obs |
|---|---|---|---|---|---|
| 1 | IOP | 0.820 | Excellent | Per-site | 3,151 |
| 2 | Kannoura | 0.660 | Good | Per-site | 580 |
| 3 | Shirasaki | 0.603 | Good | Global | 420 |
| 4 | Hirasawa | 0.594 | Good | Global | 2,696 |
| 5 | Futone | 0.566 | Good | Global | 780 |
| 6 | Echizen | 0.455 | Fair | Per-site | 2,652 |
| 7 | Kumomi | 0.477 | Fair | Global | 1,980 |
| 8 | Miyakejima | 0.434 | Fair | Per-site | 941 |
| 9 | Futo | 0.430 | Fair | Global | 3,493 |
| 10 | Kushimoto | 0.426 | Fair | Global | 3,168 |
| 11 | Kerama | 0.250 | Poor | Global | 1,533 |
| 12 | Ishigaki | 0.150 | Poor | Global | 1,473 |
| 13 | Yonaguni | 0.050 | Unpredictable | Global | 4,826 |
| 14 | Osezaki Bay | 0.001 | Unpredictable | Global | 1,200 |
R² evaluated on test data. Global = trained on all sites combined; Per-site = trained on individual site data.
Characteristics of Predictable Sites
Open ocean exposure
Sites like IOP and Futone face open ocean, where weather and sea conditions (waves, swell, wind) directly affect visibility. The rain → river runoff → turbidity causal chain is clear.
Clear seasonal patterns
IOP (winter-high, summer-low) and Echizen (summer peak) have regular monthly patterns that AI learns easily. The month_sin/cos features work well here.
Characteristics of Unpredictable Sites
Current-driven visibility changes
Yonaguni and Ishigaki visibility swings with Kuroshio path changes, but current paths can't be predicted from weather data alone. Adding ocean current features is a future goal.
Enclosed bay topography
Osezaki Bay has R²≈0.00. Enclosed bays are insulated from open ocean conditions, dominated by tidal and subtle topographic factors that our current features can't capture.
Consistently high visibility year-round
Sites like Yonaguni (24.5m annual avg) have minimal variation, making small changes hard to predict. Paradoxically, low prediction accuracy can mean consistently clear water — not necessarily a bad thing.
Practical Implications for Divers
Sites where AI forecast is useful
IOP, Hirasawa, Shirasaki, Kannoura — Izu to Kii Peninsula sites. Check our forecast a few days out, like a weather forecast.
Sites where local intel beats AI
Yonaguni, Ishigaki, Kerama, Osezaki Bay. For these, check the local shop's recent posts or SNS. Our AI forecast is just a rough reference.
Future Accuracy Improvements
- Adding ocean current features (Kuroshio path/speed) → expected improvement for Yonaguni, Ishigaki
- Adding tidal data → better predictions for enclosed bay sites
- Real-time satellite data (chlorophyll, turbidity) → baseline improvement for all sites
- Continued data collection → more observations means better model accuracy
About the Data
R² values from LightGBM model evaluation on test data (time-series split). Global model R²=0.824 is the baseline. Per-site models trained on sites with 1,000+ observations. Results as of March 2026.
🌊 Check Visibility Forecasts
View AI-powered 7-day visibility forecasts for 30+ dive sites across Japan.
Open Forecast App →