For those just joining, Amos is a statistical model created to predict the outcome of each NFL game. Amos uses team level statistics from 2005 to 2017 to learn what wins games, and uses that information to assign likelihood of the home team winning the game.
Amos is not alone the NFL prediction space; there are a number of sources for game predictions. I have identified Microsoft’s Bing Predicts (Bing), ESPN’s Football Power Index (FPI) and Nate Silver’s Elo (Elo) as benchmarks for comparison to Amos this year. The selection is due to the statistical approach to each of these prediction methods, which provides similar, but not exact, grounds for comparison.
Before viewing the model data below, it’s important to first point out the methodology used to track NFL season performance and its limitations. When a model predicts a home team win likelihood above 50%, this is counted as a prediction that the home team will win. Conversely, when a model predicts a home team win likelihood of below 50%, this is counted as a prediction that the away team will win. The models therefore increase performance by correctly predicting whether a home or away team will win and decrease performance by incorrectly predicting whether a home or away team will win. This is called a model’s classification accuracy. While this allows us to track season performance, it does not allow us to credit models that identify ‘close’ games (e.g., predicting a home team win likelihood of 51% and the away team wins) or penalize models for wildly off the mark predictions (e.g., predicting a home team win of 95% and the away team wins). In these instances, classification error may be more appropriate. Nonetheless, just as ‘close’ only counts in horseshoes and hand grenades, NFL teams don’t make it to the Super Bowl by almost winning games. In similar spirit, we’ve chose to use classification accuracy.
The below graphs keep track of three key pieces of information. First, cumulative season performance is tracked. This is represents the number of correct predictions by the number of games played. Second, performance is broken down by week. Similar to cumulative season performance, weekly performance represents the number of correct predictions by the number of games played in a given week. Finally, each models’ weekly predictions are presented. These predictions are each models’ assigned likelihood of a home team win.
Date | Day | Time | Home Team | Away Team | Amos | Elo | Bing | FPI |
---|---|---|---|---|---|---|---|---|
5-Jan | Sat | 4:35 PM | Houston Texans | Indianapolis Colts | 50.1% | 56.0% | 51.0% | 61.3% |
5-Jan | Sat | 8:15 PM | Dallas Cowboys | Seattle Seahawks | 81.0% | 54.0% | 47.0% | 57.3% |
6-Jan | Sun | 1:05 PM | Baltimore Ravens | Los Angeles Chargers | 48.9% | 60.0% | 51.0% | 49.2% |
6-Jan | Sun | 4:40 PM | Chicago Bears | Philadelphia Eagles | 48.9% | 61.0% | 69.0% | 69.1% |
12-Jan | Sat | 4:35 PM | Kansas City Chiefs | Indianapolis Colts | 63.5% | 66.0% | 61.0% | 74.9% |
12-Jan | Sat | 8:15 PM | Los Angeles Rams | Dallas Cowboys | 82.9% | 66.0% | 70.0% | 78.7% |
13-Jan | Sun | 1:05 PM | New England Patriots | Los Angeles Chargers | 65.7% | 58.0% | 61.0% | 65.0% |
13-Jan | Sun | 4:40 PM | New Orleans Saints | Philadelphia Eagles | 81.0% | 64.0% | 70.0% | 80.9% |
20-Jan | Sun | 3:05 PM | New Orleans Saints | Los Angeles Rams | 74.9% | 64.0% | 55.0% | 63.2% |
20-Jan | Sun | 6:40 PM | Kansas City Chiefs | New England Patriots | 71.1% | 61.0% | 61.0% | 66.8% |
3-Feb | Sun | 6:30 PM | Los Angeles Rams | New England Patriots | 71.1% | 47.0% | 45.0% | 52.4% |
Have thoughts on the predictions? See a missing game? Leave a comment below or send us an email at TrevorBischoff@gmail.com
See Amos’ predictions from the 2017 NFL season.
27 Comments
Riff Raff
September 19, 2018 at 11:10 pmWhat happened to the Bing percentages?
jph55
September 20, 2018 at 8:01 amNot sure. When you do “Bing NFL Pick” search now, it just says “Bing predicts XXX will win” with no percentages. Sucks, because I used them in conjunction with ELO and a couple of others to get average rankings for a pool I’m in.
TrevorBischoff
September 20, 2018 at 10:12 amAs jph55 stated, it appears that Bing is not publishing the percentages they previously published. For the time being, I’m labeling a games as 75% chance for a home team win if Bing predicts a home team win and 25% chance for a home team win if Bing predicts an away team win.
I will keep monitoring and update as needed.
Thanks,
Trevor B.
Riff Raff
September 23, 2018 at 9:32 amFoxsports has a good one. You can replace Bing if you want.
https://www.foxsports.com/nfl/predictions?season=2018&seasonType=1&week=3
TrevorBischoff
September 24, 2018 at 11:02 amHi Riff Raff,
This is helpful. I was not aware of Foxsports’ predictions; it looks like it could be a viable candidate to replace Bing should Bing continue to omit game likelihoods. I’ll continue to monitor Bing and contemplate swapping out Bing for Foxsports. It appears that Foxsports allows you to predictions from the beginning of the season so it would be easy to swap out mid-season.
Thanks,
Trevor B.
Frank
September 26, 2018 at 12:18 pmIf you go to bing predictions and click on the game, the pred % shows up in the individual games. It is a pain to click on each game, but you can see the percentage.
Frank K
Frank
September 26, 2018 at 12:20 pmVikings vs Rams
NFL · Week 4 · NFL schedule
Minnesota
Vikings
1-1-1
FOX
Tomorrow, 8:20 PM
Bing predicts LAR wins
Los Angeles
Rams
3-0-0
Partly Sunny, 77°F at kickoff · Los Angeles Memorial Coliseum, Los Angeles · Line: LAR -6.5 · O/U: 49
27% chance MIN wins ‧ 73% LAR wins
TrevorBischoff
September 26, 2018 at 2:12 pmFrank,
Great find, and thanks for this. I had not seen these previously. Do you or anyone else have the predictions for Week 3?
Thanks,
Trevor B.
Frank K
September 26, 2018 at 3:22 pmTrevor, It seems that data is deleted on the web page. I do have the numbers that I copied.
CLE 72
KC 77
Bal 68
Hou 59
Mia 63
Gb 57
Phi 66
Cin 72
Jac 71
Atl 66
Min 79
Lar 75
Sea 55
Ari 55
Ne 64
Pit 57
Hope this helps…
Frank
TrevorBischoff
September 26, 2018 at 7:22 pmFrank,
Thank you for providing. I’ve updated the historicals and updated the current week with the likelihoods.
Something note is that your numbers had Arizona winning, but my notes had Chicago winning. A few other sources also had Chicago winning so I assigned it 55% likelihood of Chicago winning so that Bing would not be penalized. All other numbers aligned.
Again, thanks for providing. I was unable to find anywhere else.
Thanks,
Trevor B.
Jeffrey Venner
September 27, 2018 at 7:24 amRams win 27-23 is my call on this one. I like FPI model lately (on the rise).
TrevorBischoff
September 30, 2018 at 3:11 pmHi Jeff,
Good call on the Rams’ win. FPI takes an early lead in Week 3. I’m interested to see how the models progress and if FPI can keep the edge.
Cheers,
Trevor B.
Jeff
September 20, 2018 at 3:15 pmIt would be interesting to see what factors drive Bing’s model vs yours Trevor on the Cincinnati game, almost total opposites.
TrevorBischoff
September 24, 2018 at 10:58 amHi Jeff,
This is something I’ve been tracking, and I hope to publish some ongoing statistics around model similarity here soon. Since I don’t have the specifics of any of the underlying models, it will be difficult to attribute similarities to any one specific factor and the similarity will only take into account the models’ output. I’m excited to share what I’ve been working on once its ready nonetheless.
Thanks,
Trevor B.
Frank
September 26, 2018 at 9:29 pmSorry, Chi 57 …I had written down Ari 43!
Sound better?
Frank
TrevorBischoff
September 30, 2018 at 3:08 pmHi Frank,
Yes, that fits much better. I’ve updated.
Thanks again,
Trevor B.
TOM HIGGINS
September 27, 2018 at 11:45 amSorry, but what happened to the at-a glance format you were using last year where you listed the prediction percentages by each source? (Unless it’s somewhere else on the site, or a way to convert it and I’m missing it) This cryptic, ‘labor-intensive’ graph format you are using now is anything but user-friendly. Call me lazy but having to figure out each color-coded source, click to check the percentage instead of just being able to read it from a prepared printed page like last year is a bit too involved for people who don’t have the time, patience or concentration level to have to figure this out when all they want is some quick reference data to aid in making informed decisions in determining their weekly picks. I used to use you as my primary go-to best source for this info but now have to rely on others for information. I don’t know that I’m alone in this opinion but also don’t expect you to change (though I wish you would). Straight up readable data is the best TRUST ME!
Frank
September 27, 2018 at 1:28 pmOne thing that I personally like about this chart is that you can see at a glance the scatter for the home team or away. Click on the dot and you can see the percentage and what souce it came from. Takes time getting used to it, and I think it was uses a couple of years ago.
Tom H
September 28, 2018 at 8:29 amSorry, but this format just doesn’t work for me. I liked having all the data in front of me on one page and being able to scroll between the different sources to compare percentages in order to ‘analyze’ a trend as to which teams might be favored. You had the best collection of info for that in the biz. This doesn’t allow that.
Good luck going forward.
TrevorBischoff
September 30, 2018 at 3:18 pmFrank and Tom,
I appreciate the input on the graphs. I’m always intrigued by how different ways of visualizing data resonates with different individuals. While I find the scatter to help quickly determine model agreement/disagreement for each game, I hear you, Tom. To help, I’ve added a simple table underneath the scatter graph.
Enjoy!
Trevor B.
Frank Kammerer
October 4, 2018 at 12:23 amTrevor,
That’s very helpful! I like it!
Frank
TOM H
October 11, 2018 at 5:15 pmTrevor,
Thanks for not only listening, but responding with a nice graphic Personally, to be able to compare different data across the board like this helps a lot. Thanks again!
TH
jph55
October 14, 2018 at 11:29 pmTrevor,
Do you have the tables for weeks 1-5 that you could either post or link to?
Thanks,
Jason
Bobby Norris
November 24, 2018 at 4:31 pmFPI percentage for The Jets/Patriots game is 20%.
TrevorBischoff
November 27, 2018 at 9:00 amHi Bobby,
Thanks for catching. The Jets’ win likelihood has been updated for FPI.
Thanks,
Trevor B.
Tara
December 23, 2018 at 1:23 pmI’m interested in seeing the record each model has this season.
TrevorBischoff
January 16, 2019 at 10:02 amHi Tara,
The record of each model would be helpful and is something I’ll plan on incorporating into next years’ model. In the meantime, I plan on making a dataset available with all information so others can perform their own analysis. I will post something on this blog once that is available.
Thanks,
Trevor B.