How does the Action Score work and What is the performance?

What you'll learn

• Theoretical 2016 backtested results and model portfolio results
• Top 20 performance results for full universe
• Top 20 performance results using filtered universe
• How the Action Score uses QVG metrics

2016 was the first year our Old School Value Action Score that you see on the stock analysis software took to the streets.

What is an "Action Score"?

In a nutshell, it's a set of 9 (mostly value based) metrics used to score a stock between 0-100. Stocks are then graded based on their scores.

Stocks receive the following scores and grades.

• > 85 = A grade
• 75 - 84 = B grade
• 65 - 74 = C grade
• 50 - 65 = D grade
• < 50 = F grade

Stocks with the highest scores should receive higher priority and attention, thereby requiring "Action".

Action refers any of the following: reading, thinking, looking up numbers, studying competitors, getting differing opinions, buying or selling.

The #1 goal of the Action Score is to reduce the time spent trying to find ideas.

If you believe in the 80/20 rule, you'll see that you tend to spend 80% of your time looking for stocks. Less (much less) than 20% of the time is spent on analyzing, thinking, valuing and learning about companies.

The #2 goal of the Action Score is to utilize a mix of uncommon metrics and fundamentals to create a quantitative strategy for people who are short on time, yet enjoy the DIY investing approach.

As it is too long to cover everything in detail again, here are the links you should visit to get a better understanding of how the system works.

• Quality rating score creation, results and discussion
• Value rating score creation, results and discussion
• Growth rating score creation, results and discussion

What You Must Know About Backtests

There are two sets of numbers I'm reporting today. One is based on the backtest. The other is a more "realistic" version.

Here's why.

Previously, when I read academic papers with backtested performances, I took it at face value. Didn't think much about the details.

But spending more than 2 years to build OSV Online and the Action Score opened my eyes to areas you don't think or know about.

The truth is that academic papers and results are just that - academic/theory. The results are the upper range you should ever expect, provided everything goes perfectly.

In the real world, there are variables and challenges that are not considered with theoretical backtests.

• universe of stocks
• volume
• slippage
• fees
• taxes
• emotions
• human meddling

These simple points alone can chop 10% off the theoretical results.

That's why I created a manual version of the portfolio to track.

It was't perfect, but it gives you another look at what type of returns you can expect to receive.

An OSV Member did the same thing and I'll share the list he sent along and how it did.

There's a lot of numbers and tables here so let's get do it. This is the fun part.

(Any question or comments, leave them at the end of the article)

2016 Action Score Performance (Theoretical Backtested Performance)

There are two main methods of calculating returns with backtested performance.

1. Action Score is tested against the entire universe. This includes all sectors and exchanges.
2. Action Score is tested against a filtered universe. No OTC, financials, miners, utilities. No limit on market cap and stock prices.

The first one I call "Full Universe".

The second I call "Filtered Universe".

Each portfolio is backtested by picking the Top 20 Action Score stocks at the start of the year. If there are only 12 stocks with an A grade, the last 8 will be B grade stocks. Doesn't all have to be A's. Just the top 20 highest scores.

• Each stock is held for one year
• 1 year rebalancing
• If a stock is bought out, no new purchase
• Does not consider stock price, volume or market cap
• No slippage of fees included

In 2016, the Action Score for the Full Universe gained 27.97% (price return) compared to the S&P (Total Return) of 11.96%

OSV Ratings Final Performance | Click to Enlarge

Over an 18 year period, it has achieved a CAGR of 29.65%.

Stunning.

... but theoretical.

The 3, 5 and 10 year CAGR is impressive though.

Using the rule of 72, a CAGR of 29% means you'll double your money every 2.5 years.

At 18%, you double your money every 4 years.

Now what does something more "realistic" look like?

That's where the filtered universe comes into play.

No OTC, Financials, Miners, Utilities in Backtest | Click to Enlarge

In 2016, the Action Score for the filtered universe achieved an 8.12% price return.

No slouch, but nearly 20% less when you eliminate OTC, financials, miners and utilities.

Nevertheless, the CAGR over the past 18 years using this filtered universe comes in just shy of 20%.

If you compare the two tables, the individual 2016 Q,V,G performances were close to the market. The difference was that the Full Universe Value stocks soared with a 57.8% return compared to 18.12% for the filtered universe. The full universe had 3 resource companies making up the bulk of the gains as the resource sector bounced back in a huge way.

So far so good. You can see that the results are impressive, but...

Does the OSV Action Score Really Identify the Winners?

This question comes up regularly which I answered this in a post titled fittingly, "Does the OSV Rating System Really Work".

What people mean by this is whether the algorithm properly identifies the A grade stocks with higher stock performance than B,C,D and F stocks.

Good news.

December of 2016, a new OSV Insider with a background and black belt in statistics, asked and wanted to know the same thing.

I could try and do the same tests and provide as much info as possible, but in the end, it's an algorithm I created for which I could be easily biased. Right?

Well, George (OSV Insider) ran exhaustive tests, cross validations and scenarios to verify the Action Score.

I'll be sharing the full report with you later on but here's a snippet that answers this question based on the filtered universe.

Having validated the Action Score and backtested their performance, I wanted to better understand the holistic performance of the Action Scores, rather than focusing only on the subset of annually top 20 ranked Action Score stocks. If the Action Scores represented a fundamental association between superior financial statements and higher annual stock returns, then we should expect to see that as Action Scores increased through the full range from 0-100, the annual stock returns should increase on average as well.

Each gray point is the average annual return for a single selection of 20 stocks going from the lowest 20 Action Score stocks to the highest 20 Action Score stocks. As the variability in the returns were quite large, I applied a moving average (500-unit window) across the Action Scores to smooth out the returns in order to capture the general trend (shown by the blue line). We can then clearly see that the average annual returns gradually increase as Action Scores increase, with the lowest amount of variability (spread of gray points) and highest overall returns in the upper ranges (>80) of the Action Score. In fact, there are very few gray points below the red line (average annual return of 0%) for Action Scores above 80, while in contrast, there are much larger proportions of gray points below the red line for Action Scores below 20.

George took the data for the filtered universe (even found an error with the 2015 results), and ranked all the stocks from all the years and plotted the average annual returns using a sliding window of 20 stocks from the lowest Action Score to the highest 20 Action Score stocks.

This is my simpler version organized by grades for the Top 20 in the Full Universe.

There are lots of numbers and charts so I'm making this spreadsheet of results available for download at the end of this report.

Top 20 Stock Portfolio of each grade - Full Universe

Notice the Top 20 A grade has a CAGR of 29% and drops down to 10% by the time you get to the D's.

Same thing with the filtered universe.

Top 20 Stock Portfolio of each grade - Filtered Universe (No OTC, Financials, Utilities, Miners)

20% CAGR with the A grade stocks.

D grade portfolio returns 11.46%.

Here's an expanded look at the tables to show the number of winners and losers for each grade and the average gain/loss.

Top 20 Full Universe data breakdown for A & B grade Action Score stocks | Click to Enlarge

Top 20 Full Universe data breakdown for C & D grade Action Score stocks | Click to Enlarge

For the filtered universe:

Top 20 Filtered Universe data breakdown for A & B grade Action Score stocks | Click to Enlarge

Top 20 Filtered Universe data breakdown for C & D grade Action Score stocks | Click to Enlarge

2016 Action Score Performance (Manually Selected Portfolio)

As I mentioned near the beginning, I'm going over results for the "theoretical" backtested portfolio and a "realistic" portfolio.

This is the "forward looking" model portfolio to see how it could work in real life.

To create this portfolio, I looked up the Action Scores at the beginning of the year and added 20 stocks that I figured an investor could accumulate to initiate a position.

Again, it wasn't a very scientific. Definitely a quick job in creating each Top 20 model portfolio.

If I didn't have enough stocks to make up 20, I moved down the list and grabbed the next one to fill it up.

Top 20 Action Score Model

• Ended 2016 up 22%.
• OUTR was bought out and was replaced with another stock, but I didn't enter the transaction properly so the gain from OUTR was not included.
• 3 stocks were very thinly traded that could affect the final results.
• there were stocks I didn't want to add like ZAGG and PERI which ultimately came last. But the same thing could have happened with a winner.

2016 Top 20 Action Score Model Portfolio | Click to enlarge

Top 20 Quality Score Model

• +10.86% on the year. This is price return, so total return wise, it's the same as the market.
• Plenty of names that I would never discover on my own.

2016 Top 20 Quality Model | click to enlarge

Top 20 Value Score Model

• The value model under-performed with an end result of 3.52%. About 8% below the market.
• Some stocks like ZAGG show up multiple times. But doesn't mean the stock is a guaranteed winner.

2016 Top 20 Value Score Model Portfolio | click to enlarge

Top 20 Growth Score Model

• Growth beat quality and value in 2016.
• The term I use for growth in the Action Scores is not related to EPS growth. Growth is based on top line growth and asset utilization growth. It's the idea that an increase in efficiency and operations leads to an increase in sales and profits. Therefore we look for gross profitability as researched by Robert Novy-Marx.

2016 Top 20 Growth Score Model | click to enlarge

A Second Opinion

It's good to see OSV Insiders thinking and questioning the methodology.

If you are going to leverage the Action Scores, it's better to test and see how it works.

Kirk is another member who wanted to test out the performance and did a better job of tracking than I did.

Here's the end result of what he tracked.

2016 Action Score Portfolio tracked by OSV Member Kirk | CLick to Enlarge

Kirk kept things real simple and clean and ended with a 16% model portfolio.

• Looks like there weren't enough to fill up 20 stocks. Ended up with 19.
• Only A grade stocks.
• No OTC
• Eliminated tiny market caps.

Download the Performance Spreadsheet

If you made it this far and went over the numbers in detail. Congrats *high five*.

To make it easier, here's the link to download the spreadsheet with all the data and more. You can see it for yourself how certain performance numbers are calculated.

Too Long; Didn't Read Summary

• Backtested performance for our A grade stocks using the full universe returned 28% in 2016
• Backtested performence for our A grade stocks using the filtered universe returned 8.12% in 2016
• Manual model portfolio of A grade stocks returned 22% in 2016
• OSV Member forward looking model of A grade stocks returned 16% in 2016

Overall 2016 was a great start with the launch of OSV Online and the final performance.