Presentation Type
Oral
Student
Yes
Track
Other
Abstract
Would AI Stocks Estimate Be as Surprised to USDA Stock Reports as Private Market Analysts?
Keywords: Machine Learning, Random Forest, Agricultural Commodities Market, Informational Impact, Efficient Market Hypothesis.
The USDA survey-based Quarterly Grain Stocks reports are the primary source of information regarding the relative supply of U.S. corn, soybeans, and wheat for the last fifty years. Previous research has examined the accuracy of the USDA stock reports and their relevancy to the market, given alternative sources of estimates (e.g., Isengildina-Massa et al., 2021). For example, private industry analysts also estimate expected quarterly grain stock reports before USDA releases their reports. Market information firms such as Bloomberg and Reuters publish a subset of these estimates a few days before the USDA reports.
Previous research has found that when industry analysts have significant differences in stock expectations compared to what the USDA releases for grain stocks, market prices tend to adjust rapidly to what the USDA found in their survey. Many media outlets and previous research attribute the differences in expectations and changes in market prices to a "market surprise."Karali et al. (2020) found compelling evidence that the discrepancy in USDA reports from private analysts' expectations plays a vital role in explaining grain futures price movements on report days.
Market analysts, USDA officials, and researchers have given four reasons for market surprises in the grain stocks reports. First, USDA surveys may fail to account for grain in transit when surveying stocks. Second, many private analysts use standard conversion rates (e.g., average test weight per bushel of reported corn) for products derived from grain inputs to estimate their expected grain stocks after a quarter. However, these conversion rates may vary because of the quality of the grain and be less (more) than what private analysts estimate. Third, errors in estimating what portion of existing stocks is from old or new crop production may cause surprises in the final annual report before a change in the marketing year. For example, USDA asks in their survey how much old crop corn is on hand on September 1st, although some crops taken in by grain wholesalers can be new crops by this date. Fourth, USDA survey-based stock reports contain survey noise. It is still being determined whether market analysts can correctly consider survey noise when reconciling their estimates versus the USDA and smooth future estimates, assuming some portion of the previous report was due to noise and survey error.
What is the primary reason market analysts are frequently surprised by USDA QAS reports? Given the recent surge in grain movement data, available grain quality data, and data on the output of significant demand sources of grain, particularly at a state level, it is possible to use advances in analyzing high dimensional data (e.g., random forest, gradient boosting) to develop an objective artificial intelligent (AI) market analyst. This paper aims to explore additional public data sources related to commodity demand and supply in the corn, wheat, and soybean markets and apply AI techniques to determine whether data analytics improves the prediction of QAS reports released by USDA for corn, soybeans, and wheat. Our primary research objective is to determine if AI can more accurately predict QAS estimates from USDA than the survey of Market analysts that Bloomberg and Reuters have historically provided. Our secondary objective is to attempt to decompose the surprise into by source of surprise.
We will use Random Forest ML model to predict the stock estimate of the three major commodities (Corn, Soybean, and Wheat) with all the publicly available data before the national announcement of the Quarterly Stock Report. We will compare the stock estimate provided by our AI techniques to private market analysts, which have been a critical component of information before the announcement days. Our research findings will also decompose the variables most important for explaining market surprises. Specifically, does the amount of grain in transit, changes in demand due to grain quality, or the mixing of new crops and old crops in stock estimates mainly explain the surprise? Further, our findings may determine if private analysts have problems reconciling noise in previous USDA surveys when making future estimates for future reports.
Start Date
2-7-2023 11:00 AM
End Date
2-7-2023 12:00 PM
Included in
Agribusiness Commons, Agricultural and Resource Economics Commons, Other Economics Commons
Session 7: Would AI Stocks Estimate Be as Surprised to USDA Stock Reports as Private Market Analysts?
Herold Crest 253 C
Would AI Stocks Estimate Be as Surprised to USDA Stock Reports as Private Market Analysts?
Keywords: Machine Learning, Random Forest, Agricultural Commodities Market, Informational Impact, Efficient Market Hypothesis.
The USDA survey-based Quarterly Grain Stocks reports are the primary source of information regarding the relative supply of U.S. corn, soybeans, and wheat for the last fifty years. Previous research has examined the accuracy of the USDA stock reports and their relevancy to the market, given alternative sources of estimates (e.g., Isengildina-Massa et al., 2021). For example, private industry analysts also estimate expected quarterly grain stock reports before USDA releases their reports. Market information firms such as Bloomberg and Reuters publish a subset of these estimates a few days before the USDA reports.
Previous research has found that when industry analysts have significant differences in stock expectations compared to what the USDA releases for grain stocks, market prices tend to adjust rapidly to what the USDA found in their survey. Many media outlets and previous research attribute the differences in expectations and changes in market prices to a "market surprise."Karali et al. (2020) found compelling evidence that the discrepancy in USDA reports from private analysts' expectations plays a vital role in explaining grain futures price movements on report days.
Market analysts, USDA officials, and researchers have given four reasons for market surprises in the grain stocks reports. First, USDA surveys may fail to account for grain in transit when surveying stocks. Second, many private analysts use standard conversion rates (e.g., average test weight per bushel of reported corn) for products derived from grain inputs to estimate their expected grain stocks after a quarter. However, these conversion rates may vary because of the quality of the grain and be less (more) than what private analysts estimate. Third, errors in estimating what portion of existing stocks is from old or new crop production may cause surprises in the final annual report before a change in the marketing year. For example, USDA asks in their survey how much old crop corn is on hand on September 1st, although some crops taken in by grain wholesalers can be new crops by this date. Fourth, USDA survey-based stock reports contain survey noise. It is still being determined whether market analysts can correctly consider survey noise when reconciling their estimates versus the USDA and smooth future estimates, assuming some portion of the previous report was due to noise and survey error.
What is the primary reason market analysts are frequently surprised by USDA QAS reports? Given the recent surge in grain movement data, available grain quality data, and data on the output of significant demand sources of grain, particularly at a state level, it is possible to use advances in analyzing high dimensional data (e.g., random forest, gradient boosting) to develop an objective artificial intelligent (AI) market analyst. This paper aims to explore additional public data sources related to commodity demand and supply in the corn, wheat, and soybean markets and apply AI techniques to determine whether data analytics improves the prediction of QAS reports released by USDA for corn, soybeans, and wheat. Our primary research objective is to determine if AI can more accurately predict QAS estimates from USDA than the survey of Market analysts that Bloomberg and Reuters have historically provided. Our secondary objective is to attempt to decompose the surprise into by source of surprise.
We will use Random Forest ML model to predict the stock estimate of the three major commodities (Corn, Soybean, and Wheat) with all the publicly available data before the national announcement of the Quarterly Stock Report. We will compare the stock estimate provided by our AI techniques to private market analysts, which have been a critical component of information before the announcement days. Our research findings will also decompose the variables most important for explaining market surprises. Specifically, does the amount of grain in transit, changes in demand due to grain quality, or the mixing of new crops and old crops in stock estimates mainly explain the surprise? Further, our findings may determine if private analysts have problems reconciling noise in previous USDA surveys when making future estimates for future reports.