JDS Journal of Data Science 1683-86021680-743X1680-743X School of Statistics, Renmin University of China JDS1098 10.6339/23-JDS1098 Computing in Data Science An Assessment of Crop-Specific Land Cover Predictions Using High-Order Markov Chains and Deep Neural Networks✩ https://orcid.org/0000-0002-0446-1328 SartoreLucalsartore@niss.org12∗ BoryanClaire2 DauAndrew2 WillisPatrick2 National Institute of Statistical Science, 1750 K Street NW Suite 1100, 20006 Washington DC, USA United States Department of Agriculture, National Agricultural Statistics Service, 1400 Independence Avenue SW, 20250 Washington DC, USA

The findings and conclusions in this presentation are those of the authors and should not be construed to represent any official USDA or US Government determination or policy. This research was supported in part by the intramural research program of the US Department of Agriculture, National Agriculture Statistics Service.

Corresponding author. Email: lsartore@niss.org. 20233132023212333353 Supplementary Material

Supplementary materials of the paper entitled “An Assessment of Crop-Specific Land Cover Predictions Using High-Order Markov Chains and Deep Neural Networks”

29720222332023 2023 The Author(s). Published by the School of Statistics and the Center for Applied Statistics, Renmin University of China.2023 Open access article under the CC BY license.

High-Order Markov Chains (HOMC) are conventional models, based on transition probabilities, that are used by the United States Department of Agriculture (USDA) National Agricultural Statistics Service (NASS) to study crop-rotation patterns over time. However, HOMCs routinely suffer from sparsity and identifiability issues because the categorical data are represented as indicator (or dummy) variables. In fact, the dimension of the parametric space increases exponentially with the order of HOMCs required for analysis. While parsimonious representations reduce the number of parameters, as has been shown in the literature, they often result in less accurate predictions. Most parsimonious models are trained on big data structures, which can be compressed and efficiently processed using alternative algorithms. Consequently, a thorough evaluation and comparison of the prediction results obtain using a new HOMC algorithm and different types of Deep Neural Networks (DNN) across a range of agricultural conditions is warranted to determine which model is most appropriate for operational crop specific land cover prediction of United States (US) agriculture. In this paper, six neural network models are applied to crop rotation data between 2011 and 2021 from six agriculturally intensive counties, which reflect the range of major crops grown and a variety of crop rotation patterns in the Midwest and southern US. The six counties include: Renville, North Dakota; Perkins, Nebraska; Hale, Texas; Livingston, Illinois; McLean, Illinois; and Shelby, Ohio. Results show the DNN models achieve higher overall prediction accuracy for all counties in 2021. The proposed DNN models allow for the ingestion of long time series data, and robustly achieve higher accuracy values than a new HOMC algorithm considered for predicting crop specific land cover in the US.

 categorical prediction crop rotation patterns deep neural networks transition probability References Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, et al. (2015). TensorFlow: Large-scale machine learning on heterogeneous systems. Software available from tensorflow.org. Ba J, Caruana R (2014). Do deep nets really need to be deep? In: Ghahramani Z, Welling M, Cortes C, Lawrence N, Weinberger KQ (Eds.), Advances in Neural Information Processing Systems volume 27. Barnett S (2009). Quantum Information, volume 16. Oxford University Press. Bernstein E, Vazirani U (1997). Quantum complexity theory. SIAM Journal on Computing, 26(5): 1411–1473. https://doi.org/10.1137/S0097539796300921 Berthiaume A, Brassard G (1994). Oracle quantum computing. Journal of Modern Optics, 41(12): 2521–2535. https://doi.org/10.1080/09500349414552351 Bishop CM, Nasrabadi NM (2006). Pattern Recognition and Machine Learning. Springer. Boryan C, Yang Z, Mueller R, Craig M (2011). Monitoring us agriculture: The US Department of Agriculture, national agricultural statistics service, cropland data layer program. Geocarto International, 26(5): 341–358. https://doi.org/10.1080/10106049.2011.562309 Breiman L (2001). Random forests. Machine Learning, 45(1): 5–32. https://doi.org/10.1023/A:1010933404324 Breiman L, Friedman J, Olshen R, Stone C (1984). Cart. Classification and Regression Trees. Routledge. Broughton M, Verdon G, McCourt T, Martinez AJ, Yoo JH, Isakov SV, et al. (2020). Tensorflow quantum: A software framework for quantum machine learning. arXiv preprint: https://arxiv.org/abs/2003.02989. Buciluǎ C, Caruana R, Niculescu-Mizil A (2006). Model compression. In: Ungar L, Craven M, Eliassi-Rad T (Eds.), Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 535–541. Cho K, Van Merriënboer B, Bahdanau D, Bengio Y (2014). On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint: https://arxiv.org/abs/1409.1259. Chollet F, et al. (2015). Keras. https://keras.io Cirq Developers (2022). Cirq. See full list of authors on Github: https://github.com/quantumlib/Cirq/graphs/contributors. Congalton RG, Green K (2019). Assessing the Accuracy of Remotely Sensed Data: Principles and Practices. CRC Press. Cypher R, Sanz JL (2012). The SIMD Model of Parallel Computation. Springer Science & Business Media. Dagum L, Menon R (1998). OpenMP: An industry standard API for shared-memory programming. IEEE Computational Science & Engineering, 5(1): 46–55. https://doi.org/10.1109/99.660313 Deutsch D (1985). Quantum theory, the Church–Turing principle and the universal quantum computer. Proceedings of the Royal Society of London. Series A, Mathematical and Physical Sciences, 400: 97–117. 1818. Efron B (1979). Bootstrap methods: Another look at the jackknife. The Annals of Statistics, 7: 1–26. ESRI (1998). ESRI shapefile technical description. Technical report, Environmental Systems Research Institute, Inc. Fokianos K, Kedem B (2003). Regression theory for categorical time series. Statistical Science, 18: 357–376. https://doi.org/10.1214/ss/1076102425 Gibney E (2017). D-wave upgrade: How scientists are using the world’s most controversial quantum computer. Nature, 541(7638): 447–448. https://doi.org/10.1038/541447b Grover LK (1996). A fast quantum mechanical algorithm for database search. In: Proceedings of the Twenty-Eighth Annual ACM Symposium on Theory of Computing, 212–219. Gupta MM, Qi J (1991). Theory of t-norms and fuzzy inference methods. Fuzzy Sets and Systems, 40(3): 431–450. https://doi.org/10.1016/0165-0114(91)90171-L Halmy MWA, Gessler PE, Hicke JA, Salem BB (2015). Land use/land cover change detection and prediction in the north-western coastal desert of Egypt using Markov-CA. Applied Geography, 63: 101–112. https://doi.org/10.1016/j.apgeog.2015.06.015 Heald J (2002). Usda establishes a common land unit. ArcUser Online. https://www.esri.com/news/arcuser/0402/usda.html Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR (2012). Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint: https://arxiv.org/abs/1207.0580. Hochreiter S, Schmidhuber J (1997). Long short-term memory. Neural Computation, 9(8): 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735 Hopfield JJ (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences of the United States of America, 79(8): 2554–2558. https://doi.org/10.1073/pnas.79.8.2554 Jacobs PA, Lewis PA (1983). Stationary discrete autoregressive-moving average time series generated by mixtures. Journal of Time Series Analysis, 4(1): 19–36. https://doi.org/10.1111/j.1467-9892.1983.tb00354.x Jeswal S, Chakraverty S (2019). Recent developments and applications in quantum neural network: A review. Archives of Computational Methods in Engineering, 26(4): 793–807. https://doi.org/10.1007/s11831-018-9269-0 Johnson DM, Mueller R (2021). Pre-and within-season crop type classification trained with archival land cover information. Remote Sensing of Environment, 264: 112576. https://doi.org/10.1016/j.rse.2021.112576 Jolliffe I (1986). Principal Component Analysis. Springer. Kahan W (1996). IEEE standard 754 for binary floating-point arithmetic. Lecture Notes on the Status of IEEE, 754(94720-1776): 11. Kingma DP, Ba J (2014). Adam: A method for stochastic optimization. arXiv preprint: https://arxiv.org/abs/1412.6980. Latour A (1998). Existence and stochastic structure of a non-negative integer-valued autoregressive process. Journal of Time Series Analysis, 19(4): 439–455. https://doi.org/10.1111/1467-9892.00102 Li H, Reynolds JF (1997). Modeling effects of spatial pattern, drought, and grazing on rates of rangeland degradation: A combined Markov and cellular automation approach – Chapter 10. In: Goodchild MF, Quattrochi DA (Eds.), Scale in Remote Sensing and GIS, 211–230. Logan JA (1981). A structural model of the higher-order Markov process incorporating reversion effects. The Journal of Mathematical Sociology, 8(1): 75–89. https://doi.org/10.1080/0022250X.1981.9989916 Mahammad SS, Ramakrishnan R (2003). Geotiff – A standard image file format for GIS applications. Map India, 28–31. McCulloch WS, Pitts W (1943). A logical calculus of the ideas immanent in nervous activity. The Bulletin of Mathematical Biophysics, 5(4): 115–133. https://doi.org/10.1007/BF02478259 Menneer T, Narayanan A (1995). Quantum-inspired neural networks, Tech. Rep. R329. Miyashita D, Lee EH, Murmann B (2016). Convolutional neural networks using logarithmic data representation. arXiv preprint: https://arxiv.org/abs/1603.01025. Nair V, Hinton GE (2010). Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning. Osman J, Inglada J, Dejoux JF (2015). Assessment of a Markov logic model of crop rotations for early crop mapping. Computers and Electronics in Agriculture, 113: 234–243. https://doi.org/10.1016/j.compag.2015.02.015 Panchi L, Shiyong L (2008). Learning algorithm and application of quantum BP neural networks based on universal quantum gates. Journal of Systems Engineering and Electronics, 19(1): 167–174. https://doi.org/10.1016/S1004-4132(08)60063-8 Parker DC, Manson SM, Janssen MA, Hoffmann MJ, Deadman P (2003). Multi-agent systems for the simulation of land-use and land-cover change: A review. Annals of the Association of American Geographers, 93(2): 314–337. https://doi.org/10.1111/1467-8306.9302004 Pegram G (1980). An autoregressive model for multilag Markov chains. Journal of Applied Probability, 17(2): 350–362. https://doi.org/10.2307/3213025 Raftery AE (1985). A model for high-order Markov chains. Journal of the Royal Statistical Society, Series B, Methodological, 47(3): 528–539. Ritter N, Ruth M, Grissom BB, Galang G, Haller J, Stephenson G, et al. (2000). Geotiff format specification geotiff revision 1.0. SPOT Image Corp, 1: 154–172. Santos ES (1969). Probabilistic Turing machines and computability. Proceedings of the American Mathematical Society, 22(3): 704–710. https://doi.org/10.1090/S0002-9939-1969-0249221-4 Sartore L, Boryan CG, Willis P (2022). Developing entropies of predictive cropland data layers for crop survey imputation. In: IGARSS 2022-2022 IEEE International Geoscience and Remote Sensing Symposium, 1404–1407. IEEE. Schuld M, Sinayskiy I, Petruccione F (2014). The quest for a quantum neural network. Quantum Information Processing, 13(11): 2567–2586. https://doi.org/10.1007/s11128-014-0809-8 Shannon CE (1948). A mathematical theory of communication. The Bell System Technical Journal, 27(3): 379–423. https://doi.org/10.1002/j.1538-7305.1948.tb01338.x Shor PW (1994). Algorithms for quantum computation: discrete logarithms and factoring. In: Proceedings 35th Annual Symposium on Foundations of Computer Science, 124–134. IEEE. Tong H (1975). Determination of the order of a Markov chain by Akaike’s information criterion. Journal of Applied Probability, 12(3): 488–497. https://doi.org/10.2307/3212863 USDA, FSA (2017). Farm Service Agency (FSA) Common Land Unit (CLU) information worksheet. https://www.fsa.usda.gov/Assets/USDA-FSA-Public/usdafiles/APFO/support-documents/pdfs/clu_infosheet_2017_Final.pdf Wolfram S (1984). Universality and complexity in cellular automata. Physica D, 10(1–2): 1–35. https://doi.org/10.1016/0167-2789(84)90245-8 Yaramasu R, Bandaru V, Pnvr K (2020). Pre-season crop type mapping using deep neural networks. Computers and Electronics in Agriculture, 176: 105664. https://doi.org/10.1016/j.compag.2020.105664 Yuen C (1975). A note on base-2 arithmetic logic. IEEE Transactions on Computers, 100(3): 325–329. https://doi.org/10.1109/T-C.1975.224216 Zadeh LA (1973). Outline of a new approach to the analysis of complex systems and decision processes. IEEE Transactions on Systems, Man and Cybernetics, 3: 28–44. https://doi.org/10.1109/TSMC.1973.5408575 Zhang C, Di L, Lin L, Guo L (2019). Machine-learned prediction of annual crop planting in the US Corn Belt based on historical crop planting maps. Computers and Electronics in Agriculture, 166: 104989. https://doi.org/10.1016/j.compag.2019.104989 Zhao R, Wang S (2021). A review of quantum neural networks: Methods, models, dilemma. arXiv preprint: https://arxiv.org/abs/2109.01840.