This repository provides a standardized archive of the US Department of Agriculture Farm Service Agency (FSA) Farm Payment Files. These files contain detailed records of payments made to agricultural program participants, originally released as Microsoft Excel files.

The included R script automates the discovery, download, and conversion of these files into a partitioned, analysis-ready format.

📁 Repository Contents

  • fsa-payment-files.R: Main R script to download, extract, clean, and write the payment files to a partitioned Parquet format.
  • README.Rmd: This file. Contains background information, usage instructions, and archive structure.
  • data-raw/: Folder used for downloading and unpacking Excel and CSV files.
  • fsa-payment-files/: Final output directory containing the processed data as a Parquet dataset, partitioned by state and year.

🔁 Processing Workflow

The fsa-payment-files.R script performs the following steps:

  1. Discovery: Scrapes the FSA payment files website to find downloadable Excel files (2004–2024).
  2. Download: Saves original files to data-raw/.
  3. Extraction: Parses each Excel file, handling annual variation in schema and formatting.
  4. Standardization: Renames fields, standardizes formats (e.g., FIPS codes, program names), and removes duplicates.
  5. Archiving: Writes the full dataset to the fsa-payment-files/ directory as partitioned Parquet files (State FSA Name= and Accounting Program Year= directories).
  6. Upload (optional): Uses the AWS paws package to upload the full Parquet archive to a public S3 bucket for remote access.

☁️ Public Access via S3

The full archive is hosted in a public Amazon S3 bucket:

s3://sustainable-fsa/fsa-payments/

You can access the data directly using:

AWS CLI

aws s3 ls s3://sustainable-fsa/fsa-payment-files/ --no-sign-request

In R with arrow and s3:// support

library(arrow)

## 
## Attaching package: 'arrow'

## The following object is masked from 'package:utils':
## 
##     timestamp

dataset <- 
  open_dataset("s3://sustainable-fsa/fsa-payment-files/", 
               filesystem = s3_bucket("sustainable-fsa", anonymous = TRUE))

🧭 Notes

  • File structure varies by year, and the script includes heuristics to detect column shifts and missing headers.
  • Partitioning by state and year facilitates fast querying and cloud-native workflows.
  • The processing script can be rerun to include new years as they are released.

📅 Update Schedule

This dataset is refreshed annually after the USDA FSA releases new payment files, typically in the spring. Additional years or corrections may be processed as available.

🔗 Related Resources

📍 Quick Start: Visualize data in the FSA Farm Payment Files in R

This snippet shows how to load data from the Farm Payment Files archive and create a simple map using sf and ggplot2.

# Load required libraries
library(arrow)
library(sf)
library(ggplot2) # For plotting
library(tigris)  # For state boundaries
library(rmapshaper) # For innerlines function

# Example accessing payment files on S3
# A map of 2024 LFP Payments by county

lfp_payments <-
  # arrow::s3_bucket("sustainable-fsa/fsa-payment-files",
  #                  anonymous = TRUE) %>%
  "fsa-payment-files" |>
  arrow::open_dataset() |>
  dplyr::filter(`Accounting Program Description` %in% 
                  c(
                    "LIVESTOCK FORAGE PROGRAM",
                    "LIVESTOCK FORAGE DISASTER PROGRAM"
                  )) |>
  dplyr::group_by(`FSA Code`) |>
  dplyr::summarise(
    `Disbursement Amount` = sum(`Disbursement Amount`, na.rm = TRUE)
  ) |>
  dplyr::collect()

## The Normal Grazing Period data files use FSA county definitions
## Download from the FSA_Counties_dd17 archive
counties <- 
  sf::read_sf("https://sustainable-fsa.github.io/fsa-counties-dd17/fsa-counties-dd17.topojson",
              layer = "counties") |>
  sf::st_set_crs("EPSG:4326") |>
  sf::st_transform("EPSG:5070")

## Calculate the 2024 Normal Grazing Period duration for Native Pasture, and
## combine with the county data
lfp_payments_counties <-
  lfp_payments |>
  dplyr::select(id = `FSA Code`,
                `Disbursement Amount`) |>
  dplyr::right_join(counties) |>
  sf::st_as_sf() |>
  dplyr::mutate(
    `Disbursement Amount` = 
      tidyr::replace_na(`Disbursement Amount`, 0)
  )


# Plot the map
ggplot(counties) +
  geom_sf(data = sf::st_union(counties),
          fill = "grey80",
          color = NA) +
  geom_sf(data = lfp_payments_counties,
          aes(fill = `Disbursement Amount`), 
          color = NA) +
  geom_sf(data = rmapshaper::ms_innerlines(counties),
          fill = NA,
          color = "white",
          linewidth = 0.1) +
  geom_sf(data = counties |>
            dplyr::group_by(state) |>
            dplyr::summarise() |>
            rmapshaper::ms_innerlines(),
          fill = NA,
          color = "white",
          linewidth = 0.2) +
  khroma::scale_fill_batlowK(limits = c(0, NA),
                             breaks = seq(0, 80000000, 10000000),
                             # trans = "log",
                             name = "Payments\n($ Millions)",
                             labels = scales::label_currency(scale = 0.000001, suffix = "M")) +
  labs(title = "FSA Livestock Forage Disaster Program",
       subtitle = "Total Payments, 2008–2024") +
  theme_void()

🧭 About FSA County Codes

The USDA FSA uses custom county definitions that differ from standard ANSI/FIPS codes used by the U.S. Census. To align the Farm Payment Files with geographic boundaries, use the FSA-specific geospatial dataset archived in the companion repository:

🔗 sustainable-fsa/fsa-counties-dd17

FSA county codes are documented in FSA Handbook 1-CM, Exhibit 101.

📜 Citation

If using this data in published work, please cite:

USDA Farm Service Agency. Farm Payment Files, 2004–2024. Archived by R. Kyle Bocinsky. Accessed via GitHub archive, YYYY. https://sustainable-fsa.github.io/fsa-payment-files/

📄 License

  • Raw FSA Farm Payment Files data (USDA): Public Domain (17 USC § 105)
  • Processed data & scripts: © R. Kyle Bocinsky, released under CC0 and MIT License as applicable

⚠️ Disclaimer

This dataset is archived for research and educational use only. It may not reflect current FSA payments and policy. Always consult your local FSA office for the latest program guidance.

To locate your nearest USDA Farm Service Agency office, use the USDA Service Center Locator:

🔗 USDA Service Center Locator

👏 Acknowledgment

This project is part of:

Enhancing Sustainable Disaster Relief in FSA Programs
Supported by USDA OCE/OEEP and USDA Climate Hubs
Prepared by the Montana Climate Office

📬 Contact

R. Kyle Bocinsky
Director of Climate Extension
Montana Climate Office
📧 kyle.bocinsky@umontana.edu
🌐 https://climate.umt.edu