Repository navigation
degrad resolver + tests - #346
Draft
phlebailly6 wants to merge 7 commits into
Draft
phlebailly6 wants to merge 7 commits into
phlebailly6 wants to merge 7 commits into
Conversation
Member
|
In development: webiste is kind of abandoned. It also blocks cloud tests so actions will likely break. Still, since there are no data updates, the links will not change. |
…FRAGILE notes - R/degrad.R: replace operation_crs "+proj=poly ... +ellps=aust_SA" with IBGE's own equal-area projection (+proj=aea, GRS80, central meridian -54; IBGE, Malha Municipal Digital e Areas Territoriais 2024, Notas metodologicas 01/2025, Apendice B). The old string is the same wrong one load_deter() had: a near-copy of EPSG:5880 (polyconic, not equal-area) with its ellipsoid mistyped to a legacy PROJ preset. It inflated `area` the further a polygon sits from longitude -54: +0.41% over the 10 years (+0.30% to +0.74% by year), up to +4.7% west of -70. The new `area` matches an ellipsoidal geodesic reference to within 0.001%. Output rows differ by -1 to +6 boundary slivers per year. - R/degrad.R: three "# FRAGILE:" notes (raw CRS handling, operation_crs, st_intersection() row splitting) carrying the figures measured on live data. - Manifest: docs_url for the 10 degrad rows. - actions/scripts/fragility_notes.R: register R/degrad.R in FRAGILITY_SOURCES. - tests: test-fragility-notes.R expects >= 3 tags in R/degrad.R; test-resolution-matrix-v2.R declares the 10 new docs_url cells. - NEWS.md: entry for the `area` change. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The 13% and 0.72 km2 figures cited in the CRS/datum-shift FRAGILE tag (load_degrad(), ~line 107) were from two different municipalities, not one -- the original wording implied a single 0.72 km2 municipality with a 13% area move. Corrected: Santa Maria das Barreiras (PA) moves +13.2% (its own DEGRAD polygon area there is small, 1.97 km2, so a fixed metric shift reads as a large percentage); Barra do Corda (MA), at a similar small scale (0.72 km2), moves only -0.15%. Matches the already corrected wording in the shared SKILL.md and Issue #350.
The vignette's CRS guidance was generic and out of date ("verify CRS
after loading, metadata sometimes unclear") -- written before
load_degrad()'s CRS fix. Replaced both mentions (the "Note on CRS"
paragraph and "Spatial Considerations" point 3) with the same accurate
description already approved for vignettes/DETER.Rmd: load_degrad()
reprojects internally to IBGE's official equal-area Albers projection
(SIRGAS 2000/GRS80, central meridian -54) before computing area and
overlaying municipality boundaries, not UTM zones. No mention of
internal resolver/FRAGILE mechanics, per Antonio's review comments on
the update-deter PR.
Checked against Antonio's other update-deter review comments for
applicability to DEGRAD: the archive_file-goes-stale-after-fallback
bug doesn't apply (resolve_degrad.R has no renaming/fallback
mechanism, pure verification only); NEWS.md/README.md/this vignette
were already free of internal resolver mentions; resolve_degrad.R's
header has no stale claims to fix.
Harmonize each year's raw geometry to SIRGAS 2000 (EPSG:4674) via sf::st_transform() right after download, before bind_rows() -- replaces the old post-bind_rows() block that only handled the "missing CRS" case. This closes both bugs tracked in #350: - A time_period spanning a raw-CRS group boundary (2009:2010, 2014:2015, 2007:2016) no longer crashes bind_rows() with "arguments have different crs". Related to #283. - The SAD69->SIRGAS2000 datum shift (59.5-75.0 m per polygon for 2010-2014-era data) is now actually applied, since EPSG:4618->4674 is a real, PROJ-registered transform -- previously operation_crs's unnamed-datum proj4 string silently applied a zero-shift transform. raw_data = TRUE is unaffected (harmonization runs after that early return) -- verified byte-identical to the previous code on 2010:2012, 2007:2009, 2015:2016, 2010:2014. Net effect on 10-year totals: area -0.0009%, row count -36 of 231,489 (7 tiny municipality-year slivers appear/disappear, 0.031 km2 total). Municipality-level deltas: median 0.004-0.1%, 95th pct 1.2-2.2%, with outliers up to ~15% on municipalities with <4 km2 of DEGRAD that year. Updates the 3 '# FRAGILE:' tags in R/degrad.R with these measurements (replacing the earlier placeholder notes), and documents the fix plus a newly-visible pre-existing data gap (2007-2009 raw files lack view_date, so multi-year loads spanning those years now return NA there instead of crashing) in NEWS.md. Full test suite green (584 expectations, 0 failures); test-fragility- notes.R still finds exactly 3 tags.
… (refs #350) Add tests/testthat/test-degrad-crs.R: three synthetic polygons at the same lon/lat, declared with no CRS / SAD69 + TOWGS84 / SIRGAS 2000, with external_download() mocked. It covers a time_period crossing both raw-CRS group boundaries (no "arguments have different crs"), the real SAD69 -> SIRGAS 2000 shift (about 60-75 m), and raw_data = TRUE keeping each year's own CRS. It fails on the pre-fix code, passes now, and runs in about 1 s with no network. Widen actions/scripts/check_all_loads.R: load_degrad now loads c(2009, 2010, 2016) instead of 2016 alone, so the monthly real-data check crosses both CRS group boundaries in one call (about 75 s; fails on the pre-fix code with "arguments have different crs").
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.