What you’ll do (automated path)

1. Prepare & reshape data (wide → long for grouped analysis)

2. Verify time-window continuity to ensure valid comparisons

3. Compare cross-year periods at two granularities (weekly vs daily)

4. Run auto-selected group tests via run_group_test() with assumptions and effect sizes

5. Fit trends with one call via explore_trend_auto() (automatic model family & smoothing)

6. (Optional) ARIMA readiness check to decide if time-series modeling is warranted

Selected functions used

• filter_by_groupcol() — subset/structure data by groups and date column
  
• check_continuity_by_window() — confirm continuous windows across years/months
• compare_monthly_cases() — cross-year comparison with chosen granularity & aggregation
  
• compare_distribution_by_granularity() — sanity-check distributional changes (day vs week)
  
• run_group_tests() — automated test selection + assumptions + effect sizes
  
• explore_trend_auto() — automatic trend modeling (Gaussian/Gamma, splines)
  
• check_rate_diff_arima_ready() (optional) — stationarity, seasonality & differencing hints

Dataset

EuStockMarkets contains daily closing prices for four European stock indices (DAX, SMI, CAC, FTSE) from 1991-01-01 to 1996-02-03.
For clarity and speed, this vignette focuses on DAX and CAC and a two-year cross-year window.

1. Installation and Setup

This section loads the necessary packages and prepares the built-in dataset EuStockMarkets for analysis.

We will: - Convert the built-in time-series object to a data.frame with an explicit date column. - Reshape it from wide format (one column per market) to long format (one column for market names, one for index values), which is easier to group, filter, and visualize in trendtestR workflows.

1.1 Load required packages

library(trendtestR)
library(dplyr)
library(tidyr)
library(lubridate)

1.2 Data Preparation

The built-in dataset contains daily closing prices of four European stock market indices: DAX (Germany), SMI (Switzerland), CAC (France), and FTSE (UK), covering the period 1991-01-01 to 1996-02-03.

# Load the built-in dataset

data("EuStockMarkets")

# Create a dataframe with a date column and the stock indices

eu_df <- data.frame(
  date = seq(as.Date("1991-01-01"), by = "day", length.out = nrow(EuStockMarkets)),
  as.data.frame(EuStockMarkets)
)

# Preview the last few rows
tail(eu_df)

# Reshape the dataset to long format for easier grouping and filterin
eu_long <- eu_df %>%
  pivot_longer(
    cols = c(DAX, SMI, CAC, FTSE),
    names_to = "market",
    values_to = "index"
  ) %>%
  mutate(market = factor(market))

# Preview the first few rows
head(eu_long)

2. Data Filtering

We keep only DAX (Germany) and CAC (France) for a smaller, faster analysis.
filter_by_groupcol() lets us select specific groups while keeping the data in long format.

# 
ecoDaxCac <- filter_by_groupcol(
  eu_long,
  group_col = "market",    # grouping variable
  value_col = "index",     # values to analyze
  datum_col = "date",      # date variable
  keep_levels = c("DAX", "CAC"),
  to_wide = FALSE,         
  keep_other_cols = TRUE   
)

# Preview the first few rows
head(ecoDaxCac)

3. Data Continuity Check

We use check_continuity_by_window() to verify there are no date gaps in the selected period, ensuring data quality before running further functions.

checkconti <- check_continuity_by_window(
  date_vec = ecoDaxCac$date,
  years = c(1991, 1993),
  months = c(10, 9), 
  window_unit = "day", 
  use_isoweek = TRUE, 
  allow_leading_gap = TRUE
)

# Display continuity results
cat("Data is continuous:", checkconti$continuous, "\n")
cat("Data range:", as.character(checkconti$range), "\n")
# Output: Data is continuous: TRUE 
# Output: Data range: 1991-10-01 1993-09-30

4. Cross-Year Data Comparison

We use compare_monthly_cases() to compare values between years over a cross-year period, allowing flexible month selection and time aggregation.

4.1 Weekly Granularity Comparison

We first compare 1992–1993 data aggregated weekly:

# Compare 1992-1993 data with weekly granularity
reseuro <- compare_monthly_cases(
  ecoDaxCac, 
  datum_col = "date", 
  value_col = "index", 
  group_col = "market",
  years = c(1992, 1993),
  months = c(10:12, 1:9), # Oct–Dec + Jan–Sep (cross-year)
  granularity = "week",    
  agg_fun = "mean",
  shift_month = "mth_to_next" #alternative param: mth_to_prev, none 
)

# Note: Function automatically excludes groups with no data (1991, 1995)
# Shows standardization info and data characteristics

4.2 Daily Granularity Comparison

We repeat the analysis at daily granularity to compare results:

# Compare same period with daily granularity
reseurod <- compare_monthly_cases(
  ecoDaxCac, 
  datum_col = "date", 
  value_col = "index", 
  group_col = "market",
  years = c(1992, 1993),
  months = c(10:12, 1:9),
  granularity = "day",    
  agg_fun = "median",
  shift_month = "mth_to_next"
)

# View statistical test results
print(reseuro$tests)
# Results show Kruskal-Wallis test with large effect size (eta² ≈ 0.31)
# Includes assumption checks and post-hoc Dunn tests

4.3 Distribution Comparison Across Granularities

We then compare distributions between granularities to guide aggregation choice:

# Compare distributions using Q-Q plots
compare_distribution_by_granularity(reseuro, reseurod)


#Shows normality tests and variance tests for different granularities
#Helps determine optimal time aggregation level

This helps determine the most suitable time aggregation level for subsequent statistical analyses.

5. Automated Statistical Testing

We use run_group_tests() to automatically select and perform the most appropriate statistical test based on data characteristics, including assumption checks and effect size calculation.

# Run automated group comparison tests
test_results <- run_group_tests(
  reseuro$data, 
  value_col = "index", 
  group_col = "market",
  effect_size = TRUE,
  report_assumptions = TRUE
)

print(test_results)
# Function automatically excludes groups with no data (FTSE, SMI)
# Recommends Mann-Whitney U-Test due to violated normality assumptions

6. Trend Modeling

We start with automatic model selection using explore_trend_auto(), which evaluates multiple candidate families (e.g., Gaussian, Gamma, Poisson, ZINB) and chooses the most suitable one based on AIC and model diagnostics.

This step provides a quick, data-driven baseline model before fine-tuning parameters such as spline degrees of freedom in the next section.

6.1 Automated Trend Exploration

# Automatically select the most appropriate trend model
trend_auto <- explore_trend_auto(
  reseuro$data, 
  datum_col = "date", 
  value_col = ".value", 
  group_col = "market",
  family = "auto", 
  kdf = 5
)

print(trend_auto$summary)
# Function compares Gaussian vs Gamma GLM and selects optimal model
# Shows AIC comparison and model selection rationale

6.2 Spline Degrees of Freedom Optimization

While explore_trend_auto() already selects a reasonable default, users may wish to manually fine-tune model complexity for deeper exploration.

Here we illustrate one such approach: selecting spline degrees of freedom based on the largest AIC drop compared to the previous candidate, rather than simply picking the absolute AIC minimum.
This captures the point of maximum improvement before diminishing returns.

This is just one possible workflow — any of the explore_*_trend() functions can be used interactively to test different model families, spline settings, or grouping structures for more tailored analysis.

# Create AIC comparison dataframe
aic_df <- data.frame(
  df_spline = integer(),
  AIC = numeric()
)

# Loop through different degrees of freedom
for (df in 4:7) {
  tmp <- explore_continuous_trend(
    reseuro$data, 
    datum_col = "date", 
    value_col = ".value", 
    group_col = "market",
    family = "gaussian",  
    df_spline = df
  )
  aic_df <- rbind(aic_df, data.frame(df_spline = df, AIC = AIC(tmp$model)))
}

# Find optimal degrees of freedom
aic_drop <- diff(aic_df$AIC)
optimal_df <- aic_df$df_spline[which.max(-aic_drop)] + 1 # largest negative drop
cat("optimal spline degrees of freedom:", optimal_df, "\n")

6.3 Modeling with Optimal Parameters

We refit the trend model using the optimal spline degrees of freedom found above, ensuring the model complexity is justified by the largest improvement in fit.

euexp <- explore_continuous_trend(
  reseuro$data, 
  datum_col = "date", 
  value_col = ".value", 
  group_col = "market",
  family = "gaussian", 
  df_spline = optimal_df  # Use df=5 for optimal fit
)

# View model summary
summary(euexp$model)

7. Model Diagnostics

After fitting the trend model, we run diagnose_model_trend() to check whether model assumptions are met. This step validates residual behavior, tests for normality and variance homogeneity, and helps decide if further model adjustments are necessary.

# Perform model diagnostics
diagnosis <- diagnose_model_trend(euexp$model)
# Provides residual plots, normality tests, and homogeneity checks
# Includes Kolmogorov-Smirnov, Shapiro-Wilk, and Levene tests

8. ARIMA Modeling Preparation

Before applying ARIMA, we run check_rate_diff_arima_ready() to assess if the data meets key time-series assumptions. This step checks for outliers, trend and seasonality patterns, and suggests whether differencing is required, ensuring a more stable ARIMA fit.

# Pre-ARIMA modeling checks
arima_check <- check_rate_diff_arima_ready(
  rate_diff_vec = eu_df$DAX,
  date_vec = eu_df$date,
  frequency = 52,
  plot_acf = TRUE,
  do_stl = TRUE
)

# Shows comprehensive analysis: outliers, stationarity tests, 
# seasonal decomposition, and differencing recommendations

Other Functions (at a glance)

We also provide utilities beyond this recommended workflow.