Total Pageviews

Saturday, June 14, 2025

How to copy files from multiple servers to staging server and merge them into one consolidated file using shell script (Pull Method)

Below script can be used to scp all the csv files to staging server and merge them into single csv file.

 servers=("srv1" "srv2")

for srv in "${servers[@]}"
do
  scp oracle@"$srv":/tmp/OLAM_CP/CP_report/file.csv /tmp/OLAM_CP/CP_report/file_"$srv".csv
done

# Copy header from the first file
head -n 1 /tmp/OLAM_CP/CP_report/file_srv1.csv > /tmp/OLAM_CP/CP_report/final_report.csv

# Append data (skip header) from all files
for file in /tmp/OLAM_CP/CP_report/file_*.csv
do
  tail -n +2 "$file" >> /tmp/OLAM_CP/CP_report/final_report.csv
done


No comments:

Post a Comment