Horje

Bash - Read a file (data stream, variable) line-by-line (and/or field-by-field)?
Start
  Looping through a file line by line 
while IFS= read -r line; do
   echo "$line"
done <file

If file may not include a newline at the end, then:

while IFS= read -r line || [ -n "$line" ]; do
   echo "$line"
done <file
  Looping through the output of a command field by field

Let's assume that the field separator is :

while IFS= read -d : -r field || [ -n "$field" ];do
    echo "**$field**"
done < <(ping google.com)

Or with a pipe:

ping google.com | while IFS= read -d : -r field || [ -n "$field" ];do
    echo "**$field**"
done
  Read lines of a file into an array 
readarray -t arr <file

Or with a loop:

arr=()
while IFS= read -r line; do
   arr+=("$line")
done <file
  Read lines of a string into an array 
var='line 1
line 2
line3'
readarray -t arr <<< "$var"

or with a loop:

arr=()
while IFS= read -r line; do
   arr+=("$line")
done <<< "$var"
  Looping through a string line by line 
var='line 1
line 2
line3'
while IFS= read -r line; do
   echo "-$line-"
done <<< "$var"

or

readarray -t arr <<< "$var"
for i in "${arr[@]}";do
    echo "-$i-"
done
  Looping through the output of a command line by line 
while IFS= read -r line;do
    echo "**$line**"
done < <(ping google.com)

or with a pipe:

ping google.com |
while IFS= read -r line;do
    echo "**$line**"
done
  Read a file field by field

Let's assume that the field separator is : (colon) in the file file.

while IFS= read -d : -r field || [ -n "$field" ]; do
   echo "$field"
done <file

For a content:

first : se
con
d:
    Thi rd:
    Fourth

The output is:

**first **
** se
con
d**
**
    Thi rd**
**
    Fourth
**
  Read a string field by field

Let's assume that the field separator is :

var='line: 1
line: 2
line3'
while IFS= read -d : -r field || [ -n "$field" ]; do
   echo "-$field-"
done <<< "$var"

Output:

-line-
- 1
line-
- 2
line3
-
  Read fields of a file into an array

Let's assume that the field separator is :

arr=()
while IFS= read -d : -r field || [ -n "$field" ]; do
   arr+=("$field")
done <file
  Read fields of a string into an array

Let's assume that the field separator is :

var='1:2:3:4:
newline'
arr=()
while IFS= read -d : -r field || [ -n "$field" ]; do
   arr+=("$field")
done <<< "$var"
echo "${arr[4]}"

Output:

newline
  Reads file (/etc/passwd) line by line and field by field 
#!/bin/bash
FILENAME="/etc/passwd"
while IFS=: read -r username password userid groupid comment homedir cmdshell
do
  echo "$username, $userid, $comment $homedir"
done < $FILENAME

In unix password file, user information is stored line by line, each line consisting of information for a user separated by colon (???? character. In this example while reading the file line by line, the line is also split into fields using colon character as delimiter which is indicated by the value given for IFS.

Sample input

mysql:x:27:27:MySQL Server:/var/lib/mysql:/bin/bash
pulse:x:497:495:PulseAudio System Daemon:/var/run/pulse:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
tomcat:x:91:91:Apache Tomcat:/usr/share/tomcat6:/sbin/nologin
webalizer:x:67:67:Webalizer:/var/www/usage:/sbin/nologin

Sample Output

mysql, 27, MySQL Server /var/lib/mysql
pulse, 497, PulseAudio System Daemon /var/run/pulse
sshd, 74, Privilege-separated SSH /var/empty/sshd
tomcat, 91, Apache Tomcat /usr/share/tomcat6
webalizer, 67, Webalizer /var/www/usage

To read line by line and have the entire line assigned to variable, following is a modified version of the example. Note that we have only one variable by name line mentioned here.

#!/bin/bash
FILENAME="/etc/passwd"
while IFS= read -r line
do
  echo "$line"
done < $FILENAME

Sample Input

mysql:x:27:27:MySQL Server:/var/lib/mysql:/bin/bash
pulse:x:497:495:PulseAudio System Daemon:/var/run/pulse:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
tomcat:x:91:91:Apache Tomcat:/usr/share/tomcat6:/sbin/nologin
webalizer:x:67:67:Webalizer:/var/www/usage:/sbin/nologin

Sample Output

mysql:x:27:27:MySQL Server:/var/lib/mysql:/bin/bash
pulse:x:497:495:PulseAudio System Daemon:/var/run/pulse:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
tomcat:x:91:91:Apache Tomcat:/usr/share/tomcat6:/sbin/nologin
webalizer:x:67:67:Webalizer:/var/www/usage:/sbin/nologin

Parameters

Parameter Details
IFS Internal field separator
file A file name/path
-r Prevents backslash interpretation when used with read
-t Removes a trailing newline from each line read by readarray
-d DELIM Continue until the first character of DELIM is read (with read), rather than newline
 

Read Article
https://develop.horje.com/learn/1434/reference

 

Horje