Getting data home Getting data home FTP Getting

  • Slides: 53
Download presentation
Getting data home ?

Getting data home ?

Getting data home ? FTP

Getting data home ? FTP

Getting data home ? FTP tapes

Getting data home ? FTP tapes

Getting data home ? FTP: not secure tapes: incompatible

Getting data home ? FTP: not secure tapes: incompatible

Getting data home ? FTP: not secure tapes: incompatible distracting

Getting data home ? FTP: not secure tapes: incompatible distracting

Getting data home ? http: //bl 831. als. lbl. gov/~jamesh/beamline/moving_data. html

Getting data home ? http: //bl 831. als. lbl. gov/~jamesh/beamline/moving_data. html

Getting data home ? http: //bl 831. als. lbl. gov/~jamesh/beamline/moving_data. html http: //bcsb-web. als.

Getting data home ? http: //bl 831. als. lbl. gov/~jamesh/beamline/moving_data. html http: //bcsb-web. als. lbl. gov/beamline/Sector 8/Data. Backup. htm

Getting data home ? http: //bl 831. als. lbl. gov/~jamesh/beamline/moving_data. html http: //bcsb-web. als.

Getting data home ? http: //bl 831. als. lbl. gov/~jamesh/beamline/moving_data. html http: //bcsb-web. als. lbl. gov/beamline/Sector 8/Data. Backup. htm http: //ucxray. berkeley. edu/~jamesh/elves/manual/tricks. html

Can you login? ssh user@home

Can you login? ssh [email protected]

Universal directory-transfer tar cf -. | ssh user@home “cd /dir ; tar xvf -”

Universal directory-transfer tar cf -. | ssh [email protected] “cd /dir ; tar xvf -”

Universal directory-transfer tar cf -. | ssh user@home “cd /dir ; tar xvf -”

Universal directory-transfer tar cf -. | ssh [email protected] “cd /dir ; tar xvf -”

Universal directory-transfer tar cf -. | ssh user@home “cd /dir ; tar xvf -”

Universal directory-transfer tar cf -. | ssh [email protected] “cd /dir ; tar xvf -” Password: Ctrl-z bg

Making it faster tar cf -. | compress | ssh user@home “cd /dir ;

Making it faster tar cf -. | compress | ssh [email protected] “cd /dir ; uncompress | tar xvf -”

The easy way http: //bl 831. als. lbl. gov/~jamesh/elves/scripts/sendhome. user@home: /dir

The easy way http: //bl 831. als. lbl. gov/~jamesh/elves/scripts/sendhome. [email protected]: /dir

Real Time http: //bl 831. als. lbl. gov/~jamesh/elves/scripts/sendhome first_001. img user@home: /dir

Real Time http: //bl 831. als. lbl. gov/~jamesh/elves/scripts/sendhome first_001. img [email protected]: /dir

Through a firewall tar cf -. | ssh user@firewall ‘ssh fileserver “cd /dir ;

Through a firewall tar cf -. | ssh [email protected] ‘ssh fileserver “cd /dir ; tar xvf -”’

The other way ssh user@home cd /dir ssh user@beamline “cd /data ; tar cf

The other way ssh [email protected] cd /dir ssh [email protected] “cd /data ; tar cf -. ” | tar xvf –

The other way, faster ssh user@beamline “cd /dir ; tar cf -. | compress”

The other way, faster ssh [email protected] “cd /dir ; tar cf -. | compress” | uncompress | tar xvf –

The easy other way ssh user@beamline “download yourname” | tar xvf –

The easy other way ssh [email protected] “download yourname” | tar xvf –

What if it stops?

What if it stops?

What if it stops? ssh user@beamline “download yourname frame_046. img” | tar xvf –

What if it stops? ssh [email protected] “download yourname frame_046. img” | tar xvf –

What if it stops? http: //bl 831. als. lbl. gov/~jamesh/elves/scripts/sendhome frame_046. img user@home: /dir

What if it stops? http: //bl 831. als. lbl. gov/~jamesh/elves/scripts/sendhome frame_046. img [email protected]: /dir

What if I’m confused?

What if I’m confused?

What if I’m confused? 4 steps

What if I’m confused? 4 steps

1) Generate a filename list find. -name ‘*. img’ >! here. log

1) Generate a filename list find. -name ‘*. img’ >! here. log

2) Generate a “done” list ssh user@home “cd /dir ; find. ” >! there.

2) Generate a “done” list ssh [email protected] “cd /dir ; find. ” >! there. log

3) Make the “todo” list diff there. log | awk ‘/^>/{print $2}’ >! 2

3) Make the “todo” list diff there. log | awk ‘/^>/{print $2}’ >! 2 bsent. log

4) Feed “still left” list to tar cat 2 bsent. log | tar c.

4) Feed “still left” list to tar cat 2 bsent. log | tar c. Tf - - | ssh [email protected] “cd /dir ; tar xvf -”

What if I can’t login? ssh user@home warning: Connecting to home failed: Connection Refused

What if I can’t login? ssh [email protected] warning: Connecting to home failed: Connection Refused

“Sneaker. Net”

“Sneaker. Net”

“Sneaker. Net” Tape Firewire DVD

“Sneaker. Net” Tape Firewire DVD

Fire. Wire OS Format Mac HFS Windows NTFS Linux FAT 32

Fire. Wire OS Format Mac HFS Windows NTFS Linux FAT 32

Realtime Firewire 1)plug in

Realtime Firewire 1)plug in

Realtime Firewire 1)plug in 2)share it

Realtime Firewire 1)plug in 2)share it

Realtime Firewire 1)plug in 2)share it 3)smbclient

Realtime Firewire 1)plug in 2)share it 3)smbclient

Realtime Firewire tar cf -. | smbclient //laptop/share –U user%password" -N -Tx -

Realtime Firewire tar cf -. | smbclient //laptop/share –U user%password" -N -Tx -

Realtime Firewire find. | tar c. Tf - - | smbclient //laptop/share –U user%password"

Realtime Firewire find. | tar c. Tf - - | smbclient //laptop/share –U user%password" -N -Tx -

The easy way http: //bl 831. als. lbl. gov/~jamesh/beamline/backup_firewire. com. txt backup_firewire. com.

The easy way http: //bl 831. als. lbl. gov/~jamesh/beamline/backup_firewire. com. txt backup_firewire. com.

DVD OS Format Mac HFS Windows NTFS Linux UFS

DVD OS Format Mac HFS Windows NTFS Linux UFS

DVD OS Format Mac ISO Windows ISO Linux ISO

DVD OS Format Mac ISO Windows ISO Linux ISO

HOWTO make a DVD mkisofs -r -J. >! image. iso

HOWTO make a DVD mkisofs -r -J. >! image. iso

HOWTO make a DVD mkisofs -r -J. >! image. iso dvdrecord dev=0, 0, 0

HOWTO make a DVD mkisofs -r -J. >! image. iso dvdrecord dev=0, 0, 0 –data image. iso

The easy way http: //bl 831. als. lbl. gov/~jamesh/beamline/backup_local. com. txt backup_local. com.

The easy way http: //bl 831. als. lbl. gov/~jamesh/beamline/backup_local. com. txt backup_local. com.

The easy way http: //bl 831. als. lbl. gov/~jamesh/beamline/backup_local. com. txt backup_local. com. http:

The easy way http: //bl 831. als. lbl. gov/~jamesh/beamline/backup_local. com. txt backup_local. com. http: //bl 831. als. lbl. gov/~jamesh/beamline/autolabel. com. txt autolabel. com

DVD data archive

DVD data archive

DVD data archive

DVD data archive

DVD data archive

DVD data archive

DVD data archive

DVD data archive

DVD data archive

DVD data archive

DVD data archive

DVD data archive

DVD data archive

DVD data archive

Acknowledgments Beamline 8. 3. 1 staff George Meigs Jane Tanamachi

Acknowledgments Beamline 8. 3. 1 staff George Meigs Jane Tanamachi