Locating and using meteorological data

Table of contents

1. Buoy data
2. Radar profiler data
3. Coastal Waves 96 data
1. High rate Coastal Waves 96 data
4. Meteorological data on coast (needs to be updated)
5. Recommended reading
6. Useful information (getting computer help, UCSD web sites, etc)
1. How to constuct a web page to archive plots
2. How to read a netcdf file into matlab 5

Buoy data stored on "coast"

1. Getting a quick look at the data
1. A program called guiplot.m guides you through plotting the buoy data.  To use it, type "guiplot" at your matlab prompt.  You will need to add the following two lines to your /home/jessica/matlab/startup.m file so matlab can find the program.  Once you have added them, type "startup" at the matlab prompt to update matlab.
1. path(path,'/home/kate/matlab') ;
2. path(path,'/home/robb/matlab/robb') ;
2. Where is data located?
1. /project/zoo/ndbc/buoydata/ contains directories where the buoy data is grouped by year.  For example, the 1996 data is found in /project/zoo/ndbc/buoydata/1996.
2. Each buoy has its own file for the year's worth of data it contains.  For example, buoy 54 has data in /project/met/surface_met_stations/NDBC_mets/1996/46054_96.csa
3. How does matlab access these files?
1. path(path,'/project/zoo/ndbc/buoydata/1996') ; % Adds a directory to the ones searched by matlab
2. You may want to put this command in your /home/jessica/matlab/startup.m file, which is run every time you start a matlab session on coast.  That way, matlab will always know to look in this directory.
4. What do the buoy files contain?
1. 5 header  lines:
1. start_time      npts    samp_interv     elev    lat     lon
2. yymmddhhmmss    #_pts   seconds meters  deg     deg
3. <date of the first entry, with 2 digits indicating year, month,day, minute, second> <number of data lines in the file> <time interval data is recorded, in seconds.  Hourly data = 3600 seconds> <observations' elevation, which is 10 m > <latitude of buoy location <longitude of buoy location>
4. north   east    atm_pressure    atm_temp_dry    sea_surf_temp
5. m_s     m_s     hPa     deg_C   deg_C
2. Many lines containing data.  Columns contain:
1. <10 m north wind component (m/s)> <10 m east wind component (m/s)> <10 m atmospheric pressure> <10 m atmospheric temperature (C)> <sea surface temperature (C)>
5. What programs exist to read the data?
1. You may want to copy these matlab programs into a directory called /home/jessica/matlab/buoy_programs.  That way you can change them.  Or, add the following line to your /home/jessica/startup.m file and matlab will always call them up:  path(path,'/home/kate/matlab') ;
1. /home/kate/matlab/chunk_o_data.m    Reads a buoy file you choose.  It calls a number of other functions that you will need to copy too:  timediff.m , endtime.m, and maybe some others from /home/kate/matlab
6. How do I plot the data?
1. A program called guiplot.m guides you through plotting the buoy data.  To use it, type "guiplot" at your matlab prompt.  You will need to add the following two lines to your /home/jessica/matlab/startup.m file so matlab can find the program:
1. path(path,'/home/kate/matlab') ;
2. path(path,'/home/robb/matlab/robb') ;
2. Another, less versitile, example of a file to plot buoy data is /home/kate/matlab/buoy_plot.m , which calls a lot of other programs from /home/kate/matlab.
7. How can I add a "time" column to a .csa file, such as the buoy data?
1. /home/kate/matlab/put_julian_col.m reads in a .csa file, takes the segment between the start/stop time you choose, and adds a first column which is the julian day.  The julian day starts at 1 on January 1 and goes to 365 on December 31.

Profiler data stored on "coast"

1. Where are the profiler data files?
1. /home/robb/profilers/radar_915 contains files such as BBY.mat (all data vs time, altitude)
2. /home/robb/temp_work/W_Coast_paper/profilers/radar/BBY_inv.mat (Robb derived the inversion height vs time from the profiler files.  His routine which produced the inversion heights,  /home/robb/matlab/prof_tools/inv_bs_tp.m  is automated and you might want to compare it to the individual profiles to see what it is picking out).
2. What kind of data do the profiler files contain?
1. There are profilers at Bodega Bay (BBY.mat) Crescnet City (CRC.mat) Fort Ord (FTO) Point Arena (PAA) Goleta (GLA) Newport (NPT) and Point Piedras Blancas (PPB)
2. Wind speed, direction and virtual temperature vs altitude
3. Variable names all begin with the station identifier followed by a 2 (like "BBY2" for Bodega Bay), and are BBY2Eh (east wind component (m/s) at lower resolution in the boundary layer)         BBY2Nl  (north wind component (m/s) at high resolution in the boundary layer)        BBY2lat  (latitude of station)       BBY2time (a time stamp with 2 digits for year, month,day, hour, minute, and second)

BBY2El  (east wind component (m/s), high resolution in the boudary layer)        BBY2Temp (virtual temperature (C))      BBY2levels (altitude (m) of the data)
BBY2Nh  (north component of velocity (m/s), low resoliton in the BL)      BBY2elev (elevation of the station (m))      BBY2lon (longitude of the station)
3. How can I get a quick look at profiler data?
1. http://www-ccs.ucsd.edu/~kate/profilers.html.  Scroll to the bottom for individual days' time series.
4. What programs exist to read the profiler data?
1. You will need to put the following lines in your /home/jessica/matlab/startup.m file so that matlab knows where to search for the programs which read profiler data in:
1. path(path,'/home/robb/profilers/radar_915') ;
2. path(path,'/home/kate/matlab/profilers') ;
2. Robb wrote routines to derive the inversion base and top from the profiler data which you can find in /home/robb/matlab/prof_tools.  Check the derived heights against the actual profiles to see what is being selected as the base and top.
3. You can extract a subset of the data, for example Crescent City data from June 1, 1996 to June 30, 1996 , using /home/kate/matlab/profilers/multi_read_915.m.
5. What programs exist to plot the profiler data?
1. /home/kate/matlab/profilers/plot_radar_hours.m  % Will contour profiles vs time, as explained in the program.  There are other plotting programs in /home/kate/matlab/profilers, feel free to copy any.
6. Known problems with the data
1. Sometimes the radar can't see the whole boundary layer, and the signal dies out before the BL top is reached.  Sometimes the minimum data height is too high, and you've missed the inversion base.
2. Winds are significantly slower than those a couple of km offshore (ie, aircraft or buoy winds).
7. How does the profiler work?
1. Read http://www-ccs.ucsd.edu/~kate/profiler_pics/profiler_info.html

Coastal Waves 96 data stored on "myfanwy2"

1. What data was recorded during Coastal Waves 96?
1. A list of variables we obtained can be found in /worker/twelve/coastal_waves/coastal_doc/110_params.txt.
2. There are many useful bulletins at http://raf.atd.ucar.edu/Bulletins/, including #9, "Standard Output Variables", which tells how the data was obtained from the instruments.
2. Where is the data stored?
1. On myfanwy2, each flight day has a directory.  For example, for the June 7 flight, the lowrate data is in /worker/twelve/coastal_waves/jun07/data/960607C_s_new.mat
2. The high rate data is in /worker/twelve/coastal_waves/jun07/data/960607C_h_new.mat
3. How do I access a specific variable?
1. Example: Let's say you want to plot mixing ratio vs time for the June 7 flight.
1. Load /worker/twelve/coastal_waves/jun07/data/960607C_s_new.mat into your matlab workspace
2. Go to /worker/twelve/coastal_waves/coastal_doc/110_params.txt to look up the variable name that represents mixing ratio:  mrla and mrla1.  Which one to use?  Check in /worker/twelve/coastal_waves/coastal_doc/110_QA.txt, where mrla is recommended over mrla1.
3. Check in 110_QA.txt  to see if there were any known problems with mrla on this flight.  No.
4. plot(spm,mrla) ; % Note that spm = seconds past midnight in GMT time.
2. What if I just want to plot data from a single profile or level run?  You'll probably work out your own system, but here's what I do.
1. To get data from a level run, for run 4 on June 11 flight for example
1. Load /eport/home/edwards/programs/find_it/flat11.mat, which contains the start and end indexes for the flat runs flown on June 11.  Each level run is associated with a variable like "run4", which contains a short command telling matlab the time this run was recorded:  spm>=83673&spm<=85445.  What that means is "find the seconds past midnight (the time variable) greater than 83673 and less than 85445."
2. For example, to obtain the mixing ratio that was recorded on run 4:                                                  mr_run4 = mrla(eval(run4)) ;  % type this at the matlab prompt
2. To get data from a profile, for example profile 7 on June 12:
1. Load /export/home/edwards/programs/find_it/profile_index12.mat into matlab.   Each profile is associated with a variable like pro7 = 'spm>62649&spm<62807'.  To find the mixing ratio recorded during profile 7, type mr_pro7 = mrla(eval(pro7)) ;
4. What are known problems with the data?
1. Some problems with the data are listed in  /worker/twelve/coastal_waves/coastal_doc/110_QA.txt
2. The surface temperature drifts and must be corrected for the fact that some radiation reaching the radiometer is reflected, not emitted.  One solution: /export/home/edwards/matlab/time_series/correct_sst.m
3. The liquid water content must be zeroed out in dry air for each profile, since it contains an offset.
4. Alitude:  the radar altitude (hgm232) is accurate but spikes above 600 m.  For altitudes above this, correct the pressure altitude (palt), which is derived from the hydrostatic equation and standard surface pressure, with the radar altitude below 600 m, for example using /export/home/edwards/matlab/correct_palt.m
5. Where can I find programs that are already written, so I don't reinvent the wheel?
1. /seven/matlab contains the programs common to David Roger's group to derive meteorological variables such as saturation vapor pressure, etc.

High-rate Coastal Waves 96 data

1. How can I access high-rate data for fluxes?
1. Where is the data?
1. The high rate data (25 Hz) for June 7, for example,  can be found in /worker/twelve/coastal_waves/jun07/data/960607C_h_new.mat.  It is in columns by 1/25 of a second, and the time variable (spm) is the same as the low rate data
2. How can I get more information on the high-rate variables?
1. Sometimes a certain high rate variable is recommended for fluxes.  For example, you may want to use MRLA1 rather than MRLA for mixing ratio after reading /worker/twelve/coastal_waves/coastal_doc/110_QA.txt.   You can learn about the instruments that measure high rate data at http://raf.atd.ucar.edu/Bulletins/bulletin9.html .  You should check in http://raf.atd.ucar.edu/Bulletins/bulletin3.html if you have questions about how fast an instrument's response if for measuring fluxes.
3. What do I need to know in choosing the start and end of a high-rate run?
1. Is this time period long enough for a reliable estimate?   I usually use an averaging interval of 2 minutes for my flux estimates,  so the length of the time period must be greater.
2. Do the mean conditions stay fairly steady during this run?   If you can't define a mean to remove from your time series, then your estimate will be flawed.  Perhaps you can choose a segment of the run during which things were steady.
4. What do I need to know in choosing an averaging interval for my flux estimate?
1. I want to choose the averaging interval that will include all the contributions made by high frequency motions, ie the fluxes carried by the smaller eddies.  That would suggest choosing a long averaging time.  However, I also want a lot of flux estimates, which would mean a short averaging time.  As well, I don't want the contribution to the fluxes of mean or large-scale motions, so my averaging time can't be too long.    So there' s a tradeoff.
2. You can calculate an ogive curve, which is a running sum of the contributions from high to low frequencies.  As you move towards lower frequencies, the ogive should level off because the low frequencies aren't contributing much.  If this is not the case, you may need to change the averaging interval.
3. Where can I read about how to use an ogive to choose an averaging interval?
1. Enriquez thesis, introduction
2. Berger thesis
3. Oncley et al, "surface layer fluxes, profiles, and turbulence measurements over uniform terrain under near-neutral conditions."  J. of the Atmospheric Sciences 53(7), April 1996, pp 1029-1044.

Meteorology data availability on "coast" (not updated or comprehensive)

Recommended reading

1. What are the cutting-edge problems in coastal meteorology?  An overview is found in:  Coastal meteorology : a review of the state of the science / Panel on  Coastal Meteorology, Committee on Meteorological Analysis, Prediction, and Research  Washington, D.C. : National Academy Press, 1992.  Find this book at the library under: UCSD  Scripps QC994 .N37 1992 Flr 3, or David may have a copy since he was on the panel

1. What are current issues people are working on in stably stratified boundary layers?

The whole issue of Boundary Layer Meteorology volume 90 in 1999 (don't know month) is devoted to the stably stratified boundary layer.  I especially recommend the "Summary of the ...Workshop" (p. 345) for a recent overview of the field, and L. Mahrt's article (p. 375).

1. To learn more about what has been done with the Coastal Waves 96 data:

Get the preprint volume from the 12th symposium on boundary layers and turbulence, Vancouver BC, August 1997.  There were several talks given based on Coastal Waves 96 data.  Ask David or I for this volume.

1. To learn about the boundary layer height and structure along California, read section 4 of the following report.  To learn about what might cause the inversion to form, read section 5:

Studies of the structure of the atmosphere over the eastern Pacific Ocean in summer, by Morris Neiburger, David S. Johnson, and Chen-Wu Chien. Berkeley, University of California Press, University of California publications in meteorology, v. 1, no. 1.
 
2. To read about what the Coastal Ocean Dynamic Experiment found out about the 3D structure of the boundary layer off California, read:
1. Local atmospheric forcing during the Coastal Ocean Dynamics Experiment, Pt. 1, Description of the marine boundary layer and atmospheric conditions over a northern California upwelling region Beardsley, R. C.  Journal of Geophysical Research, Wash., D.C., 92(C2): 1467-1488, Feb. 15, 1987.
1. Enriquez, Amelito Gomez, (dissertation). Variability of the surface wind and ocean circulation over the northern  California shelf  1994.  You can copy this from me, or get the papers he wrote with his advisor Carl Friehe out of the library.  Also, a great description of how to process aircraft data.
3. To learn more about the jet dynamics:
1. The summertime low-level jet and marine boundary layer structure along the California coast

Burk, Stephen D. -- Thompson, William T.
Naval Research Laboratory, Marine Meteorology Division, Monterey, CA
Monthly Weather Review, Boston, MA, 124(4): 668-686, April 1996.
 
2. Mesoscale forcing of a boundary layer jet along the California coast

Holt, Teddy R.
Naval Postgraduate School, Monterey, CA
Journal of Geophysical Research, Washington, DC, 101(D2): 4235-4254, February 20, 1996.
4. To learn about David Rogers' internal boundary layers work:
1. The stable internal boundary layer over a coastal sea. Part I: Airborne measurements of the mean and

tubulence structure
Rogers, David P. -- Johnson, Douglas W. -- Friehe, Carl A.
Journal of the Atmospheric Sciences, Boston, MA, 52(6): 667-683, March 15, 1995.
 
2. The stable internal boundary layer over a coastal sea. Part II: Gravity waves and the momentum balance

Rogers, David P. -- Johnson, Douglas W. -- Friehe, Carl A.
Journal of the Atmospheric Sciences, Boston, MA, 52(6): 684-696, March 15, 1995.
 
3. Ask Ian  or David for a copy of the manuscript they wrote on internal boundary layers.

 
5. To learn about how the boundary layer reacts to changes in its surface forcing:
1. Air-sea fluxes and surface layer turbulence around a sea surface temperature front

Friehe, C. A. -- Shaw, W. J. -- Rogers, D. P. -- Davidson, K. L. -- Large, W. G. -- Stage, S. A. -- Crescenti, G. H. -- Khalsa, S. J. S. -- Greenhut,
G. K. -- Li, F.
Journal of Geophysical Research, Washington, DC, 96(C5): 8593-8609, May 15, 1991.
 
2. Structure of the marine atmospheric boundary layer over an oceanic thermal front: SEMAPHORE

experiment
Kwon, B. H. -- Benech, B. -- Lambert, D. -- Durand, P. -- Druilhet, A. -- Giordani, H. -- Planton, S.
Journal of Geophysical Research, Washington, DC, 103(C11): 25159-25180, October 15, 1998.
Also peek at the other articles on the SEMAPHORE experiment in this volume.  Dominique Lamert is a recent grad who came along on the Coastal Waves 96 experiment.
 
3. Atmospheric turbulence structure in the vicinity of an oceanic front

Khalsa, Siri Jodha Singh -- Greenhut, Gary K.
Journal of Geophysical Research, Wash., D.C., 94(C4): 4913-4922, April 15, 1989.




---

Useful information

Where to find articles?

1. Meteorological abstracts can be searched at http://www.mganet.org/basesearch.shtml
2. The list of electronic journal and abstract indexes available at SIO:  click on "electronic journals" under  http://scilib.ucsd.edu/sio/.

How to get computer help?

1. Matlab
1. Search Matlab's support site at  http://www.mathworks.com/support/
2. Type "helpdesk" at your matlab prompt, and you will get a web-driven help window.
3. If you really get stuck and no one here can help you, you can email their support people at support@mathworks.com
2. VI, the text editor
1. A guide to VI commands can be found at http://www.cs.umr.edu/unixinfo/general/packages/viguide.html

Where to get useful UCSD or SIO infomation?

1. Course schedule: https://www-act.ucsd.edu/cgi-bin/studentlink.pl/1/

How to construct a web page to archive plots

First, you have to set up the directories where the web pages and plots will reside.

1. Create a directory /home/jessica/public_html where you will store your web pages.  Go into the directory /home/jessica/public_html.  Type the following at the unix prompt, which will make the directory readable to the web browsers:   chmod o+rx .
2. Create a directory /home/jessica/plots where you will store the plots you show on the web page
3. Copy the following file into /home/jessica/public_html  : /home/kate/jessica/index.html   This is going to be your main web page.  If you want, you can view it at http://www-ccs.ucsd.edu/~jessica once you have copied it into your directory.
4. Put some postscript plots, like map.ps, in the directory /home/jessica/public_html/plots.  Web browsers can't read postscript files, so you must convert them to a format called .gif which the browser can read.  To convert the postscript file map.ps to the gif file map.gif, you would type the following command at the unix prompt within the directory /home/jessica/public_html/plots:

convert -colorspace RGB -geometry 1200x600 map.ps map.gif
If you ever want to look at a .gif file from the unix command line, type  xv map.gif  .  You will be able to see the file thought the colors are often weird.  To quit out of xv click on the window with your right mouse button, and you will get a menu.

1. Go to the web page http://www-ccs.ucsd.edu/~jessica  in netscape.
2. In the upper left hand corner of your net scape window, choose "Edit Page" from the "File" menu.  This will open up a "Netscape Composer" window for you, which you can edit.  It's a lot like word processing, you just type in the text or choose things from the menus to insert,etc.
3. To save the edited web page, choose "Save As" from the "File" menu in the upper left corner of your Netscape Composer window.  Be sure it is in the correct directory.  The default directory is almost always wrong.  Save fairly often, since Compser crashes frequently.

1. Link to another web page:  Highlight the text that you would like to be linked to the other web page. Choose "Link" from the "Insert" menu.  Type in the location of the link.  Composer often gets it wrong, in which case edit /home/jessica/public_html/index.html using vi.
2. Link to an image, like a plot (has to be .gif or .jpeg file):  Highlight the text that you would like to be linked to the image.  Choose "Link" from the "Insert" menu.  Type in the location of the gif file.  Note that, according to netscape, all files in your directory such as /home/jessica/public_html/plots/map.gif should be entered as http://www-ccs.ucsd.edu/~jessica/plots/map.gif.  Sometimes netscape messes this up.  If the link doesn't work, you will need to edit /home/jessica/index.html with vi  so that the link name is correct.
3. Insert an image:  Choose "Image" from the "Insert" menu in Composer.  Type in the file name.  Again, /home/jessica/public_html/plots/map.gif should be entered as http://www-ccs.ucsd.edu/~jessica/plots/map.gif.  You may need to edit the page in vi if Netscape messes up the file name.  You can choose the image size, but you can also just change it in vi (the link should include "Height" and "Width" settings which you could manually change.

1. If you get a lot of .gifs in the directory, they take up a lot of room, and you might want to check with a coast systems admin if you are hogging disk space.  Also, I wouldn't keep .ps and .gif copies of every file--the postscript files are quite large, and if it's just for your reference, keep the .gif file only.



---

Reading netcdf files into matlab (version 5 and higher)

1. Put the following lines in your startup.m file, so that matlab can access the netcdf commands:  path(path,'/apps/matlab53/toolbox/ccs/netcdf/') ;

path(path,'/apps/matlab53/toolbox/ccs/netcdf/nctype/') ;
path(path,'/apps/matlab53/toolbox/ccs/netcdf/ncutility/') ;
2. Open up matlab and change to the directory the netcdf file is in.
3. To see what variables are in the file /home/kate/back_trajectories/cw96_pressure_level.cdf, for example, you would type

ncdump('cw96_pressure_level.cdf') ;
This would list each variable in the file, their units, who created the file, etc.
To see an example of a file which opens cw96_pressure_level.cdf, extracts some variables from it, and plots them, look at /home/kate/back_trajectories/read_pressure_level_cdf.m.
You will have to make your own version of this file which does not try to print and read files from the directory /home/kate/back_trajectories.

The data on this page is unpublished. If it is used please cite the author Kathleen Edwards, the Center for Coastal Studies, and the Coastal Waves group at Scripps Insitution of Oceanography.