Difference between revisions of "Generating SQW files"

From Horace
Jump to: navigation, search
(accumulate_sqw)
 
(12 intermediate revisions by 3 users not shown)
Line 1: Line 1:
This page tells you how to generate an SQW file when you have a full dataset already. It also details how to accumulate data files into an SQW file as you go (e.g. during an experiment).
+
This page tells you how to generate an SQW file. There are two different situations when you will want to do this:
 +
*During an experiment, when you want to accumulate data files into an SQW file as they are collected - use <code>accumulate_sqw</code>
 +
*When you have a full set of data files already that you want to process in one go - use <code>gen_sqw</code>
 +
The two functions have almost identical syntax, as is explained in the sections below.
  
 
To generate the SQW file neutron data for each individual run needs to be provided in one of two formats: the legacy ASCII format SPE file, together with an ASCII detector parameter file (the PAR file), or their replacements the HDF (hierarchical Data Format) NXSPE file. More details about these files and how to create them can be found [[Input file formats|here]]
 
To generate the SQW file neutron data for each individual run needs to be provided in one of two formats: the legacy ASCII format SPE file, together with an ASCII detector parameter file (the PAR file), or their replacements the HDF (hierarchical Data Format) NXSPE file. More details about these files and how to create them can be found [[Input file formats|here]]
Line 10: Line 13:
  
 
<pre>
 
<pre>
spe_dir='/home/maps/maps_users/Gruenwald/SPE/';%directory where spe files are found
+
spe_dir='/home/maps/maps_users/Gruenwald/SPE/'; % directory where spe files are found
runno=[19780:19960];%anticipated list of files (we do not need all of them to exist yet)
+
runno=[19780:19960]; % anticipated list of files (we do not need all of them to exist yet)
  
 
spe_file=cell(1,numel(runno));
 
spe_file=cell(1,numel(runno));
 
for i=1:numel(runno)
 
for i=1:numel(runno)
     spe_file{i}=[spe_dir,'map',num2str(runno(i)),'_ei100.spe'];%filename of runs
+
     spe_file{i}=[spe_dir,'map',num2str(runno(i)),'_ei100.spe']; % filenames of runs
 
end
 
end
  
psi=[0:2:180 1:2:179];%list of anticipated scan angles
+
psi=[0:2:180 1:2:179]; % list of anticipated scan angles
  
par_file='/usr/local/mprogs/Libisis/InstrumentFiles/maps/4to1_124.par';%detector parameter file
+
par_file='/usr/local/mprogs/Libisis/InstrumentFiles/maps/4to1_124.par'; % detector parameter file
  
sqw_file='/home/maps/maps_users/Gruenwald/data_accumulation.sqw';%name of output file
+
sqw_file='/home/maps/maps_users/Gruenwald/data_accumulation.sqw';       % name of output file
  
efix=100;%incident energy
+
efix=100; % incident energy
emode=1;%indicates direct geometry
+
emode=1;   % indicates direct geometry
  
alatt=[5.7,5.7,5.7];%lattice parameters
+
alatt=[5.7,5.7,5.7]; % lattice parameters
angdeg=[90,90,90];%lattice angles
+
angdeg=[90,90,90];   % lattice angles
  
u=[1,1,0]; v=[0,0,1]; %orientation of sample (u//ki, v perp ki in horizontal plane)
+
u=[1,1,0]; v=[0,0,1]; % orientation of sample (u//ki when psi=0, v another vector in horizontal plane)
  
omega=0; dpsi=0; gl=0; gs=0;%offset angles for sample misalignment
+
omega=0; dpsi=0; gl=0; gs=0; % offset angles for sample misalignment
  
accumulate_sqw (spe_file, par_file, sqw_file, efix, emode, alatt, angdeg,...
+
accumulate_sqw(spe_file, par_file, sqw_file, efix, emode, alatt, angdeg,...
 
     u, v, psi, omega, dpsi, gl, gs, <optional input parameters>);
 
     u, v, psi, omega, dpsi, gl, gs, <optional input parameters>);
 
</pre>
 
</pre>
Line 48: Line 51:
 
<code>emode</code> is either 1 for direct geometry instruments, or 2 for indirect geometry.
 
<code>emode</code> is either 1 for direct geometry instruments, or 2 for indirect geometry.
  
<code>alatt</code> is a vector with 3 elements, specifying the lengths in Angstroms of the crystal's lattice parameters.
+
<code>alatt</code> is a vector with 3 elements, specifying the lengths in Angstroms of the crystal lattice parameters.
  
<code>angdeg</code> is a vector with 3 elements, specifying the lattice angles in degrees.
+
<code>angdeg</code> is a vector with 3 elements, specifying the crystal lattice angles in degrees.
  
<code>u</code> and <code>v</code> are both 3-element vectors. These specify how the crystal's axes were oriented relative to the spectrometer in the setup for which you define <code>psi</code> to be zero. <code>u</code> specifies the lattice vector that is parallel to the incident neutron beam, whilst <code>v</code> is a vector perpendicular to this in the horizontal plane.
+
<code>u</code> and <code>v</code> are both 3-element vectors. These specify how the crystal's axes were oriented relative to the spectrometer in the setup for which you define <code>psi</code> to be zero. <code>u</code> specifies the lattice vector that is parallel to the incident neutron beam, whilst <code>v</code> is a second vector in the horizontal plane. It is not necessary for <code>v</code> to be perpendicular to <code>u</code>.
  
 
<code>psi</code> specifies the angle of the crystal relative to the setup described in the above paragraph (i.e. the angle about the vertical axis through which the sample has been rotated). If a single orientation of the crystal was used for all measurements then this number can be a scalar, otherwise it is a vector. For the case of it use in <code>accumulate_sqw</code> it is a vector listing the '''expected''' values of <code>psi</code> that will be used. It is important to get this about right, as it ensures that the underlying reciprocal space grid in the SQW file is big enough to encompass all of the data you plan to collect. If it is not, then you lose all the time-saving and the file has to be generated from scratch!
 
<code>psi</code> specifies the angle of the crystal relative to the setup described in the above paragraph (i.e. the angle about the vertical axis through which the sample has been rotated). If a single orientation of the crystal was used for all measurements then this number can be a scalar, otherwise it is a vector. For the case of it use in <code>accumulate_sqw</code> it is a vector listing the '''expected''' values of <code>psi</code> that will be used. It is important to get this about right, as it ensures that the underlying reciprocal space grid in the SQW file is big enough to encompass all of the data you plan to collect. If it is not, then you lose all the time-saving and the file has to be generated from scratch!
Line 62: Line 65:
 
The optional input arguments are as follows:
 
The optional input arguments are as follows:
  
<code>grid_size_in</code>: A scalar or row vector of grid dimensions. The default, if not given or [], is is [50,50,50,50]
+
<code>grid_size_in</code>: A scalar or row vector of grid dimensions. If it is not given, or is left blank (i.e. set to []), the default value will be determined on the number and size of the contributing SPE or NXSPE files.
  
<code>urange_in</code>: The range of data grid for output as a 2x4 matrix - [x1_lo,x2_lo,x3_lo,x4_lo;x1_hi,x2_hi,x3_hi,x4_hi]. The default if not given or [] is the smallest hypercuboid that encloses the whole data range.
+
<code>urange_in</code>: The range of data grid for output along each Q and E direction as a 2x4 matrix - [x1_lo,x2_lo,x3_lo,x4_lo;x1_hi,x2_hi,x3_hi,x4_hi]. The default if not given or set to [] is the smallest hypercuboid that encloses the whole data range.
  
<code>instrument</code>: A structure or object containing instrument information [scalar or array length nfile]
+
<code>instrument</code>: A free-format structure or object containing instrument information [scalar or array length nfile]
  
<code>sample</code>: A structure or object containing sample geometry information [scalar or array length nfile]
+
<code>sample</code>: A free-format structure or object containing sample geometry information [scalar or array length nfile]
 
   
 
   
<code>'clean'</code>: Create the sqw file from fresh. This option deletes existing sqw file (if any) and forces fresh generation of sqw file from the list of data files provided. It is possible to get confused about what  data has been included in an sqw file if it is built up slowly over an experiment. Use this option to start afresh.
+
<code>'replicate'</code>: Normally the function forbids an SPE or NXSPE file from appearing more than once. This is to trap common typing errors. However, sometimes you might want to create an sqw file using, for example, just one SPE file as the source of data for all crystal orientations in order to construct a background from an empty piece of sample environment. In this case, use the keyword 'replicate' to override the uniqueness check.
+
 
<code>'replicate'</code>: Normally the function forbids an spe file from appearing more than once. This is to trap common typing errors. However, sometimes you might want toto create an sqw file using, for example, just one spe file as the source of data for all crystal orientations in order to construct a background from an empty piece of sample environment. In this case, use the keyword 'replicate' to override the uniqueness check.
+
<code>'clean'</code>: Create the SQW file from fresh. This option deletes existing SQW file (if any) and forces fresh generation of SQW file from the list of data files provided. It is possible to get confused about what  data has been included in an SQW file if it is built up slowly over an experiment. Use this option to start afresh.
  
 
==gen_sqw==
 
==gen_sqw==
  
This is the main function you will use to turn the data accumulated in multiple SPE files into a single SQW file that will be used by the rest of the Horace functions. An introduction to its use is given in the  [[Getting_started#Creating_an_SQW_file|getting started]] section. Here we list simply the format of the input and output arguments, and give a brief description of their meanings.
+
This is the main function you will use to turn the data accumulated in multiple SPE files into a single SQW file that will be used by the rest of the Horace functions. An introduction to its use is given in the  [[Getting_started#Creating_an_SQW_file|getting started]] section. The syntax is the same as for <code>accumulate_sqw</code>; the only difference is that you give a list of existing input datasets rather than the anticipated list.
  
 
The essential inputs take the following form:
 
The essential inputs take the following form:
Line 86: Line 89:
  
  
There are additional (optional) input and output arguments as with <code>accumulate_sqw</code>:
+
There are additional (optional) input and output arguments, just as for <code>accumulate_sqw</code>:
  
 
<pre>
 
<pre>
Line 93: Line 96:
 
</pre>
 
</pre>
  
<code>grid_size_in</code> is a row vector with 4 elements, each of which specifies the number of blocks of data along each Q/E direction into which the pixels will be grouped in each intermediate tmp files. If this argument is left blank then a default 50x50x50x50 grid is used. This 'coarse graining' is to speed up the process of reading data from file - it '''does not affect the resolution of your data'''.
+
Optional input arguments:
  
<code>urange_in</code> is a 2x4 matrix which specifies the the upper and lower limits along each Q/E direction. If this is left blank then the program will calculate the range of a hypercuboid that encompasses all of the data along all 4 directions.
+
<code>grid_size_in</code>: A scalar or row vector of grid dimensions. If it is not given, or is left blank (i.e. set to []), the default value will be determined on the number and size of the contributing SPE or NXSPE files.
  
<code>grid_size</code> is a vector with 4 elements which specifies the actual grid size required. e.g. if every data point has the same value of Qz then the third element will be 1.
+
<code>urange_in</code>: The range of data grid for output along each Q and E direction as a 2x4 matrix - [x1_lo,x2_lo,x3_lo,x4_lo;x1_hi,x2_hi,x3_hi,x4_hi]. The default if not given or set to [] is the smallest hypercuboid that encloses the whole data range.
  
<code>'replicate'</code>: Normally the function forbids an spe file from appearing more than once. This is to trap common typing errors. However, sometimes you might want to create an sqw file using, for example, just one spe file as the source of data for all crystal orientations in order to construct a background from an empty piece of sample environment. In this case, use the keyword 'replicate' to override the uniqueness check.
+
<code>instrument</code>: A free-format structure or object containing instrument information [scalar or array length nfile]
 +
 
 +
<code>sample</code>: A free-format structure or object containing sample geometry information [scalar or array length nfile]
 +
 +
<code>'replicate'</code>: Normally the function forbids an SPE or NXSPE file from appearing more than once. This is to trap common typing errors. However, sometimes you might want to create an sqw file using, for example, just one SPE file as the source of data for all crystal orientations in order to construct a background from an empty piece of sample environment. In this case, use the keyword 'replicate' to override the uniqueness check.
 +
 
 +
Optional output arguments:
 +
 
 +
<code>tmp_file</code>: A cell array containing the full file names of the temporary files that were created by <code>gen_sqw</code>. These will be deleted if the function ran correctly, but if there was a problem, then they will still exist and it can be useful to know their names so that they can be deleted manually.
 +
 
 +
<code>grid_size</code> is a vector with 4 elements which specifies the actual grid size of the output SQW file that was created. For example, if every data point has the same value of Qz then the third element will be 1.
  
 
<code>urange</code> gives the range in reciprocal space of the data. If <code>urange_in</code> was specified then this will be the same, but if not then it tells you the calculated range of the 4-dimensional hypercuboid which encompasses all of the data.
 
<code>urange</code> gives the range in reciprocal space of the data. If <code>urange_in</code> was specified then this will be the same, but if not then it tells you the calculated range of the 4-dimensional hypercuboid which encompasses all of the data.
 +
 +
== Applying symmetry operations to an entire dataset ==
 +
 +
In the explanation below, we wish to apply symmetrisation to the entire data file. Under the hood, what happens is that the data for each run is symmetrised, and then these symmetrised data are combined to make the sqw file. This avoids the problem of running out of memory when attempting to symmetrise large sections of the unfolded sqw file / object.
 +
 +
To use this functionality, call <code>gen_sqw</code> or <code>accumulate_sqw</code> as above, with the following additional arguments:
 +
 +
<pre>
 +
gen_sqw (spefile, par_file, sym_sqw_file, efix, emode, alatt, angdeg,...
 +
    u, v, psi, omega, dpsi, gl, gs,'transform_sqw',@(x)(symmetrise_sqw(x,v1,v2,v3)))
 +
</pre>
 +
 +
or more generally
 +
 +
<pre>
 +
gen_sqw (spefile, par_file, sym_sqw_file, efix, emode, alatt, angdeg,...
 +
    u, v, psi, omega, dpsi, gl, gs,'transform_sqw',@(x)(user_symmetrisation_routine(x))
 +
</pre>
 +
 +
 +
The first example above would build a sqw file reflected as in the example for the reflection in memory, but with the transformation applied to the entire dataset. In the second, more general, case the user defined function (in a m-file on the Matlab path) can define multiple symmetrisation operations that are applied sequentially to the entire data. An example is as follows, which folds a cubic system so that all six of the symmetrically equivalent (1,0,0) type positions are folded on to each other:
 +
 +
<pre>
 +
function wout = user_symmetrisation_routine(win)
 +
 +
wout=symmetrise_sqw(win,[1,1,0],[0,0,1],[0,0,0]);  % fold about line (1,1,0) in HK plane
 +
wout=symmetrise_sqw(wout,[-1,1,0],[0,0,1],[0,0,0]); % fold about line (-1,1,0) in HK plane
 +
wout=symmetrise_sqw(wout,[1,0,1],[0,1,0],[0,0,0]);  % fold about line (1,0,1) in HL plane
 +
wout=symmetrise_sqw(wout,[1,0,-1],[0,1,0],[0,0,0]); % fold about line (1,0,-1) in HL plane
 +
</pre>
 +
 +
see very importent notes on the technical details of symmeterizing a whole dataset in the [http://horace.isis.rl.ac.uk/Symmetrising_etc#Symmetrising_whole_data_files chapter, related to symmetrization]

Latest revision as of 14:30, 16 January 2019

This page tells you how to generate an SQW file. There are two different situations when you will want to do this:

  • During an experiment, when you want to accumulate data files into an SQW file as they are collected - use accumulate_sqw
  • When you have a full set of data files already that you want to process in one go - use gen_sqw

The two functions have almost identical syntax, as is explained in the sections below.

To generate the SQW file neutron data for each individual run needs to be provided in one of two formats: the legacy ASCII format SPE file, together with an ASCII detector parameter file (the PAR file), or their replacements the HDF (hierarchical Data Format) NXSPE file. More details about these files and how to create them can be found here


accumulate_sqw

This is a way of generating data 'on the fly' during an experiment. It saves time by appending new data to an already existing SQW file.

The syntax is as follows:

spe_dir='/home/maps/maps_users/Gruenwald/SPE/';  % directory where spe files are found
runno=[19780:19960];  % anticipated list of files (we do not need all of them to exist yet)

spe_file=cell(1,numel(runno));
for i=1:numel(runno)
    spe_file{i}=[spe_dir,'map',num2str(runno(i)),'_ei100.spe'];  % filenames of runs
end

psi=[0:2:180 1:2:179];  % list of anticipated scan angles

par_file='/usr/local/mprogs/Libisis/InstrumentFiles/maps/4to1_124.par';  % detector parameter file

sqw_file='/home/maps/maps_users/Gruenwald/data_accumulation.sqw';        % name of output file

efix=100;  % incident energy
emode=1;   % indicates direct geometry

alatt=[5.7,5.7,5.7];  % lattice parameters
angdeg=[90,90,90];    % lattice angles

u=[1,1,0]; v=[0,0,1]; % orientation of sample (u//ki when psi=0, v another vector in horizontal plane)

omega=0; dpsi=0; gl=0; gs=0;  % offset angles for sample misalignment

accumulate_sqw(spe_file, par_file, sqw_file, efix, emode, alatt, angdeg,...
    u, v, psi, omega, dpsi, gl, gs, <optional input parameters>);

The input parameters are defined as follows:

spe_file is a cell array, each element of which is a string specifying the full file name of the input SPE or NXSPE files (e.g. spe_file{1} = 'C:\data\mer12345.spe').

par_file is a string giving the full file name of the parameter file for the instrument on which the data were taken. This is required for SPE files. For NXSPE files you do not need to specify an instrument parameter file (Provide empty string '' instead), as detector information will be picked up from NXSPE files themselves. If you do specify par_file, the detector info is taken from there and overrides the information contained within the NXSPE files. sqw_file is a string giving the full file name of the sqw output file you wish to generate.

efix is the incident neutron energy for each SPE file. If a single incident energy was used for all runs then this number is a scalar, otherwise it must be a vector with the same number of elements are there are SPE files.

emode is either 1 for direct geometry instruments, or 2 for indirect geometry.

alatt is a vector with 3 elements, specifying the lengths in Angstroms of the crystal lattice parameters.

angdeg is a vector with 3 elements, specifying the crystal lattice angles in degrees.

u and v are both 3-element vectors. These specify how the crystal's axes were oriented relative to the spectrometer in the setup for which you define psi to be zero. u specifies the lattice vector that is parallel to the incident neutron beam, whilst v is a second vector in the horizontal plane. It is not necessary for v to be perpendicular to u.

psi specifies the angle of the crystal relative to the setup described in the above paragraph (i.e. the angle about the vertical axis through which the sample has been rotated). If a single orientation of the crystal was used for all measurements then this number can be a scalar, otherwise it is a vector. For the case of it use in accumulate_sqw it is a vector listing the expected values of psi that will be used. It is important to get this about right, as it ensures that the underlying reciprocal space grid in the SQW file is big enough to encompass all of the data you plan to collect. If it is not, then you lose all the time-saving and the file has to be generated from scratch!

omega, dpsi, gl, and gs specify the offsets (in degrees of various angles). gl and gs describe the settings of the large and small goniometers. omega is the offset of the axis of the small goniometer with respect to the notional u. Finally dpsi allows you to specify an offset in psi, should you wish. These angle definitions are shown below:

Virtual goniometer angle definitions

The optional input arguments are as follows:

grid_size_in: A scalar or row vector of grid dimensions. If it is not given, or is left blank (i.e. set to []), the default value will be determined on the number and size of the contributing SPE or NXSPE files.

urange_in: The range of data grid for output along each Q and E direction as a 2x4 matrix - [x1_lo,x2_lo,x3_lo,x4_lo;x1_hi,x2_hi,x3_hi,x4_hi]. The default if not given or set to [] is the smallest hypercuboid that encloses the whole data range.

instrument: A free-format structure or object containing instrument information [scalar or array length nfile]

sample: A free-format structure or object containing sample geometry information [scalar or array length nfile]

'replicate': Normally the function forbids an SPE or NXSPE file from appearing more than once. This is to trap common typing errors. However, sometimes you might want to create an sqw file using, for example, just one SPE file as the source of data for all crystal orientations in order to construct a background from an empty piece of sample environment. In this case, use the keyword 'replicate' to override the uniqueness check.

'clean': Create the SQW file from fresh. This option deletes existing SQW file (if any) and forces fresh generation of SQW file from the list of data files provided. It is possible to get confused about what data has been included in an SQW file if it is built up slowly over an experiment. Use this option to start afresh.

gen_sqw

This is the main function you will use to turn the data accumulated in multiple SPE files into a single SQW file that will be used by the rest of the Horace functions. An introduction to its use is given in the getting started section. The syntax is the same as for accumulate_sqw; the only difference is that you give a list of existing input datasets rather than the anticipated list.

The essential inputs take the following form:

gen_sqw (spe_file, par_file, sqw_file, efix, emode, alatt, angdeg,...
                                                u, v, psi, omega, dpsi, gl, gs);


There are additional (optional) input and output arguments, just as for accumulate_sqw:

[tmp_file,grid_size,urange] = gen_sqw (spe_file, par_file, sqw_file, efix, emode, alatt, angdeg,...
                                                u, v, psi, omega, dpsi, gl, gs, grid_size_in, urange_in,'replicate');

Optional input arguments:

grid_size_in: A scalar or row vector of grid dimensions. If it is not given, or is left blank (i.e. set to []), the default value will be determined on the number and size of the contributing SPE or NXSPE files.

urange_in: The range of data grid for output along each Q and E direction as a 2x4 matrix - [x1_lo,x2_lo,x3_lo,x4_lo;x1_hi,x2_hi,x3_hi,x4_hi]. The default if not given or set to [] is the smallest hypercuboid that encloses the whole data range.

instrument: A free-format structure or object containing instrument information [scalar or array length nfile]

sample: A free-format structure or object containing sample geometry information [scalar or array length nfile]

'replicate': Normally the function forbids an SPE or NXSPE file from appearing more than once. This is to trap common typing errors. However, sometimes you might want to create an sqw file using, for example, just one SPE file as the source of data for all crystal orientations in order to construct a background from an empty piece of sample environment. In this case, use the keyword 'replicate' to override the uniqueness check.

Optional output arguments:

tmp_file: A cell array containing the full file names of the temporary files that were created by gen_sqw. These will be deleted if the function ran correctly, but if there was a problem, then they will still exist and it can be useful to know their names so that they can be deleted manually.

grid_size is a vector with 4 elements which specifies the actual grid size of the output SQW file that was created. For example, if every data point has the same value of Qz then the third element will be 1.

urange gives the range in reciprocal space of the data. If urange_in was specified then this will be the same, but if not then it tells you the calculated range of the 4-dimensional hypercuboid which encompasses all of the data.

Applying symmetry operations to an entire dataset

In the explanation below, we wish to apply symmetrisation to the entire data file. Under the hood, what happens is that the data for each run is symmetrised, and then these symmetrised data are combined to make the sqw file. This avoids the problem of running out of memory when attempting to symmetrise large sections of the unfolded sqw file / object.

To use this functionality, call gen_sqw or accumulate_sqw as above, with the following additional arguments:

gen_sqw (spefile, par_file, sym_sqw_file, efix, emode, alatt, angdeg,...
    u, v, psi, omega, dpsi, gl, gs,'transform_sqw',@(x)(symmetrise_sqw(x,v1,v2,v3)))

or more generally

gen_sqw (spefile, par_file, sym_sqw_file, efix, emode, alatt, angdeg,...
    u, v, psi, omega, dpsi, gl, gs,'transform_sqw',@(x)(user_symmetrisation_routine(x))


The first example above would build a sqw file reflected as in the example for the reflection in memory, but with the transformation applied to the entire dataset. In the second, more general, case the user defined function (in a m-file on the Matlab path) can define multiple symmetrisation operations that are applied sequentially to the entire data. An example is as follows, which folds a cubic system so that all six of the symmetrically equivalent (1,0,0) type positions are folded on to each other:

function wout = user_symmetrisation_routine(win)

wout=symmetrise_sqw(win,[1,1,0],[0,0,1],[0,0,0]);   % fold about line (1,1,0) in HK plane
wout=symmetrise_sqw(wout,[-1,1,0],[0,0,1],[0,0,0]); % fold about line (-1,1,0) in HK plane
wout=symmetrise_sqw(wout,[1,0,1],[0,1,0],[0,0,0]);  % fold about line (1,0,1) in HL plane
wout=symmetrise_sqw(wout,[1,0,-1],[0,1,0],[0,0,0]); % fold about line (1,0,-1) in HL plane

see very importent notes on the technical details of symmeterizing a whole dataset in the chapter, related to symmetrization