Transforming ENSEMBLE measurements

In an email exchange, Nancy Morrison encouraged me to transform my measurements. I took this as a challenge, and I started reading everything I could find on the subject. An old post by Arne Hendon and the on-line help file for Transform Applier in VPHOT were the best sources. By trial-and-error, I taught myself to use TA in VPHOT to transform my ENSEMBLE measurements. I’ve spent several hours over a couple of days working on my Tyc 3224-2602-1 data to produce an AAVSO report suitable for input to TA. I used MPO Canopus/PhotoRed to make a preliminary AAVSO report. As I usually do, I plotted the points and removed obvious outliers. Then I went back to MPO Canopus to create an observations list with all of the measurements expressed in JD. These two files were combined in a spreadsheet and the data for each JD were aligned by removing the observations that were not included in the AAVSO report. I then edited the report to include CREF instead of ENSEMBLE in the CNAME field, and the average of the comp mags in the CMAG field. Finally, I grouped the measurements into BVRI groups (there were 305 groups). The report file was divided into three parts and saved into csv files with about 200 or fewer groups in each. These were uploaded to VPHOT and TA was used to produce transformed magnitudes. This procedure produced transformed mags with essentially the same uncertainties as the original values. What I found in every case was that the difference between the untransformed and transformed magnitudes was less than the uncertainties in the measurements. I’ve included a table below showing the minimum, maximum, and average values comparing the untransformed and transformed magnitudes. Except for the educational value, the whole exercise seems to me to be unnecessary.

Allen

untransformed - transformed mag uncertainty in transformed mag Dif. In unc. With transformation.
B V R I B V R I B V R I
max 0.009 0.006 0.029 -0.001 0.076 0.061 0.072 0.106 0 0 0.013 0.018
min 0.006 0.004 0.018 -0.006 0.006 0.004 0.01 0.016 0 0 0 0
average 0.007 0.005 0.024 -0.005 0.020 0.015 0.031 0.041 0.000 0.000 0.000 0.000

Average Color of Ensemble Comps versus Color of Target?

Allen:

What was the average color of the comps in the ensemble versus the color of the target? IF the color difference was very small, (i.e., color ensemble - color target =0), the magnitude difference with and without transformation would also be very small (0).

IF the selected comp and target have the same color, the color correction resulting from transformation for your non-standard filter is zero! IOW, if you select a comp that has the same color as the target, no transformation is necessary!

Of course, IF that condition is NOT true, transformation will yield significant differences/corrections. For example, very red Mira LPVs virtually always need transformation to yield a ‘true’ magnitude.

Ken

Ken, Thanks for responding. The target and comp information is listed below.

  star                    RA             Dec             V mag  B-V

|T|Tyc 3224-2602-1|23 01 41.8|+42 02 52.7|11.692|0.371|
|C1|GSC 3224-1284|23 02 42.3|+42 04 18.1|12.721|0.928|
|C2|GSC 3224-1898|23 03 18.5|+41 58 16.2|13.488|0.578|
|C3|GSC 3224-873|23 02 45.0|+41 57 15.5|12.344|1.120|
|C4|GSC 3224-650|23 02 21.4|+42 00 55.8|13.165|0.599|
|C5|GSC 3224-1882|23 01 52.9|+42 06 24.3|13.212|0.794|

The average V mag of the comps is 12.986, and the average B-V is 0.804, compared to 11.692 and 0.371 for the target star.

Allen:

So the delta color (target - avg comp) is ~ 0.43 if you used equal weighting. Not exactly the same! So, need to look further. What are your corresponding transform coeffs? How close to 1 and 0?

The transform equation is as follows (you can alter equation for each filter/bandpass). You should be able to calculate how much correction is expected?

Ken

Ken,
My system transform coefficients are:

#Tbv= 0.9980 +/- 0.0100
#Tbr= 1.0250 +/- 0.0090
#Tbi= 0.9830 +/- 0.0110
#Tvr= 1.0990 +/- 0.0180
#Tvi= 0.9830 +/- 0.0220
#Tri= 0.8780 +/- 0.0220
#Tb_bv= -0.0200 +/- 0.0120
#Tb_br= -0.0120 +/- 0.0070
#Tb_bi= -0.0090 +/- 0.0040
#Tv_vr= -0.0210 +/- 0.0200
#Tv_vi= -0.0100 +/- 0.0100
#Tr_ri= -0.1360 +/- 0.0240
#Tv_bv= -0.0140 +/- 0.0120
#Tr_vr= -0.1100 +/- 0.0190
#Tr_vi= -0.0590 +/- 0.0100
#Ti_vi= 0.0130 +/- 0.0130

These are from Transform Generator applied to several sets of measurements of both M67 and NGC 7790.

Allen

Allen:

So you should be able to calculate the transform correction (magnitude difference) for your targets using the equation provided. You can then compare to what you measured previously.

Ken

Ken,
Thank you for taking the time to help me. What is delta v in the set of equations you posted? Also, looking at the equations, I see a problem in the process I have used for TA. The equations call for Vcomp. I assume that is the average of the expected magnitudes of the comp stars. I know how to determine that, but I did not include that in the AAVSO report file I uploaded for TA. With a single comp star, I think the CNAME gives the program a way to find that value from its database. With an ENSEMBLE of comp stars, I am entering the average of the comp measurements. How do I input Vcomp?

Allen

Allen:

delta v is instrumental magnitudes (v tgt - v comp).

BUT, I was hoping you would just calculate the ‘predicted magnitude difference’ for each comp using the bottom RED equation, to see how significant transformation corrections are!

Somewhere in VPhot TA help there should be a comment about Vcomp which will be put into the notes field at end of line? It is an average.

Ken

PS: BTW, VPhot does all of this for your images run as a time series. Even if you don’t normally use VPhot, you could do it once to get an AAVSO AEFF Report to check out format.

Ken, I did calculate the difference, and it is not the same as the difference the transformed and untransformed mags. That’s why I looked again at my procedure. I don’t think I’m doing it quite right. VPHOT will only do a time series on one filter at a time. I can combine them, but I still have to edit the combined file to get TA to transform ENSEMBLE measurements. The results I’m getting now have comments at the end of each measurement.
APASS mags |CMAGINS=13.172|CREFERR=0.021|CREFMAG=-999.000|CX=1.2429|KMAGINS=13.849|KMAGSTD=-998.323|KMAGTRAN=-998.339|KREFERR=0.028|KREFMAG=-999.000|KX=1.2429|TAver=2.70|Tb_bv=-0.0200|Tb_bvErr=0.0120|VERR=0.076|VMAGINS=1024.272|VMAGSTD=12.100|VX=1.0215
The value of -999 is used in many programs as NOT A NUMBER. I obviously don’t have everything right.

I’ll take another look at the help files.

Allen

Ken,
I added |CREFMAG=13.577 |CREFERR=0.005| in the notes field of the first B-filter line, |CREFMAG=12.988 |CREFERR=0.009| in the notes file of the first V-filter line,
|CREFMAG=12.532|CREFERR=0.019|in the notes file of the first R-filter line, and
|CREFMAG=12.130|CREFERR=0.023| in the notes file of the first I-filter line of the report file I put into TA.
TA appears to have used these values appropriately for the rest of the lines in my file. The notes field of each line has the appropriate values for CREFMAG and CREFERR. The red equation yields differences that compare with the differences in the transformed and untransformed B-filter data. Overall the results are only slightly different from what I was getting before, but I think I’ve got everything working now. The differences between the transformed and untransformed mags are still smaller than the uncertainties in the measurements. I think I must have a very good set of filters.
Thanks for your help.
Allen

Allen,

VPhot does a time series of multiple filters now (for a year+ or so?). Try it.

Ken

When it comes to transforming your photometry to a standard, the solution IMHO is very easy. Just transform all you photometry. Get a processing procedure in place and follow it for every measurement. Rationalization and hand-waving about 0 color differences has no place in the discussion. Just do it. It is not that hard.

Jim (DEY)

Jim, I really have two questions here. The first one is, “Am I doing this right?”. If I am, then the second question is “If the differences between the transformed and untransformed magnitudes is smaller than the uncertainties in each measurement, why bother?”. You say “it is not that hard”. I probably spent more than eight hours working on a spreadsheet to prepare an AAVSO
report file that could be used to get VPHOT TA to transform one set of my time-series measurements. I’ve done this now with two data sets for two different pulsating variables. In each case, the differences were smaller than the uncertainties. I know the theory, but the reality seems like a waste of time.

Allen

I understand the “am I doing this correctly”. Always ask if you are unsure or need help.

Perhaps for one variable you may find the transformation process isn’t “significant”. Could be. However, I am suggesting just generalizing the process of transformation of your photometry to a standard system for all objects, every time. Sure it takes work, checking, validation, etc. the first time you do it. However, once you get your data processing generalized it will work for all objects you may observe.

Jim,
I’m probably stubborn enough to keep trying, but I still need to have that first question answered. It took me several tries to even get TA to process one of my input files. You can see from my initial post what I had to do to get TA transform a file with over 1400 individual measurements. I really wish someone with experience using TA to transform time-series ENSEMBLE measurements would be willing to take a look at my input files and tell me if I am doing this correctly. I have used the two-color transform option in VPHOT to transform individual pairs of images and I get very nearly the same result that I got with TA.
Allen

Allen:

Share/attach your file, if you have enough permissions yet (or email to kenmenstar@gmail.com). BUT, did you really take a time series with ~700 alternating filters like BVBVBVBV…? Or something else? ‘Doing it right’ starts with taking a multi-filter cadence that TA and any other software is set up to handle. Just any random filter order choice will typically not work. Most software is not that smart! As you may have noticed, TA even tries to use certain colors reported in the notes field to help with such issues BUT it is more complicated.

Start with something simple like one time series containing one BVRI set of images. Start with a single comp in your sequence. Does that work for you? Then proceed to an ensemble in VPhot next. Does that work?

Ken

Ken,
I’m pretty sure I’m taking the data right. I’ll send you my input files with a bit of explanation. Thanks.
Allen

Is that 1400 measurements or 1400 images?
Stars don’t change much over 5 or 10 minutes unless they are spinning.
VPHOT can easily stack your images for you. I think that reduces your error bars by 1/n^2.
TA in VPHOT does ensemble comps with multiple filters.
As far as I know, it wants the same number of images in each filter.
Try stacking, you will be pleased.

Ray

Allen:

I look forward to receiving your TA input files. However, I’m a little disappointed by your response in that it will only take to a few minutes in VPhot for you to respond to my simple starting point questions in my last paragraph above?

Are you able to generate a single BVRI times series in VPhot, save an AAVSO AEFF Report, and then import this file into TA and generate the corresponding TA transformed report from the middle TA box?

Ken

Ken,
I can, and have done all of those things in VPHOT, but I think it is more practical for me to do the photometry on my computer rather than uploading the large number of files I am generating.
My workflow involved taking two to three hundred images over several hours on several nights. The sequence was BVRIBVRI… Tyc 3224-2602-1 is a HADS star with a period of just over 2 hours, and I think the cadence I used is appropriate. Ken, I looked back at the files I sent you, and found that for some reason the JD values were rounded off to only three significant figures. The programs I used produced five or six. I’ll send you some different files.
Allen