Marketing Syrup as Reverse Osmosis or Pipeline Free

[Wannabe]

So are some of you guys saying that bigger batch boiled syrup in flat pans doesn't taste any different then syrup that's RO'd and spends minimal time in the evaporator?

[DrTimPerkins]

The question won't be "Do you like this?" or "Does this taste like maple syrup?". It will be "Which do you like better?" It would not surprise me to get "I like them all."

If you are looking for differences, the best approach would be to give each taster 3 samples of syrup to taste (and put them in containers they can't see into). Put the same syrup in 2 of the containers, and a different one in the third. Ask them to tell you which is different (sort of like that Sesame Street game, "One of these things is not like the other", but technically called the "Triangle Test"). If enough people can detect which one is different (compared to pure chance), then there is a difference. Give some people 2 RO syrup and 1 non-RO and some 2 non-RO and 1 RO. If you do this with about 30 people, and 14-15 correctly detect the sample that is different from the other two, then the flavor is different.

[DrTimPerkins]

So are some of you guys saying that bigger batch boiled syrup in flat pans doesn't taste any different then syrup that's RO'd and spends minimal time in the evaporator?

That was not part of our testing in that particular series of experiments. We simultaneously boiled sap that had not gone through an RO side-by-side with the same batch of sap that went through the RO in the same type of evaporator tuned as closely as possible to each other (a 3' x 10' oil-fired CDL evaporator). In different years, we compared sap vs 8% concentrate (from the original batch of sap), then sap (2%) compared to RO at 8, 12, 15 (again, same starting sap), then RO concentrate at 8 vs 21.5 Brix (same starting sap).

For each of those studies, there were some slight differences in chemistry, but in blind taste-tests, experienced maple people couldn't detect which was which.

http://www.uvm.edu/~pmrc/Effects%20o...erg%202015.pdf

It sounds simple put that way, but to do this we had to build a special facility to hold up to 4 evaporators and associated equipment and tanks, purchase 4 evaporators, hire staff to run them, and BUY sap or concentrate to do the experiments. For each year, we did multiple boils to ensure we had good representation of results. In some years we were spending $30k+ buying sap/concentrate because we don't have enough trees to generate that much sap. Then we had to do all the chemical analyses, which in some cases (flavor profiles) is rather costly.

Research isn't cheap.

[DrTimPerkins]

When I concentrated into the teens, the target number was 18 Brix, the nitre would burn in the flue pan and I made dark syrup from the beginning of the season right to the end.

Burning niter on the pan would definitely cause darkening. It isn't something we looked at in our research, because generally people try to avoid doing that (to avoid the darkening of syrup and burnt-niter off-flavor). However coping with a higher rate of niter deposition is definitely something people have to deal with as they concentrate to higher levels. When you concentrate the sugar, you're also concentrating the minerals, so niter will build up correspondingly faster as your concentration level increases. Had you not burnt the niter, you'd most likely have had lighter syrup.

[Cedar Eater]

If you are looking for differences, the best approach would be to give each taster 3 samples of syrup to taste (and put them in containers they can't see into). Put the same syrup in 2 of the containers, and a different one in the third. Ask them to tell you which is different (sort of like that Sesame Street game, "One of these things is not like the other", but technically called the "Triangle Test"). If enough people can detect which one is different (compared to pure chance), then there is a difference. Give some people 2 RO syrup and 1 non-RO and some 2 non-RO and 1 RO. If you do this with about 30 people, and 14-15 correctly detect the sample that is different from the other two, then the flavor is different.

I wouldn't be interested in difference because any two batches could be different regardless of whether RO was involved. My interest would be in preference and if preference consistly came up ambiguous or in favor of one or the other that would be data.

[ecolbeck]

I wouldn't be interested in difference because any two batches could be different regardless of whether RO was involved. My interest would be in preference and if preference consistly came up ambiguous or in favor of one or the other that would be data.

Does preference not indicate a difference?

[McKenney Maples]

Right I think the point is the samples can't be distinguished from each other in a blind test. If a sample could be singled out potentially the taster could develop an opinion.

[Cedar Eater]

Does preference not indicate a difference?

I guess, but that's irrelevant. The ability to detect a difference is of no concern to me. Since I am not going to be using the same batch of sap for both cooks, I would expect a difference. Knowing one was detected would tell me nothing interesting. If there's a preference for one or the other, I would want to know which and if there is no preference, I would want to know that.

[DrTimPerkins]

I guess, but that's irrelevant. The ability to detect a difference is of no concern to me. Since I am not going to be using the same batch of sap for both cooks, I would expect a difference. Knowing one was detected would tell me nothing interesting. If there's a preference for one or the other, I would want to know which and if there is no preference, I would want to know that.

If they express a preference, then obviously they are detecting some difference to base that judgment on. Essentially what you are doing is making what is called a "two-sided comparison" (is there a difference in either direction) into a "one-sided comparison" (is there a positive difference in flavor), which statistically is harder to do and is kind of jumping the gun (can one say there is a preference -- a positive effect -- before one determines whether there is any difference -- any effect).

If you do it that way, there should be several choices: Prefer sample #1, Prefer sample #2, No Preference.

I guess I'd argue that unless you did several samples of RO syrup and several samples of non-RO syrup, that you would end up with results that didn't mean much. You would have to be able to declare that, over many different batches, the RO (or non-RO) syrup was significantly better (more frequently preferred) than the other.

Since you are doing this with different batches of sap/syrup, there really is no way for you to be able to tell what the preference/non-preference is related to. While it might be RO vs non-RO, it might also be the starting sap chemistry or it might be that a squirrel took a bath in one of your sap buckets before crawling out again. There are always some days when the stars seem to align and you make contest-grade syrup....other days you don't. Whether that happens on an RO or non-RO day is not controlled for in your study, and is thus just random noise.

[Cedar Eater]

If they express a preference, then obviously they are detecting some difference to base that judgment on. Essentially what you are doing is making what is called a "two-sided comparison" (is there a difference in either direction) into a "one-sided comparison" (is there a positive difference in flavor), which statistically is harder to do and is kind of jumping the gun (can one say there is a preference -- a positive effect -- before one determines whether there is any difference -- any effect).

If you do it that way, there should be several choices: Prefer sample #1, Prefer sample #2, No Preference.

I guess I'd argue that unless you did several samples of RO syrup and several samples of non-RO syrup, that you would end up with results that didn't mean much. You would have to be able to declare that, over many different batches, the RO (or non-RO) syrup was significantly better (more frequently preferred) than the other.

Since you are doing this with different batches of sap/syrup, there really is no way for you to be able to tell what the preference/non-preference is related to. While it might be RO vs non-RO, it might also be the starting sap chemistry or it might be that a squirrel took a bath in one of your sap buckets before crawling out again. There are always some days when the stars seem to align and you make contest-grade syrup....other days you don't. Whether that happens on an RO or non-RO day is not controlled for in your study, and is thus just random noise.

I agree that it would take multiple samples for each participant before any meaningful conclusions could be drawn. The meaningful conclusions could be anything from Participant A never picks the RO sample to Participant R never expresses a preference to Participant W always picks the darker sample regardless of processing method. But if none of the participants ever selects RO or if most of the participants express no preference, then I will have data about preferences instead of data about noticing differences. There are two "No Preference" conditions that I would listen for. If someone says, "I can't taste a difference" that would be different from "I can taste a difference but I like them both equally." I could follow up the latter result by asking them to describe the difference they detected.

I suspect that even if the same batch of syrup is split and tested both ways, you would still be dealing with the possibility of difference being introduced as a result of either two different evaporators being used or one 1/2 batch of sap sitting-in-wait while the other was occupying the evaporator. I don't believe a perfectly controlled test scenario exists. Fortunately, by asking about preference, I get right to what is important, whether the RO is introducing an off taste or removing a desirable taste. Enough samples over a long enough period of time would tell me whether the RO is having a negative effect to offset its positives.