According to Puur, the questions used were taken from prior surveys, which had already been conducted in Estonia and were in turn were based on international equivalents.

Himma: Various political forces have argued that this survey was definitely necessary, as nothing like this had been studied before and so it was very unique. Are you aware, as a social scientist, that there are other [existing] studies on the subject?

For me, studies like this are part of [raising] societal awareness. The need to study a topic like this right now came about because previous data sets of a similar form date back ten years or even more. In short, the motivation [to study this] comes more from the lack of up-to-date information on certain issues.

Himma: Is that still the case? It was just last year that the European Social Survey collected this kind of data and it is publicly available for everyone to use!

Yes. It is very important in surveys, in order to be able to isolate and monitor some types of trends, that questions are asked in more or less the same way at different points in time. And the things I referred to are probably not comparable to what is in the European Social Survey. Here, the devil is in the detail.

Libe: How did these questions come about? Did you formulate the questions and the answer options? Did you have anybody to help you and how did that process go?

I didn't do this work completely alone. There were a couple of people from the foundation side, then a colleague of mine and also Raul Eamets involved in this activity a little bit. There were around five people. But, yes, I took on the role of doing the final editing.

And if you're asking me how these questions came about or where they came from, as I said earlier, the desire to be comparable with an earlier time period and to understand what has changed in this area was central to this research activity. Because of that, we followed two research models. One line of research was population surveys, the first round of which was done in Estonia in the 1990s regarding fertility and family sizes, with the second round taking place in 2004-2005.

And the second is a series of studies done more by health scientists, the title of which is the "Women's Health Survey" ("Naiste tervise uuring"), conducted in 2004-2005. These were the two source materials that we more or less combined. Both of these surveys also have international equivalents. That's how it came about. There's not a lot there now that's particularly original.

Libe: I have to admit that when it comes to the wording of these questions, I have a number of women friends who would probably like to give you a piece of their mind. At least based on the feedback I've heard from them. Has it come as a surprise to you that the backlash has been so strong, particularly in relation to the wording of the survey?

If I'm being honest, it still is. Perhaps it has to do with the fact that I've also been involved in these earlier surveys, but when I mentioned these annual figures, you understand that we're talking about maybe 15 years ago. And it seems that society has changed quite a lot in that time. In fact, the questions were the same [as those used in the past], and that is why I did not expect such reactions, knowing that thousands and thousands of people in Estonia have been asked these same questions before.

Himma: I have to admit that I felt kind of sick and humiliated. As a woman, I am not part of this survey, because I have children. And let it also be said that [I have] two children and so I am probably not allowed to speak, as some people have said here. But when reading the question, 'have you had sexual experiences (sexual intercourse, petting) with a partner of the same sex as you?' it begs the question, that this is probably not even the kind of information I would want to share with my husband. How would you answer a question like that?

And that's the thing about all these questions in the survey, and it's [written] in the survey invitation that it's voluntary, right.

Himma: But nowhere was there a warning that these are very, very intimate questions.

Yes, it's one of those things now, that if you write in the survey invitation that the questions are really, really intimate, then there's a separate consideration or risk that the response rate will be so low that people won't even get to consider the question. There are usually two filters involved in the system of weighing this up. One is the polling company it goes through and their experience. And the other is the ethics committee, which unfortunately, due to a tragic mistake so to say, is now outside of the process for the moment.

Himma: But what is this "petting," which has been used in the question anyway? Personally, I don't know that word. So what is it?

This question was taken from a women's health survey that was conducted in 2014. I remember that questionnaire in quite some detail. This question is written just as it was right there. And the English word is in parentheses there.

Himma: It's just that, for sociological surveys this is not good practice. As a rule, you don't use words that are not common knowledge. I did a test yesterday and asked 50 people about that same word and half of them didn't know what it meant. Which means that, in turn, it is not possible to answer the question with certainty and make an informed choice. Why was this chosen?

Let's put it this way: mistakes like this due to inattention can happen when surveys are made. First of all, it came from a previous survey and we didn't filter it out and neither did the polling company. I guess it's something that for many people... Well, no one has considered whether everyone understands it in the same way. I think it could have been explained in there.

Libe: So the tragic error that you mentioned was that they started collecting the data before there was approval from the ethics committee. You can read in Postimees that the foundation's assessment of the situation was, that the data could be collected and only after that, when the data was being processed, would the approval of the ethics committee be required. I have never been involved in science myself, but I could see how this kind of approach to data collection might be a little dubious. What are you doing there in the first place that enables such tragic mistakes to be made?

A pretty large number of hours were spent on the questionnaire, but it is fair to say that there should have been even more.

I dare not speak on behalf of the people at the foundation, who were responsible for and organized the conducting of this survey, as well as the communication with the polling company, and whose mistake caused this to happen. That's something they are better able to answer themselves.

Himma: You say that this is a longitudinal survey, that it should be repeatable over a long period of time. How often was it planned to be repeated?

That is now a bit misleading. It is not a longitudinal survey in the sense that we interview the same people several times over a very long period of time. At the end of the survey, there was a question, which asked whether people would agree to [their responses to questions about] their future intentions and the factors influencing them being combined with family or childbirth data from the population register, in order to assess whether these intentions would materialize in the future. That was the original idea.

It is very often done in this way, because it avoids confusing people and also enables us to better understand which  factors influence the realization of people's intentions. The longitudinal element has somewhat misleadingly taken on a life of its own here, and the foundation will probably decide to do away with that element altogether, so that people have a greater sense of security.

Himma: We know that this data has been collected without approval from the [University of Tartu's] ethics committee, and also that most likely none of it is in line with GDPR (General Data Protection Regulation). What will be done with the data that has been collected so far?

At the moment, the data collected is in the possession of the polling company (RAIT Faktum & Ariko – ed.) and it is up to the Data Protection Inspectorate and the ethics committee to decide [what will happen next]. I'm not going to make any predictions. There have been rumors in the press as to what the options are, or what the likely options are, but we will wait until the end of the process.

Libe: In conclusion, a large number of scientists have also pointed out in today's papers that all of this has undermined the credibility of [Estonian] science. What is your message, what lessons can be learned from all this?

To our fellow researchers, certainly, apologies from all members of the board for not having been diligent enough in this process. Knowing ourselves what the regulations and requirements are, trust is one of those assets, which is very easy to lose but very hard to win back. I believe the only thing that can be done in this case is to explain as much as possible what the purpose of this research activity was. Where exactly where and in what way these mistakes happened.

That science, as a broader phenomenon, which contributes to our understanding of societies, should not be undermined in terms of trust by, say, the mistakes of certain individuals. That is what we must now try to ensure.

Himma: Based on what we know now, what would you do differently in hindsight?

On the substantive organization side, this survey instrument probably ought to have been tested more. Another comment that could be added here, is that in previous surveys, questions of this kind did not focus on one target group at a time, in the way we are talking about now, but rather on respondents from different family sizes, who were questioned at the same time in the same survey. And in the current situation, the idea was not to survey these different target groups at the same time, but next to each other so to speak, or separately. Perhaps this was also something that went wrong. And, of course, more effective monitoring and self-regulation [is needed] on the organizational side.

--

Follow ERR News on Facebook and Twitter and never miss an update!