What do we know about the structures of our knowledge and its connectivity? Do they differ between novices and experts, especially on the topics of experts?

Imagine if I asked you to come up with as many words as possible related to quantum mechanics. How many words would you generate? How about if you had to come up with words related to cognitive psychology? I’d bet “cognitive psychology” would yield considerably more words than “quantum mechanics” for readers of this blog. For physicists, the outcome would likely be flipped.

I interviewed Cynthia Siew (pictured below) on her recently published paper on the topic of knowledge structures and networks. In the interview, you’ll learn about the research, get a great, basic description of networks, hear about her R package that you may find handy for your research, etc.

Transcription

Intro

Lai: You’re listening to All Things Cognition, a Psychonomic Society podcast. Now, here is your host, Laura Mickes.

Intro to the interview with Cynthia Siew

Mickes: In this interview, I get to talk with Cynthia Siew [pictured below], who is at the National University of Singapore, about her work on semantic fluency and networks. The research is published in the Psychonomic Society journal, Memory & Cognition.

Here we go.

Interview with Cynthia Siew

Mickes: Hi Cynthia. Thanks so much for chatting with me about your recent paper published in Memory & Cognition. It’s really nice to meet you and hear all about this research.

Siew: Hi Laura. Thanks for having me looking forward to the chat.

Mickes: Great. The paper that we’ll talk about is called “Investigating the network structure of domain-specific knowledge using the semantic fluency task.”

You published this work with someone else. Who was that?

Siew: Oh, so I published this with Anutra Guru [pictured below] and she’s a research assistant in the lab. Now she’s doing her Master’s at King’s College in Psychology.

Mickes: I think it’ll help the listeners if we give some definitions.

What is semantic memory?

Siew: Semantic memory is a part of memory that stores all the concepts that you know. Not just the content of the concepts itself, but also the relationship and the associations between these concepts.

Mickes: So an example of semantic memory would be something like: um. I’m meeting you and your affiliation is the National University of Singapore. And now I’ve activated my brain to think of everything I know about the National University of Singapore, which includes a couple of my friends and, and really that’s it. And so my network is pretty small.

Do you think that’s a good example for people?

Siew: Yeah. That is precisely what semantic memories are about. You have information that you know about a certain thing, of someone, of a place. And what happens is that when you activate a concept, right, of a place that activates other concepts that are also related to that place.

Siew: So if you think of the concept of animals, right, that leads to you to think about all the animals that you know – your dog, uh, your neighbor’s cat, you might also activate many animals that you see in a zoo.

Mickes: Yeah.

Siew: Right. And essentially that is semantic memory. It’s all the things that you know, and how these things are related to each other.

Mickes: And what is the research background for the paper?

Siew: The starting point of this research was actually with a classic question in psychology, which is: What distinguishes an expert and a novice? This is an important question in expertise research. And what we know from prior work is that experts seem to have this deeper abstract, conceptual knowledge of the domain of expertise that they are in. So, you know, a classic psychology experiment is experts in chess can remember chess formations on a board, uh, much better than a novice, right? That’s because they’re drawing on their knowledge of chess.

And so what we wanted to do here was to compare the knowledge structures of quote, unquote, expert and novice, and I’m using quotes. And you can’t see the, air quotes,

Mickes: <laughs>

Siew: because we are just using the number of years of education as a distinguisher of someone who has more, uh, experience in specific academic domains of knowledge versus someone with fewer years of experience,

Mickes: Right. What were your hypotheses?

Siew: Our hypothesis is that the network structure of the experts should have a smaller path length and should be also less modular. And I think I need to explain two concepts a lot more before getting deeper into it. The idea of an average shortest path length and modularity are network measures that give us some way of quantifying the internal representations of memory.

Let’s start with the shortest path. The average shortest path length is the number of steps it takes you to travel from one node in a network to another node in a network. So a simple way of thinking about this is like in a social network, this is the six degrees of separation idea. So despite the size of the social network, you really only need a few steps to get from one place to another place. And so if a network has a shorter path length that indicates more efficient ways of traversing or traveling or activating different concepts in memory.

The second concept is modularity. And you can think of it as a measure of how good the subclasses are in the semantic network. So a network with very high modularity, you can think of it as it being very clumpy. In a social network, you have all these cliques, right? And they’re very tight cliques – that would have a very high modularity. But a network with lower modularity indicates that these subdomains or sub-clusters have connections with each other.

So translating these two ideas into a semantic network or a knowledge network, a network that has a shorter average path length would be a semantic network that is very easy to navigate and to move around in. And we expect the experts to have that structure. On the other hand, we also expect that experts should have less modular structures because they are able to draw connections between subdomains, right? Subdomains of knowledge that they know, whereas a novice, we sort of expected that a novice would have more clumpy and not very well integrated pockets of knowledge of a given area.

Mickes: That is a really good explanation. I love the idea of using social networks to explain that.

Okay, so we have your hypothesis. Who were the experts and who were the novices?

Siew: Yes, I do have to say that the initial impetus to do the project was because there were a small group of students from the NUS High School of Mathematics and Science. They wanted to do a psychology project for their research module. So I should maybe explain what is NUS High School. It’s an independent school in Singapore that offers a diploma and they have specializations in mathematics and science.

So they have to do a research module. They wanted to do something about language and about memory. And they approached me somewhat out of the blue. And I was like, oh, well, what can we, what can we do? And it was important to perhaps give them a task that wasn’t too challenging. So the fluency task was actually perfect. It was easy for them to create the experiment. And they wanted to ask questions about, ‘oh, can we do fluency?’ I mean, normally we do fluency tasks with animals, right: Name as many animals as you can.

Mickes: Yes.

Siew: And the students actually were like, well, well, can we measure something else other than animals?

Mickes: Really? They asked?

Siew: Yes. Yeah. They asked. I was like, yeah, I already had these thoughts in my head. You know, like, yeah, can we use it to measure something else? And in through additional discussions with them and their supervisors, we sort of landed onto this design where we would collect fluency lists for academic subjects, like biology, chemistry. I insisted psychology be put in there so that we could then also collect the same data from the experts, quote, unquote experts, which are our undergraduates from the NUS psychology program.

Mickes: Okay. And who are studying biology, psychology and some of these …

Siew: That’s right. Exactly.

Mickes: They’re the experts, and the high school students are the novices,

Siew: That’s right.

Mickes: I’ve never met high school students who would be so motivated. It’s so nice. What a nice experience they must have had and you must have had.

Siew: Oh, they were great. They asked really good questions. They were just genuinely interested in psychology. They’ve never taken psychology in their curriculum. That’s my understanding.

Mickes: Yeah.

Siew: They may have some exposure to it. They mainly do mathematics and, uh, biology …

Mickes: … and physics and chemistry.

Siew: Mm-hmm <affirmative>

Mickes: Do you think any of them will pursue psychology then? You’ve probably created a little army of cognitive scientists.

Siew: Yeah. Perhaps just by doing research, they all get infected and they’re now keen on cognition and memory.

Mickes: Yes. That’s great. That’s what happened to me. <laugh> I joined a lab, and thought, oh, I love this.

Siew: You know, it’s the same for me too. I think everyone has a similar story about how they got into cognition and, and memory.

Mickes: I think that a lot of our listeners can relate to that.

Siew: Yeah.

Mickes: Okay. Back on track. You did the semantic fluency task, and usually it is exactly what you said. I’m gonna give you a category and you’re gonna name all of the items in that category that you can think of, like animal and then you time them and they, they say all the animals, cat, dog, pigs, so forth.

Siew: Yes, yes.

Mickes: Yes. And then you did the other subjects. It’s a little different because you did it. You had to do it online of course because of the pandemic. So people were typing instead of …

Siew: That’s right.

Mickes: …verbally…

Siew: that’s yes mm-hmm <affirmative>.

Mickes: Yeah.

Siew: So we would flash the semantic categories. So if it’s animals, it’s animals, if it’s, uh, chemistry it’s chemistry. And they just have to type out as many items or concepts that are related to, let’s say chemistry. We made an application that would make the typed responses, when they hit enter, disappear from the screen so that it doesn’t stay on the screen and affects the next responses.

Mickes: Right.

Siew: Uh, so that tries to mimic the original way that the fluency task is normally done, which is for people to just produce the words. So produce the concepts out loud.

Mickes: Right. And you also maybe want to look at errors and so you won’t be able to look at repeat words if it’s still on the screen, they might not repeat the word again.

Siew: Mm-hmm <affirmative> yes.

Mickes: that’s…

Siew: Although we found very few errors in our … it seems like they all understood the task and…

Mickes: Wow.

Siew: Maybe, maybe they’re all really young and …

Mickes: <laugh> all keen and excited to do well

Siew: …excited and they, yes. So yeah, we had very few errors. The data was actually quite good.

Mickes: What did you find in terms of the behavioral data?

Siew: What we found is that the expert group tended to produce more responses to all of the cue words that were presented to them as compared to the, uh, novice group. [See plot below.]

Mickes: Okay. So it didn’t matter whether they were animals. You’ve also had the category, fruits. It didn’t matter if they were fruits and animals or the subject?

Siew: Across the board, they produced more…

Mickes: yeah.

Siew: …responses.

Mickes: Okay. I love Figure 1. It’s visualizations of semantic fluency networks. I’ll put it on the website so that people listening can look at it [see below]. They should also read your paper, of course!

Will you tell us about what you found?

Siew: We have a list of fluency responses and we need to estimate a semantic network from those fluency responses. So it’s somewhat technical in nature and there were four different network estimation methods.

Mickes: Yeah.

Siew: But the general idea there is what is the network structure that is most likely to give you the observed fluency responses that you see? Right. And so then, I reverse engineered that using some statistical or computational methods, different methods have different ways of approaching this problem. But that’s the general idea that just trying to figure out, given the data, given what we see in the fluency responses, what’s the most likely network,

Mickes: Right.

Siew: Once we have the network that’s estimated from the network estimation technique, what we can then do is to retrieve some information about its structure. So computing the average shortest path length that I mentioned earlier and also the modularity of the network. And so these network measures will give us some way of quantifying, right. The differences between the networks of the novices and the experts.

Mickes: What can you tell us about the networks?

Siew: What we found was, as what was initially hypothesized, the expert network had a shorter average path length and also less modular network as compared to the novice network.

Mickes: So there would be fewer clusters with the experts? Is that right? Did I understand?

Siew: No, not, not really fewer, but more like the clusters are not so well defined.

Mickes: Oh.

Siew: And that suggests interconnectivity or interconnections between sub domains of knowledge in the experts.

[See below for two examples where the words were “animals” and “psychology.”]

Mickes: Okay. All right. So what do you think all this means?

Siew: That’s a big question. <laugh>

Mickes: I know!

Siew: yeah. So for one, this is, this is really cool because it means that we can use fluency tasks and network analysis to uncover knowledge representations. And we are finding these differences in networks that have more experience or more expertise, where they seem to have a structure that is showing these connections between sub domains of knowledge. And also have a short average path length. And so that sort of indicates an ease of navigation, right? The ease of traversing between different ideas or concepts in a given knowledge space.

So this has very cool implications, I think, for education, right?

Mickes: Yeah.

Siew: So it compliments things like assessments that, that really maybe focus a little bit on the retrieval of knowledge or whether you can apply that knowledge to a problem. What this technique is really showing us is that there is this organization among concepts and memory, and there’s some interesting, meaningful structure that could correlate with expertise.

Mickes:

Outstanding.

Do you have any follow up ideas? Are you working on other projects related to this?

Siew: Yeah, so we, we have quite a few things going on in the lab. One is that we want to keep pushing the fluency tasks and the network analysis a little bit more. I have a master’s student who’s collecting fluency responses to the category of emotions.

Mickes: Ooohh!

Siew: Just come up with as many emotion words as you can in two minutes. So it’s the same task, but now we are trying to measure some aspect of the emotional lexicon if, if you will. And what’s sort of also very interesting is that we have data about the level of psychological distress that the participant is experiencing. And so we are trying to see if the structure of these emotion networks now, do they sort of correlate with higher and lower levels of, uh, psychological distress.

Siew: The general finding so far is still a work in progress.

Mickes: Okay.

Siew: And still analyzing.

Mickes: Yeah.

Siew: But what we’re seeing is that the networks that are built from the participants with higher distress have more modular structures. So remember modularity is this idea that the network is very clumpy. So it means that in terms of their emotional representations of emotional concepts, there are very sharp clusters between positive emotions, negative emotions, anxiety kind of emotions may suggest that there are some possible relationships or correlations there with distress and the abilities of how to navigate that emotional network efficiently.

Mickes: It’s really clever if you’re seeing things out of people who are not doing so well in terms of mental health and these clusters, there then could be a way that you could add that to maybe some sort of cognitive-behavioral therapy. What a cool idea.

What else are you working on?

Siew: I guess something to note is that the current approach is a very cross-sectional approach. We have the experts and the novice from two different groups, but ideally it’ll be really neat to follow the trajectory of how expertise develops. That means collecting data on a group of participants who are going through the educational system and then giving them fluency lists along the way, and then tracking their development. I think that would be a nice compliment to showing that we are not just finding the group level differences.

There are also ways to estimate the semantic networks or the knowledge networks for individual subjects or individual participants. But we do need to collect a lot of fluency list per participants so that we are able to then estimate the network. Because individual network analysis is quite computationally expensive and it helps to have a lot of data for a given participant.

Mickes: Right.

Siew: But these are potential ways forward.

Mickes: Oh, that’s, that’s a cool idea.

When I was researching you, I saw that you published an R package that you developed called spreadR is that how you pronounce it?

Siew: I say “spreader”

Mickes: This package is in the paper in another Psychonomic Society journal of Behavior Research Methods. So spreader, what does it do? It’s got to be about networks, right?

Siew: Yeah. It’s all about networks! What spreader does is that it takes a network that you give it and you put activation units on the nodes. A very simple way of thinking about it is that you can think of activation as just some cognitive resource. Just think of it as like water and you drop the water in the nodes that you want to activate and you just let the water spread based on the structure of the network. What this does is that it gives you a way to simulate the process of spreading activation in a network.

Mickes: Ohhh. That is,

Siew: That’s it.

Mickes: That’s it? That’s really cool.

Siew: That’s all it does. Yeah.

Mickes: Yeah.

Siew: Mm-hmm <affirmative>.

Mickes: So that’s a tool for anyone who is interested in simulating network data

Siew: …Spreading activation in the network. Yeah. So it can be any network that you’re interested in. If you’re interested in language networks or semantic networks, then you would basically load the network that you’re interested in and decide where to put the activation on which nodes and then let the activation spread for a certain number of time steps. And you get information about the activation level of all the nodes in the network. And so what you can do is you can then maybe correlate that with some behavioral data that you have. Yeah.

Mickes: So anyone who’s interested in spreading activation analyses would use it. Have you got a lot of users?

Siew: Well, I think a lot of people are downloading it, whether people are actually using it… I’ve seen a few publications that have used spreadr in some great, in some ways, basically trying to relate how activation spreads to priming, semantic priming.

Mickes: Oh.

Siew: So hopefully more people will use it to ask questions about how the structure of the network affects various kinds of processes.

Mickes: Right.

Siew: I don’t know if a lot of people are using it actually to do the research because I do see a few citations, uh, here and there, but

Mickes: That’s good.

Siew: Yeah. They’re a bit more just to say like, look, that’s a tool that you can use, uh, and so on, but maybe it takes time for things to be published.

Mickes: That’s right.

Siew: Yeah. So yeah, maybe it’s a bit slow now, but maybe more people will use it.

Mickes: The fear is that you create this package, and nobody uses it. I, I guess you use it, but…

Siew: I guess yeah, I use it.

Mickes: You want to share it with the community though, so I hope a lot of people use spreadr.

? I think I asked everything that I wanted to.

Siew: I’m really excited to share my work on this platform.

Mickes: Oh good!

Siew: It’s a great opportunity and a great way to learn about psychological science. I listened to a few of the podcasts and wow,

Mickes: Thank you!

Siew: I learned a lot about things that I wouldn’t ever have come across in my own research. And so it’s a great, yeah. This is a great platform. Thanks for doing this.

Mickes: Oh, thank you so much. I always feel very selfish because I love doing it for the same reason. So thank you so much for talking about your research.

Siew: Well, Laura, thanks for having me, and talk to you next time.

Concluding statement

Lai: Thank you for listening to All Things Cognition, a Psychonomic Society podcast.

If you liked this episode, please consider following the podcast and leaving us a review. Reviews help us grow our audience so that we can reach more people with the latest research about cognition and the psychological sciences.

By the way – we run a blog, too! You can find it online at featuredcontent.psychonomic.org. New articles are published regularly, covering the latest research from all seven of our scientific journals.

We’ll catch you next time on All Things Cognition.

Featured Psychonomic Society article

Siew, C.S.Q., Guru, A. Investigating the network structure of domain-specific knowledge using the semantic fluency task. Mem Cogn (2022). https://doi.org/10.3758/s13421-022-01314-1