I am still mentally processing the ideas and insights I gained from the last British Academy of Management conference. One of these was a brief comment on whether we might see the end of the literature review, or at least of a literature review that can be published. Why? Because AI can now do pretty good literature reviews and surveys, and bespoke tools like Consensus and Elicit have improved a lot since a few years ago. This post will get tech-y again. If you are interested in any of my other posts looking into how AI tools might be useful for research, writing, disseminating, and so much more, here are a few to check out:
Consensus, for example, positions itself as a tool researchers deploy under their control. It’s one I want to try out a bit more in the future. But this summer, Claude launched “Claude Science”, and it is, well, quite science-y. And I do not need no AlphaFold and protein, RNA, whatever thingies. So what can someone like me do with this “Claude Science”? Without being a “scientist” (in the English sense). Let’s not forget, in other European languages, we are all “scientists”, because the meaning is more akin to “knowledge worker” What if a new tool was launched and nobody noticed it?It’s been a while now since Claude Science launched, and I started to play with it. I am not sure if there has been a massive buzz around it — if so, not in my neck of the woods. So what I am writing about here is me trying it in the summer. Since then, they released a desktop version and probably multiple updates, so this might be out of date, but I doubt it because… It's clearly not designed for social scientists. The setup makes this obvious: I kept toggling off all sorts of protein-folding, medical research standards, and god knows what the rest of them were… It’s probably a very powerful tool for someone. But because most of our contemporary and historical data doesn't live in accessible online spaces, the only real research I felt I could do with it was literature research. There are probably other applications I haven’t thought about - if you can think of one or more, do let me know! The state of management and organizational historyTesting out new AI tools works normally best if they have them do some research for you on something that you know and understand well. So I asked it to map the recent literature in my subfield (business history, historical approaches to management/organizations, including "uses of the past," organizational memory, and qualitative-historical methods). Claude Science worked through way too many results from that initial search and began scoping by journal and domain, which was a reassuring step a human researcher would take. It came back with some leading journals:
It then began a thematic mapping, with “momentum, " which, in practice, means developing an early vs recent period split. Based on this it created a rising and falling set of themes:
Here’s the bibliographic portrait the tool developed:
So far, so good - at a general level, this did not surface anything that seemed off-topic, and the trends analysis is genuinely interesting. It then offered up a more analytic take:
All the names checked out and the most-cited works surfaced roughly who I would have expected. It then offered me a menu of further analytical options: a coauthorship/collaboration network, a citation-based map of which foundational works are converging into a canon, or a focused review of the fast-rising "historical methods" strand. So I asked for more, and also highlighted, since it already identified me in the data, to avoid any “smarminess” or “bs” around that in particular. Interestingly, Claude Science repeats instructions in its own words, which helps confirm the message was received correctly. Another cool aspect was that when the token budget ran out, the tool held the query and asked if I wanted to continue when my token budget reset. That was much easier than in Co-Work, and I wish I could do it there. Working its way through the analyses, Claude Science provided some interesting commentary:
Now that is an interesting find - not that we did not know this, but that this can be read back by a tool from the data, and also: just how large is that chasm in citations?! Blooming heck… But what came out of the citation network analysis was even more interesting:
OK, very flattering. Maybe too flattering? We have all heard about AI sycophancy by now, so I had it change and expand the entire publication corpus in later iterations, which lowered my Betweenness Score somewhat but did not significantly change this order. More reassuring was this diagram, which is perhaps the more significant statement about the role and standing of different authors in our field (though you might have different views, and I am equally inclined to say the purely bibliographic calculation has its limitations while still being really interesting):
This diagram is what Claude Science developed this research question:
Now, Roy’s position at the top of the diagram, far away from everyone, is certainly deserved. But would he identify specifically as a “historian”? He is still credited mostly as an institutional theorist, and deservedly so. The structural advantage of those scholars moving from management and organizational studies into history rather than from history into management is pretty obvious here. The outliers are, of course, Charles and Mairi, though their lower productivity also made me wonder whether the corpus needed expanding (which I asked for in the next iteration). Are these the juiciest bits? Maybe, but the rest was also genuinely interesting. Full subscribers get the full accounting and the report generated by Claude Science. So, do you think we still need published literature reviews? Would love to hear your views. You're currently a free subscriber to History in Organizations. For the full experience, upgrade your subscription.
|
No comments:
Post a Comment