Okay, welcome everyone. I can see the participant, the participant number ticking up as people join this.
Demos, experiments and Q&A session today. Thank you for joining. And hopefully you will find lots of relevance in interest and things to ask questions about over the over the course of the session.
He said I'm just gonna give a few more minutes. In case anyone else is trying to join, find the link, all of those kind of things.
And then we'll get started.
Okay, so let's let's go to the the next slide.
So again, just, For some of you who've joined the earlier sessions today, you'll have heard our notes, but just to make you aware of Crossrefts code of conduct.
And if you are on if you are on mastered on or X then you can join the discussion.
We'd welcome you to do so on those platforms. And but I think probably the most effective thing for this session is if you've got questions, and use the Q&A box to pull those to our our presenters.
We're going to take questions for each of the presenters after after each of their sections. So that they're still fresh in your mind.
And we will share the the slides and recordings afterwards. Next slide.
I'm the Reminder, you've got I think around 2 h left, so if you haven't voted for the 2,023 board slates.
Then our ballots close pretty soon but if you wish to submit a vote or change your proxy vote then then let Lucy know in her emails. In this.
So, we've got a really good, interesting slate of candidates this year, so I'm part of being a member of Crossraff is having the capacity to be able to vote and make sure that our board is as representative as possible of you in your communities and your needs.
So we'd really encourage you to do that.
Next slide.
So I'm going to start by introducing the session. And as I said, this is product demos, experiments and questions for those who are willing to to present.
Would say the aim of this session is to give a view across the things that we are at the early stages of exploring and working on both from the product and development teams and also you'll see strong representation from the from the labs team at Crossref as well.
And we're going to give some hints and tips for using our Api's. Ones that you're probably very familiar with or would like to get more familiar with and newer ones as well and then external integrations with our with Crossref from our colleagues at the Public Knowledge Project.
For whom it's very early so so thank you Eric. And each demo will last for around 1520 min and we should have time for questions for our panelists.
So, and Luis and I are gonna keep an eye on the Q&A and try to make sure that those get to the right people.
At the right time. So.
Our first demo is is currently under development. So again, we've got for a couple of these things are under active development.
We've tried to step away for them and not from them and not touch them for the for the purposes of the demos today, but as any of you have done a live demo, know they can be all sorts of fun.
So Kudos to my my colleagues. But our first demo is of our currently under development registration form for, for journal content.
Lena Stall will share a sneak peek of the work that we've been doing to create a simple user interface to make it easier for members to register good quality.
Journal article metadata. So over to you, Lena.
Thanks, Rachel. We're actually starting out quite safe. I'm not doing a live demo.
I just have screenshots and slides so everyone can relax. I just have screenshots and slides so everyone can relax for the next 1015 min and then it gets really exciting when people start doing things like.
Yeah, great to be here today for the many of you who don't know me yet.
As Russia said, my name is Lena. I joined the product team at Crossraff over the summer.
I'm based in Beautiful, in Germany and I'd like to tell you a little bit today about the work that we've been doing to develop as much as said the simple user interface or UI.
To allow them to register gen, without having to touch an XML file. I say we have been doing this work, which is quite generous, for myself because I've only been here a few months and actually on the product side, most of this is the result of a lot of work by my colleague Sarah Bowman who many of you might know better.
But I've taken on this work because she recently went on maternity leave. So just in case you're wondering why there's a new face talking about this.
And on the tech side, chatter to Patrick Vale, I know you're watching.
Okay, so anyway, what we're doing There is basically an extension of the concept behind the new grants registration form that we released I think towards the end of last year that some of you might have used before or seen Sarah demo in the past.
So a lot of this might be familiar if you're. If you're familiar with that tool.
So if you go to the next slide, please.
Yeah, I just wanna acknowledge that. Some people at the audience might be thinking. they're already multiple ways of.
Registering records especially journal articles and it's 10 points to you if you recognise all of these options I've represented on this slide and if you can name them all.
We have a Trustee Old web deposit form that you can see at the top right. We have a metadata manager.
We have a plugin for OJS for those that use that platform to manage their journal articles. We have an admin console where you can upload XML files if you're able to create them.
And then of course that the option of you're able to create them. And then of course that the option of depositing XML and an automated way to our API.
Using HTTPS post which is represented on the bottom right. And that is of course the most kind of reliable, available, efficient way of registering metadata, but we do know that today the majority of our members are either institutions or their small publishers and not everyone runs an XML first workflow and not everybody even has the technical resources to all know how to create XML in an automated way and deposit it.
So we do want to offer these user interfaces that allow everyone to At least, perform all of the most essential tasks that you need to do to be a good cross-ref citizen.
But the goal isn't and can't really be to represent absolutely everything that you could do with direct calls to our APIs through a user interface because If you're familiar with, metadata schema, the sheer complexity behind everything that you can in principle register with us then you'll understand that that's not really realistic to do in a form that people still have fighting chance of actually using.
But that's just an aside. Getting off my, virtual, so books.
I guess the main point of this slide is just to say. All of these different routes, all of these different roads ultimately lead to the same.
Data base into the same API, so why? Would we bother creating yet another tool for depositing?
If you go to the next slide. Actually, each one of the tools that we've built so far that work around the requirement to work with XML files directly.
All of them serve their own little niche of use cases. And you have to be pretty familiar with the crossra tool ecosystem to kind of understand which one is the right one to use for which purpose and and each of our tools also has its own kind of specific limitations.
For example, metadata manager is very complex technically and for that reason it has a number of bugs and issues that we're aware of that we've never been able to and fully address.
The web deposit form is a little bit awkward to use and it doesn't cover all record types and so on and so on.
And all of this together kind of leads to a confusing user experience for those members who are trying to deposit their metadata manually and it's also quite inefficient for us to have to maintain all these different tools in parallel.
They're all the different code bases. If we make a change to a mitigator schema, for example, we have to then as a result of that make multiple change in multiple places to reflect that.
In all of our tools. And so what we've been building Yeah, since, since we started with the G's registration club last year and now with this general article registration form is different because it is being built in.
What we call a schema driven way. And by that, I just mean that with. We're creating the interface, the form that you actually enter the, into.
In an indirect way from Let's get the schema itself. So take the schema as an input and then create a form out of it.
Which means that it does a change to the schema that's much, much easier and much faster for us to reflect that in the actual interface.
And it's also an interesting. Proof of concepts or the wider community to show that it is possible to build.
Your own interfaces in that way. That suit your specific needs. Because as I already said earlier, we will likely never be able to build.
The ultimate. One single tool that. Is usable that but that also allows you to do absolutely everything that you could do if you were talking to the XML API directly.
And so the idea is that if we build a unified set of forms like the grounds, that, form and like now the new generalological form.
Eventually we'll be able to. Replace some of the tool or tools that we know aren't really So serving your community in the way that they could.
I'm talking specifically now about metadata manager which If you've used it lately, you've seen a little banner at the top.
You'll know that we've been planning to deprecate that for a while now. And if you want to read more about the reasoning behind that decision and why we're doing what we're doing, then there's a blog post linked on this slide that Sarah wrote a little while ago called Next Steps Content Registration.
I will also building all of our new tools in a way that they are, easily automatically translatable that's a process called localization.
And that they can also be used by users with disabilities.
Okay, so that's the preamble. If you go to the next slide. I just want to show you a few.
Screenshots. Hope they're not too small on your screens. Just of the current state of the prototype of this film that we've been working on, it's still at a very early stage as Rachel kind of gave a little disclaimer at the beginning and a bit of a warning.
But I just wanna show you, just give you a little bit of a flavor of what I'm talking about.
So if you've used the grounds registration, in the past or if you've seen it, then a lot of the kind of design of this interface is going to look quite familiar to you.
We're deliberately keeping it very simple, so. Sorry these screens aren't particularly shiny or telephone, but we're focusing on the most important the most widely used.
Fields in the Mesopotamia for journal articles and just again, not every little thing, but the schema in theory allows for in the next MFL can be represented in a form without making it so complex that it doesn't work for anyone anymore.
But we're focusing on the most important field just to make sure that we can. Meet the needs especially of those.
Types of members who we know. Can't afford to automatically generate Excel and who know who need these kinds of tools.
In terms of how the tool is used, we've conceived of it as a kind of wizard.
And that guides you through the sequence of necessary steps one by one. You can see a little stepper at the top.
And and also anytime that you enter something or you try to complete the steps, the tool will validate for you whether what you've just entered makes any sense or whether it fits what's allowed in the schema.
That's just because of course. Any time you make a tool that's used by human beings and there's a capacity for people making typos a copied paste errors.
Especially with such a Toysome task as registering. Dozens of general articles by hand if you've done it before I'm sure you've made some copy and paste errors and it's a lot more costly if you have to correct those after the fact so it's very important to.
Not a date inputs early on, which is what we're doing. You can see an example of this.
Actually, on the screenshot in red, towards the bottom because I've entered an invalid.
And also if you're very keen observer, you might have seen at the top right there is a Language switcher, button, this is what I mentioned earlier about localization.
A really important thing that we want to do with any of the new tools that we're building is to make sure that especially with those languages that we know a lot of our members are much more comfortable with than English, they can be used in those other languages.
And the phone can also be navigated using your keyboard. So we're designing it to be.
Accessible enough that it can be used by anyone in the community who actually need it. Yeah, so if you look at the stepper at the top, you can see that the form asks you first for information on the journal and then it goes down to the issue level and then finally the article itself that you're registering.
And if you go to the next slide you'll see some of the issue screen. There's not gonna be that many surprises there for you in terms of the fields that are represented.
You might have noticed already at the top that several titles can be entered both for the journal and for the issue and then also for the article.
That's just because we support multiple languages for titles. Yeah, and like I said, we're focusing on the key metadata fields and those that are gonna be highest value for making a high quality record and also for the people who use their metadata to have a chance to then discover.
That content. That's the issue and then if you go one slide further you'll see the article level.
And you'll be able to see that in this example I've given the article. A couple of different titles and different languages.
I don't speak somebody, so this had to do. And there's Also a roll lookup already built in.
And if you look closely, you can see for the contributor. You can. If you start entering the affiliation.
A tool will find the unique identification and the raw database for it. That's something that was quite a crowd pleasing in the Grant registration form.
And there are more of those kinds of. I guess quality of life improvements and for using the tool that we know already we'll definitely need to or want to offer in the form.
Once we actually have it go life for members to use. Just right now, not all of them are there because we've been focusing very much on getting a prototype into a state as quickly as possible that we can share it with all of you and with the wider community and get some feedback.
If you go one more slide ahead, we'll go to the fourth step. At the end of the whole registration process, if again if you know the grants registration form you'll know what this looks like.
We'll be able to give. The user adjacent file download of the record that they've just created.
The XML actually doesn't visibly get involved at any point. That's just created in the background by the tool automatically and deposited in much the same way that it would be deposited if the user directly used our XML API.
Let's see. This Jason files, so that's one another one of the things that I said we know will want to do because it makes a big difference in how useful the tool is is that We will allow for users to be able to load those Jason files back into the form later and then make changes, which is something that is already the case on the grounds registration form.
Just for this very first prototype we're focusing just on the initial registration part so that we can test that.
With the community. But we know that editing records after the fact, especially for general articles, is something that people use metadata manager for a lot.
Yeah, so that's just a note on that. And actually on that topic, if you go to the.
Next slide, or some people having issues, not seeing slides. I'm just gonna pretend I didn't read that and keep talking if it's working for most people.
Yeah, I, I think we're okay, but we'll have a look.
Okay. Yeah, okay, so. Since I've already used the word soon, Obviously, next questions are what's coming next?
What are the kind of timelines? Like I said at the beginning, We weren't quite ready and confident enough to.
To do this live yet and grow into the actual prototype and enter things and show you how it reacts.
I took the safe route out of taking speed shop. Sorry about that. And we can't quite share a link to the prototype yet either for you to play around with yourselves for the same reason, but like I said, the goal is to get there as quickly as possible.
So we're just putting some finishing touches on it. And it's again a very early version so don't expect to see it in production tomorrow just yet.
But of course we'll keep. Our community updated as we make more progress. We're going to be iterating over the coming weeks and months on early initial feedback that we're going to be getting both internally and then from, from some key members of our community so I just actually I should say thanks at this point to Those of you who are across our ambassador who have already
volunteered to. To help us test the prototype, we'll be getting in touch with you very soon.
And there are also some people who volunteered at the community forum already in the path to be involved in this process, which is great.
And then in terms of some future next steps. Again, if you've used metadata manager.
Recently or not even not so recently you will have seen that a little banner and the T is going to be deprecated It still says 2023.
Right now is the date of sunsetting it, but We don't want to rush this well.
It's still needed because what we're building. Out as a minimum viable version of this new tool.
That's what the MVP on the slide means. Minimum viable product for those. Who aren't so deep in the tech rabbit hole.
Yeah, it's we don't wanna rush this on. I've put 2024 in here because that's probably more realistic, but the idea is that once we've got.
Once we've a bit more formally reached out for community feedback like we always do was such an important project and we've iterated on that and we're getting confident that we have a viable version of this tool.
We will be able to So the ghost. Of metadata manager I guess but of a Halloween theme there for you.
To rest finally and and look to the future. So yeah, if you all. One of those people who really resonate with what I've been talking about and who maybe currently users metadata manager or even the web deposit form to register your journal article, then you want to help us shape this new thing then.
Absolutely feel free to get in touch with me. I think the next slide will tell you how to do that.
Or not? Okay, that was supposed to be a thank you slide with my email address on it.
But, I'll make sure to to add that after the fact i guess Sorry about that.
Yeah, we could put it into the chat as well.
Yeah, it's also pretty easy to guess. I'll email addresses if you know. But yeah, but I also wanted to say that the, the community forum is always also a great place to be if you want to stay up to date with what everyone is doing at Crossraff.
One of the first places that you'll find out when there's news or something to test or give input on.
So, Shadow for the community forum and I'll put my email address in the chat look forward to speaking to some of you about this more and for now I think that's it for me.
Oh, thanks, Jenny. Okay, third is already. Thanks. Thank you, Roger, I guess.
Then there's a question for you in the Q&A.
Yes, okay, we have. A question that says. Give him the driving cross traffic to get publishers to register reference nests.
Will that mean option to upload reference lists either in their entire deeper passing or as a list of DIs only.
That's a very good question. Of course, we know that references are a very very important metadata type and also one that we've been kind of.
Pushing our members when we discuss their participation reports with them. All that kind of thing. To really consider investing in so we know that we want to be representing that in this in this tool in some way.
We haven't really made a definitive decision yet on what the best way of doing that is. If there will be.
Some way to use our simple text query that we tool that we have to to get DIs for references if you don't have them yet or if it's better to do them the other way around.
I don't think that I would, I have a definitive answer on this yet, but that's exactly why we trying to, get this out for community testing and feedback as quickly as possible.
So that we can know these sorts of things down. But that's the kind of thing that we don't want to make.
A decision on to early on before we know what's actually most valuable. So if you think you have input on that then in touch with me for sure.
And I think we let's, I can see there are other questions but I want to.
Yeah.
And I think some advice as well. I want to keep us moving and jump on to the next presentation just so that we don't run over time.
But we can answer those in the Q&A as well and come back to them if we've got time at the end I would suggest.
Is that all right?
Perfect, thanks. Yeah, it's not ready question anyway, I think. Okay.
Cool. Thank you. Okay, so, next up we have Eric Hampson from the Public Knowledge Project or PKP who's going to demonstrate the latest Crossrail plugin.
. 3.4 the DIY workflow has been completely rewritten, addressing usability and issues and providing a better overall user experience.
So, and providing a better overall user experience. So with so many of our members using OJS extensively, and these improvements are really timely and thanks to the team for sharing them with us and giving us the opportunity towards the end of, last year to test them out.
I'm Eric, can we hand over to you?
Yeah, thank you. Start. By sharing my screen if I can.
Does not look like I am able to share my screen.
Would you like to have another go?
Yes.
Oh no, and Perhaps really typical Zoom. Live demo contact side recently we set my computer and unfortunately do have 2 briefly log off of zoom and restart and I'm terribly sorry about this.
Do you want to, what we can do is we can, we can tweak the order a little bit if you want to leave and then we join.
Yeah, that sounds perfect.
I think a couple of us have been hit with Zoom updates this morning, which is of course another, another variable in all of this.
So we'll keep a look out for you, add you back in. And, and I think that will work.
I'm Martin, Ripman, if you're happy to go next.
Thanks.
Yeah, no problem. I can jump in. So yeah, let me share my screen and hope that all the, to work.
Yep, that works.
So yeah, this is awesome. Good. And so yeah, this is a presentation about a new API endpoint that, that we have.
And I'm going to demonstrate it via a Jupiter notebook. I realize this is in Python and some of you might be, I realize this is in Python and some of you might be, might be not familiar with Python and some of you might be might be not familiar with Python.
Don't worry, I will explain and point out the most relevant bits as we as we go through, but basically this is looking at different ways that we can, that we can look across with metadata.
So I'm gonna talk about relationships. Firstly, how you. I can get hold of relationships and, where, how we will combine those into one endpoint in the future.
And so if you are familiar with Python, all I'm doing here is using the requests library to query APIs.
Here's the endpoints. If you'd like to try this out, then, then you can go to, to this, this URL, which, which has the, the endpoints.
And I'll give you a few more pointers at the end for that. I've also, I'm not running this off a co-lab note, but I have a co-lab notebook and I'll share the link for that at the end.
As well. So yeah, we're just gonna start the notebook with a few useful functions, which are to show you the URL that we're querying.
See that in a moment. And then this is this is the function that does the the work and it queries some URL and it expects a JSON output.
So this is that Jason is the the format of the data that gets returned to our APIs. And so we're talking about just now very timely.
References are very important and many of our members send us references. And what we can, what we, what we do at crossf is display those 2 a works endpoint.
So each item has an entry, and you can query the works endpoint, using, using this, URL.
So, you know, API that crossword. Slash works and then we have in in there this fill to which says does this item have references or not just give me the ones with references.
And just to give you examples, we have, you know, 8 references for this DOI, 89 for this.
I don't know what this this is, maybe it's a book or something but 424 references I feel sorry for the the poor production people who who have to go through all of those.
What do these actually look like in, in JSON? Well, this is the, this is the JSON.
List of references. I just pulled out the reference part of the response. And we can see that this DOI here is citing a number of, things and this is this is one of them that I'm highlighting here it has a DOI it also has the title the authors.
And we've got this unstructured reference as well. You don't have to. Pass references to put them into our.
To deposit them with with us. You can have just unstructured text like here. You don't even need a DI.
We will try and match that for you. So you can see just kind of, you know, more examples with similar kind of information.
So that's references. There's another way that you can add. Links between outputs in, slightly, more descriptive ways.
And that's using the relationships. Part of our schema and, Again, I'm going to query the works endpoint, get the query the same endpoint instead of saying has references, says has has relations.
Does it have a relationship and that's That's run nice and quickly, thankfully. It's always always risky doing a live demo anyway.
Here's just one example. And we have here we have the type of relationship.
Do we just pull the relationship? The Jason response. And then it says. It says what the relationship is too as well.
And this is just an idea. I, an identifier says it's a DOI, you can add other types of identifier in there as well.
And we say who it's asserted by. So subject means that it's asserted, asserted by the, the, the member that deposited this metadata.
If you said object, then it would have been asserted by the member who deposited the metadata for the related item.
So it's, you know, there's 2 sides to a relationship and One, this happens in life as well, but you know, one person says one thing, one person says another thing.
And as you know, they don't always agree. And here's just a few more examples.
So we have reviews, we have comments, and these are mostly reviews. This is one with a an article with a preprint.
And we can see that, you know, some of these were asserted by the, the member who deposited the the object the metadata some of them were asserted by for example this one was asserted by the member who deposited the comment and said it comments on this and we match up in our as has already been mentioned we match up the reverse relationship here.
So that's kind of nice, but you might notice that this output looks very different from what we saw before with the references, different kind of.
Of metadata available. So I mentioned briefly that you can link to things outside of Crossf. We have a project called Event Data, which where we go and look for relationships.
From crossraf diocese that are kind of you know mentioned around the web so for example on wikipedia hypothesis annotations, we look on on Reddit and just kind of blogs and websites.
And, I'll just show you one example here. And we're looking at specifically at Wikipedia.
And now the The eagle light among you might notice that, the event data API is being closed down from the end of next month.
This is because we will continue to collect events. But we want to be able to, we want to focus on the new endpoint.
The relationships endpoint. As you'll, which you'll see in, in a moment.
And the best use of our resources is to Stop is to stop with event data put all of the event data into the relationships endpoint and make that usable for for the users that that are doing that so if you are using event data you want to look at the at the relationships end point.
As, yeah, we're in the next few weeks. And so this is an example of an event and you'll see again it looks very different to the references.
It looks different to the relationship. Metadata that we saw. But essentially we've got the same kind of information.
So we got an identifier for the subject. In this case, it's a URL.
Wikipedia page. We have an identifier for the object, which is a Crossworth DOI.
And we say how these 2 things are linked together. And this is as a reference. So you've seen here this 3 different ways that we essentially deliver the same metadata.
And you know, we've been looking, a great deal in, you know, in the past few months and years.
The research nexus. And that doesn't really preference any kind of item or any kind of relationship.
And so we said, well, why don't we reflect that in our APIs and provide all of the relationships that we know about.
To a single endpoint and in the same format. And this is what we've been been building for a while now.
And as I said at the beginning, this is the, this is the URL. And I'm just looking for, you know, a single day.
So we've got, we've got a number of of filters, time filters for example on the on the output and you see on this day they were about 2.4 million relationships.
I'm not going to show you all of those but this is just one of them. And Okay, we've we've tried to pair it down to the the minimum kind of useful information and we'd love feedback as to whether this is sufficient whether we need more, whether we need less and what would make this useful for for you to use.
And so we have a subject. So this. Relationship comes from across F. DI.
And in this case, It's going to, an organization. So this I can tell you because I've looked at this way too often is Wiley and it says that this DOI was published by Wiley essentially.
And this information was asserted in instead of just saying subject to object or or a word, we're using a raw identifier for Cross W.
So basically crossf is asserting that Wiley is, is permitted to, to manage the metadata record of of this DOI.
So in the first instance, we are focusing on data citations which have been mentioned quite a number of times already so I'm really happy to have heard that it kind of Hopefully, yeah, I don't need to give too much motivation for why data citations are interesting and important.
And and again we're going to query the relationships endpoint. We're going to look for things that have the type data set and we're just going to look for a single day and see what the what we find.
And we find actually there were hundreds and 147 data citations. Or, found on this day.
Now, this, what this means is subject type data set. This is with the capital D is the is used by data site.
So these are actually not data citations deposited by Crossworth members. These are data citations of being deposited by data site members.
They very kindly pass that information on to us and we're able to make it available. And through the through to our endpoints and you know we do that reciprocally as well.
And let's just have a look at the the output there so we can see you know, there's various relationship types which are used.
So, you know, this This, this. This TOI here, so this one here is is metadata for this crossf DI.
This data site DOI is posted content which is so yes, this is a cross FDI, which is posted content.
And, is cited by, this, data site DI which is in Zenodo and and so on and and so forth.
So of course, I mean, I don't know whether doing a live presentation on this make make sense, but you know, the idea is that this is machine readable and if you want to look up more information about a DOI you can just go to our work standpoint and and retrieve the the metadata for that.
This is really focused on the on the relationships. Looks like I should have, shortened the output for this one.
Anyway, You can, this, this is what I'm not convinced is going to work because this query has been taking rather a long time at the moment.
But you can query for everything which has been registered by data site. and then the output I've, I've done a short, kind of put the output here, but you know, you, this is, this is the output that you will see.
So just to, to show you, you know, more kinds of queries that you can make.
And, so I'm not quite sure how I'm doing for time here, but I carry on for
Yeah, we can let people get to the end what else? In the but yeah just about a minute left be great
Okay, fine. Yeah, so just very very quickly then you can you don't have to look at DOI you can look at orchids you can look at organizations you could this funding metadata in here so yeah, this is, as the orchid was an author of, of that.
This, we can see that cargo is deposited 50,000, references and you know we've got funding information in here as well.
Sorry to kind of blast through that very quickly but I will put this I will put this collab notebook link.
Into the chat and you can have a look and have a look by yourself.
Yeah, thanks. Thanks a lot for your attention.
Awesome. Thank you very much. That was really interesting. There's the link to the notebook for anyone to have a look at.
I am really conscious of time. Maybe one question. We want to take and then we can get to the other ones in the chat.
Sure. Alright, perfect. There is one question maybe you can answer very quickly.
Yeah, so yeah, is event data going to be available in relationships API from day one? And the answer is actually probably not, there will be a gap we have committed to making all of event data available.
By the end of January, 2024. But we, you know, event data for those of you been using it has been unstable for quite some time and we've decided that it's better to focus on something new.
And stable and there may be a gap in which some events are not available. We've really focused on data citation because we know that that's a very important use case and all of data site.
All the data citations that we know about will be available. And from day one, the rest we will add, gradually over, over time.
That's great. Thank you. Nissa, you can, there's a few other questions in the chat that you can pick up.
Yeah, I love.
Thank you. Eric, should we try again?
Yes, fingers crossed. Thank you so much for moving things around for me.
No worries.
Yep, we can see your screen.
Perfect. Okay, so jumping back. To a few minutes ago. For those of you who are unfamiliar with, open internet systems, or OJS that I get just a brief.
Overview of what it is. It's an open source journal management and publishing platform.
It encompasses, submission workflow, peer review, as well as the production workflow.
And it's currently used by more than 30,000 journals of Y. And today I'd like to share with you a little bit about the new DOI workflow and the latest release 3.4 which came out in June ninth but in order to provide a little bit of context around that I'd first like to show you a little bit from the previous version, 3.3.
Set up the why these changes are so big and helpful, especially for people who are managing lots and lots of DIs.
So for those of you who are unfamiliar with OJS, this is the home screen you for.
And an editor would log in. I just want to show an example of
What's some of the pain points are when dealing with the, in the previous version. So to start with one of the major ones are around DIY creation.
It's spread out throughout many different places throughout the application. So for example, if we were to this published article.
Currently I only have article, DI is enabled, but I'll show an example of where all of those would go.
So This is the kind of publication tab for. A published submission. The article DI would be under here.
Under the identifiers tab. If you have a galley, it would be separately under here. And the issues would be yet under a different tab entirely.
Another pain point that this brings about is you can see from this big red banner of this version has been published and cannot be edited.
And this often happens that an article will be published. This is especially prominent in with preprints.
Before a DI has necessarily been assigned to it. And this has previously been impossible. In OJS and encourages bad practices around things like unpublishing it to update metadata than republishing it.
Highlight one other pain point here. Is around the previous DLI pattern and this is what we call the default UI suffix generation in OJS 3.3 and previous.
If you come to assign the, you'll see this warning, which says you cannot generate a DI until this publication has been assigned to an issue.
Just quickly show at a glance what that entails going to the DI settings for this version.
And in OJS 3.3 and earlier we use a pattern based suffix generation, which does contain semantic information about the vitamin question.
The most common one is for articles and it includes information about the issue. The volume number. IDs inherent to it from OJS itself.
And this is often very problematic because DI cannot be assigned. 2 an item until it has been assigned to an issue and this is often very problematic because maybe an item shouldn't be assigned to an issue yet, but you would like to have a DI, like you shuffled around and you want to be able to assign it DOI, it's part of the pre-production.
So. A little bit of background on some of the kind of pain points of DOI. Generation.
Which brings us to. 3.4 and the same view here just as a starting point
So I'll do a quick walkthrough of the process of setting things up as well as show some of the highlights of the new workflow.
So first of all, you'll notice, you, as I featured very commonly on the sidebar here.
And so one of the major changes is that DIs are now a core part of the application. Previously, they were a separate which kind of limited the amounts of integration.
Integration into the application that was possible. Land, the Crossrail, and any of the other registration agency plugins are still.
But this new architecture allows it to be more deeply integrated into the application and I hope as I go through some of that today you'll be able to see this new architecture allows it to be more deeply integrated into the application.
And I hope as I go through some of that today, you'll be able to see some of that.
To start with. Show you what some of the settings look like. Just to provide some context.
DIs can be enabled or disabled. Part of the distribution workflow. Globally here.
Next up, it's very similar to previous versions. Items with DIs. Apparently I only have articles selected.
This also varies across the different, software applications that PKP makes for one for monographs and one for preprints but it's roughly the same idea.
One new thing here in particular is that by default, anything that OJS can create a DI for is possible here.
Now is the article, and this is usually in the most default context that published PDF.
Crossra for the Crossraff plugin, this option is disabled. And this can be configured for other registration agencies if say they didn't take issue DIs for instance.
And potentially more applicable is for monographs if different services do or do not take chapter, or that don't have the capability to distinguish between those types of things.
And I'll show what that looks like here.
Next big thing is the automatic. DI assignment. The philosophy around this was just to be able to generate DIs as early as possible so that there as part of the production process part of the.
Layout process. It's going to be done. On reaching the copy editing stage, which essentially means as early as possible.
On publication or never, in other words, manually. And when we looked at the pain point from OJS 3 3.
What that
Kind of has bearing on for this is that you needed to have assigned things to an issue. That is no longer the case.
The default suffix generation now creates. Hey, unique character suffix. And this is a pattern we from inspiration from.
A data site tool. That would create an 8 character suffix and it includes a check some digit so it can be checked programmatically to make sure there are no typos.
In a similar way to ISSNs or ISPNs. So one of the other big wins is because it's a unique character suffix.
It does not contain any semantic information and there is no temptation for editors or journal managers to want to.
The US to convey any sort of cementing information that they might want to change later, for instance, if the name of the journal changes, they won't want to change their DIY suffix because it doesn't have any information about that to begin with.
Next bit is. All information about the registration agencies has also been co-located here. So the name of the game with a lot of these changes is co-location.
So wanting all the settings to be co-located as well as the actual deal I mentioned itself.
So we go here. We can check which. Registration agency we'd like to Rich registration agency. Right now I have Crossf set up.
That makes the most sense. But all of these settings are co located here. Automatic deposit is still functional.
And all of the settings are here along with some added context about. The username and the roles for setting those up.
Go ahead and enable that. And then move on to the new DIY manager interface. So this interface here will be more or less the same regardless of which registration agency is used if a registration agency plug-in is enabled at all.
And this will be the same across all of the applications. So this was really done in an effort to.
Unify the way that journal managers and editors can work with the UIs. So a few things I'd like to highlight around this.
Are the basics of DOI assignment as well as some of the quality of life things. So some of the first things that jump out are the red badges here.
These will usually Show the top priority things that the journal manager would want to see when they come to the screen.
And this includes things like whether items still need to be assigned to DIs. Once items have been published, whether they are unregistered.
As well as if there are any errors in the registration process. So we can filter things based on.
These types of things. So you want to see all of the items that need to do your eyes.
That includes everything because nothing has DOI assigned at this time.
There is an explanation for what all of these. Individual statuses mean as well as the filters. That can be updated.
That can be viewed if you are looking here at a later time. Also possible to filtered by. Issues and if you're using an issue based workflow can manage everything.
There. One of the other big improvements in terms of the workflow for time. I didn't show an example of this in the pain point section in 3.3, but the ability to do bulk actions.
So. To paint a picture of it, imagine you had 300 items that you needed to assign. Or reregister.
Previously you had to tick all of these manually and this was just a quirk of the design of the forms.
You know, have both actions and this is where the heart of a lot of the functionality is. We filtered for the issue we want.
We selected all and now we'd like to assign some DIs to these.
And we can now see that the status of the, from, from, needs to, to unregistered.
Do I's can be edited under this expanded view if you have more than one DUI for an item, if you are using, or if you're working with monographs and have more.
Types of items that have DUIs, those can all be managed here. It can be edited here as well.
And once you're ready, it can be deposited. So it also like to just take a moment to highlight this as the.
New type of suffix. It is still more or less human readable and trying to strike a balance between it being easy to.
Share and read without having
Without being too long. And still mutating enough, unique. Possibilities. So the current pattern we have.
There's a bit. Hi, I have 1 billion different unique. Combinations that are possible per prefix.
On the off chance that we do run into duplicate issues, a new sub fix can always just be generated for that item.
Although that's quite unlikely.
Before showing the deposit workflow. I want to show you how the automatic workflow works.
So I gave a brief.
And example of that. The previous one. I will show it. This as well.
So take an item. Here that has not yet been through the review process, but will pretend. For moments that this has been accepted.
For publication.
Record that submission and now it is moved into the copy editing. And if you recall, we've now set up DIs to be assigned as soon as an item reach copy editing stage.
So if we go over to the Do I's tab that loads up. We should now see that this item that we were just looking at now has a GLI sign.
This is something that was not previously possible. And ends up causing, a lot more headaches than it might initially appear not having.
DI assignment tied to any semantic information about OJS metadata is a huge way for us and consequently It's fewer problems downstream for registration agencies like Crossf.
As a final that I'd like to just show. The register. Registration process. One other pain point from previous versions was that when you were trying to submit multiple items to be registered.
They would often time out and it was not always clear. What the threshold for that was. The registration process has been reworked to use a jobs based system.
So for each of these items that we want to. Deposit. They will now be queued in a jobs queue.
And so if we go here, what that means, go to deposit them. UI feedback should be instant and they've been queued for deposit.
And the XML generation and all of that slow process will happen one at a time. And as those are updated, we refresh the page.
We should see. Once those processes have finished that.
The deposit will be successful. I won't make you watch that because that is a very long process sometimes, which is the whole point of moving it into a background job.
And I believe that is all that I wanted to share with you today.
Thank you so much.
Great. Thanks, Eric. We got it ruling in the end. So I think there's a question that's just come in.
Yes.
From Tom. So yeah, this is like the automatic assignment.
Yes. Yes. The way it works is copied the copy editing stage would be the first opportunity.
For things to be assigned. When it moves to the production stage, it could also be assigned. So the idea is that This accommodates workflows where, editors and journals.
Either use or do not use the copy editing stage but one way or another it will have to go, either the copy editing stage or the production stage.
Before being published. So. The automatic assignment will it will attempt the automatic assignment at both of those stages.
And this is also true if the legacy pattern generation is also still in place because obviously that would require an issue to be assigned.
So it will first try at the copier editing stage. If it skips that, it will try at the production stage and if it gets that.
Or if it's still not possible, it will try again on publication. So it will try as many times as possible, but we'll rewrite.
Okay, that answers your question.
Awesome, thank you. I'm conscious of time. There's another question that's just come into the chat and I think probably my question is around just where people can get information on upgrading to OJS 3.4 so they can take advantage.
A lot of improvements I think will just really help kind of ease a lot of those kind of pinpoints.
So yeah, appreciate it. Thank you.
Yeah, absolutely. I'll put my email if anyone has any questions and I will also link out to the release posts about OJS. 3.4.
I'll add that to the chat.
Excellent. Thank you. So at this point I'm gonna hand over to, Luis Montier who's been
Who's been leading things on the technical side.
Sure. Oh, can you share my slides for me, please?
Yes, I can indeed. Let me just grab those.
Thank you.
Oh, sorry, 2 s.
Sorry, I've got these.
Okay.
I'm sorry, I just need to jump through to your slides.
Yeah, if you want to go ahead and introduce your session then I will I will catch up with you.
Sure, yeah, let's start. Perfect. Thank you very much. Well, thank you, everybody, for being here.
My name is Liz Montilla. I'm the technical community manager at Crosscraft. And today I will be talking about, well, giving up a very basic introduction through the Crossraff API.
Which we would start in the minutes. Next one, yes, perfect. Next one, please. So this presentation will be about Api's, but more importantly, it will be about the possibilities that the community has to interact with the best network of relationships between research objects.
Which is what we call the research In more practical terms, we are talking about metadata.
Which are the basically the descriptors of research objects and how we can establish relationships between them. Next one, please.
So, we make this metadata open available via our APIs and our public data files.
Enabling the, paper machines to incorporated into their research tools and services. And at the same time, we collect and we operate distribute this metadata.
As part of the ever growing research Nexus. It's safe to assume that not everybody understands what an API actually means.
They're for the purpose of this presentation it's to provide a quick introduction to the use of the Krosrov API.
So if we see the slide, we see then, an API is an software intermediary that allows 2 applications to talk to each other.
And Api's are often compared to a weather service facilitating communication between customers or during items from a menu and the kitchen producing those items.
Next one, please.
So we make this metadata open through our REST API. You can visit the URL to access or documentation.
I also shared the link in the chat. And the presentation before. And then you can use this to perform chorus directly from your web browser.
There is no registration required. Cross riff once again it's committed to providing an open and anonymous as possible access to scholarly metadata.
And keep this your hand for the next examples, please.
However, we do have some recommendations and a ticket suggestions to follow if you wish to retrieve metadata.
First by adding your email address to your request, we can contact you in case of issues. Then additionally, we, redirect this request to an a specific polite.
A pool of servers. These servers are generally more reliable because we can more easily protect them from misbehaving scripts.
In contrast with full anonymous requests. And then there is a third alternative, especially if you are using our REST API for a production service.
That require high predictability. And this option is to consider using our our paid plus service, which provides you with an authentication talking that directs you to your request.
To a pool of service that are extremely predictable. Next one, please. So for the specific examples I'm going to show you next, I'm going to use Postman, which is an API client.
We don't endorse this app, but we acknowledged that it is widely used and it provides a set of functionalities that makes metadata retrieval much easier.
However, you can also execute these words directly in your web browser once again. In most cases, your browser will display this for the results as in adjacent format.
However in some instances you might need to install a browser extension.
Next one, please. So before moving on, I would like to take a second to establish some basic vocabulary.
They will. That you will encounter throughout these presentations. So the text that you see here that at the top represents a query.
The first part that you see colored in yellow indicates the server. So this identifies the entity providing the service.
And all of our crossraf related queries we share these common root. Then you have the endpoints.
Or this is a key term that we will use to identify the digital locations that receive a connection. And I will show you soon the other rank points that we have available to the general community.
And then finally we have the parameters. You can identify them as being preceded by a question mark. And then they follow the notation of having a key.
And, value. So for example, in this specific example, you can see a query that has the mail to parameter.
And in this part basically allow you to specify resources. You are retreating and also to perform all directions such as in this case identifying yourself.
Next one, please. So if we check the documentation, we'll find that list of different locations for the endpoint that our API contains that of interest.
In this presentation, I'll show you some basic examples of the data that you can retrieve using some of these.
Perhaps you know this in the previous slide that I was using the funders endpoint. But we have endpoints specifically for journals, works, grassroots members and several others.
That's one place.
Now let's explore some basic examples. We'll start with, funder 2 article relationships.
This is interesting because funders do not automatically know when the work that they have funded is polished. And it but it's also essential for reporting.
On the impact of grants. And then finding this information can be challenging through other means. Because offframe, publishers, customers and institution institutions do not systematically report it.
Next one, please. I'm going to, this is not going to be a live demo.
This is going to be through, done through screenshots. So as you can see here, we include in the top part the string that contains our server the endpoint that I was mentioning before the funders endpoint.
And other parameters. In this specific case, I'd like to start with a very general search. So I include the mail chief parameter to make a product request and I add a query with the text German Research Foundation.
Next one, please. So when you, submit this query, you will receive best, status code.
In this case, everything, it's okay. And then we luckily we can see how many items we are retrieving with this square, which I'm highlighting here.
You can see that we are withdrawing 4 results. Next one, please. So if we examine, hopefully it's very, they have a good resolution, but if we examine the output in detail, we will notice that our per peers as part of the field alternative names.
And organization that we are looking for appears on top. And then from here, we can take notice of the ID number.
Then you can use this to refine our queries. Next one, please.
So for example, if we use this ID as part of the an additional path level in our funders and point.
We can retreat organization specific data. For example, here we can see that almost 200,000 works have the the GRF as part of their funding organization.
Next one, please. Thank you. So we can take a few seconds in case you want to try this or of course if you're watching the recording you can post their presentation.
So please visit this URL. You can scroll down to the funders section. Click get, please remember to add your image to the major field.
I'm finally included of any funding organization that you wish in the Let's have a few seconds.
Okay, next one, please.
So this screenshot shows one possible result. Please notice that you should get the Status code 200 indicating that your query proceeded correctly.
And what it's called here to response body contains your data.
Next one, please. If you're pasting these squares directly in your web browser, at the search war, it will likely look like this.
The latest versions of Firefox and Chrome and several others should let you visualize this Jason file.
I was doing some test in Safari and then it was one the one required to selling an extension. Consider this.
Next one. Thank you. So we can pro this output a little bit more by adding additional, field queries.
Once again, these are listed in our documentation. Some of these accept text rings, some others are numbers, others are 1 billion expressions, meaning true or false.
And of course they can be combined different ways. Next one, please.
So for example here, let's imagine that we are interested in knowing how many of the selected organizations funded journal articles are publicly available.
So in this case, we can add one filter. That includes the type element. In this case, we are setting these 2 journal particle.
And we are also adding the filter that asking if this contains a full text element and we are setting this to false.
Next one, please. So you can try combining some of this. I'm showing you here the example that I just used.
And then, of course, we can combine these fields more. So for example, we can also explore, we can query the specific the grants that is available in the in the work lists.
Next one, please. We can also facet, aggregate data using the facet fields. In this case, we can use this, for example, to aggregate results in terms of the publication year.
Next one. And here I'm showing the example in case you want to replicate this examples.
And what with this slide that will finish my presentation, feel free to contact me. I will add my details in the chat.
And of this presentation was kind enough to anyone who was not aware of how to use an API before. Thank you very much.
We, that was excellent. Thank you very much. And again, just to reiterate, we'll share the slides and the recordings so that you can you can dig into this more.
Louise joined the the team at Crossraff a couple of months ago. And so if you enjoyed that demo, and there'll be loads more to come in future and just help people get to to grips with how to use the the crossraf metadata.
I'm Game questions in the charts or in the Q&A and we'll take those up In the meantime, I'm going to keep screen sharing.
And I'm going to talk just really quickly about the the work that we've been doing with the with the data that's come from retraction watch and Martin Eve is going to help me out whenever I get.
But it as in when I get stuck. The
Just for context in case you weren't aware, Crossraff acquired and opened the retraction watch data in the middle of September.
And I've linked to the blog posts that that describes that in more detail. The other thing I will do as well is for detail.
The other thing I will do as well is for the context of. The other thing I will do as well is for the context of why this information is important, etc.
Last month as well. So we'll share those which will give you lots more context than we have time to to do today.
There are lots of things that we know and expect our community to do with more comprehensive complete information on retractions.
And so We wanted to make the data available. A, you know, in a very simple CSV format, but also via our labs API so that people can start to get to groups with it early doors.
So As I said, as well as the. Csv format. We made the, our more more accurately Martin made the data available via our Labs API.
What I mean when I say the labs API is that it's a it's an environment where we test stuff out.
And adding new data, a new representations of existing data. And I think the main message of this presentation is it's where we'd love some feedback on how we are displaying.
And representing the the data. There's a little snapshot of it over to the the right hand side of this, this slide.
As Louise mentioned, we asked you to use a meal too, so your email to, when you're accessing the the data and the information is available as you can see via the via the works route in the in the labs API.
Pulled some of the information out just to try to make it a little bit clearer. Martin, I'm happy again.
You can talk to the pieces that I've missed. But what sort of jumped out to me is someone who uses and is really interested in this is that we've got a couple of fields here.
So we've got the the roller ID of the Center for Scientific Integrity, who are the, the folks behind the retraction watch data.
So I think that's a that's a a nice thing because again we've got rural identifiers we want to be able to use those 2 uniquely and accurately identify institutions.
We've got We're trying to be really clear about who made the, about who made the assertion that an article has been retracted because in many instances we've got, attraction being asserted by retraction watch.
That isn't currently reflected in the publisher metadata. And again, of course, we really hope and expect that that will that that will change over time.
And we're thinking about how we model retractions, full stop. So said Martin implemented the workflow and crunched the retraction watch data to make it available via these routes.
So thank you very much. But do you want to say more about the current implementation and the plans going ahead?
Sure, so there are some limitations around what we've got at the moment. So for instance, it's not easy to do filters in our labs API system.
So this should not be at this point a full replacement for the production API and and you shouldn't expect the same same stability or performance.
What it does do though is let us test whether the information that we're representing is in a format that's of use to our community.
So what I would urge is if you are someone who's been making use of their attraction, watch data and you're thinking about using it in an API format.
If you could have a go at using the Labs API and seeing whether it meets your needs and giving us feedback on what's missing, what you need, what's there, what's good, that would all be incredibly helpful for us in thinking about how we move this through to production.
At a later stage.
Is that enough, Rachel? I'm not sure what else I can add there at this point.
Okay.
I think that I think that that's more than enough and we will again will pass on information on how you can get in touch with us about that and I know that lots of people have already so I think that's really important.
And again, please feel free to jump in to the chat or Q&A and ask us questions about the data and we'll also post a link to the the webinar from a couple of weeks ago to do that.
So we just want to flag this off. This is an area where we're really king for, for feedback.
So, So, so do go ahead and give us that. I am.
I am going to, so Martin, I'm gonna leave you to your own devices for the next demo.
So, so you've been doing work to analyze the present preservation status of content registered by Crossraff members so I'll leave it to you to share your analysis and your thinking around that, LY.
Thank you.
Thank you. Very much. Just bear with me a second while I try to work out. I'm gonna share the correct monitor or whether I'm gonna show one all my emails.
Let's try not one. Have you got a presentation there, Rachel or.
Present presentation and no emails. So perfect.
That's, that was good guess then on, on the one of 3, okay. Thanks, everyone.
So I'm gonna talk today a little bit about the state of digital preservation. And a study that we conducted across 7 million DOIs.
This is a really important aspect of crossrest practice that sometimes goes under remarked upon, which is that for Crossraff to provide a stable linking service between different platforms, we need to know that.
Material has been safely digitally preserved. Why is that? Because if the original source goes offline, Where's the DIY going to end up pointing?
If there's no digital preservation source to which we can redirect DOI. Then actually the persistence of our URLs is limited to the lifetime of an original.
HTTP endpoint.
So when you join Crossref, one of the conditions membership is that you will make best efforts to ensure that your material is safely, digitally preserved.
But we don't know what proportion of scholarly objects assigned to DIY are actually adequately preserved in a recognized dark archiving system like clocks, locks, portico or even the internet archive.
We don't really know how stable those preservation systems that underpin the persistence of these persistent identifiers is at the moment.
We haven't yet had a mass extinction event of of DOIs or anything that has really you caused us to use those systems on mass.
We don't know which actors of our membership are behaving well on this theme. Who does dish to preservation well and who does it badly?
We don't even really know in the area of this door who should be doing this. Is it libraries?
Is it publishers? Has that changed since the print era? Should it be both of those parties? And most importantly, perhaps, with a slightly apocalyptic tone again, is it already too late?
Is vastly the material that is not preserved that we're now going to struggle to get into some preservation mechanism at this late stage in the day.
So I said about building an item level preservation database system that would allow us to iterate over a sample of DOIs and to figure out where these things are preserved.
There's an existing system called the Keepers Red Street that performs a similar function, but it doesn't have an API with open use that we could just use to query at scale.
A large number of DOIs, it also looks at the container level. So it will tell you this issue and this volume of this journal are preserved, not whether this DOI is preserved.
So I had to build something that would translate between those layers. But I built a system that incorporates the archives that you can see on this slideshare.
So Kari and Iana, which is predominantly southern Latin American material clocks, control lots of copies keep stuff safe archive.
Happy trust, internet archive, locks. The more general version of clocks. PKPs private locks network which is a system at PKP implemented to help open journal system users ensure their material is safe.
At Portico, another dark archive. And the Oakhall Scholars portal in Canada.
I also needed some way of figuring out how to score members and how they're doing on digital preservation.
I mean, it's all very well to just say this is what's going on, but you need some way of ranking them and understanding the categorization.
So I invented a scale because there is no standardized scale for digital preservation. Where, I gave gold medals to those that have 75% of their content digitally preserved in 3 or more recognised archives.
Silver members in my scale of those with 50% of their content in 2 or more recognized archives.
Bronze, 25% in one or more recognized archives. And unclassified those that don't even meet the bronze categorisation.
So this is a scoring mechanism that works across 2 axes at the same time. It works across a percent preserved and in a number of archives that are a duplicate preservations in.
So the very best in this would be to have all of your content in 3 or more of the recognized archives that I've been working with.
Whereas the worst would be you have less than 25% not even in one archive.
So what does it look like overall? I looked at 7 million DOIs from our sampling framework and I found some pretty disturbing results in some ways, but perhaps not unexpected.
Those who've been looking at this for a while. So those in the real danger zone, the unclassified space with that less than 25% in one archive we found that approximately 33%, third of all the content.
From members that I looked at was in that group. Meanwhile, bronze, which, you know, is, is okay is one archive with at least 25% in it made up.
Approximately 57.7% of the members that we we examined silver again it's a much smaller percent 8.4 6 and only about 2% or so made it into that gold category.
The gold star system at the top there. So there's a worrying preservation picture emerging across Crossrail members that we need to perhaps do something about.
I want to know also which types of members were behaving in in what type of way. There's the variety of ways in which you can classify the, size of Crossrah members.
We can do it by revenue. So this is a chart that shows the breakdowns from those who are the very lowest levels of revenue at the top through to those that have the highest level at the bottom here.
And I think what you can see is the silver bar, which is perhaps the a key sign that people are doing things well with at least 2 archives backing the majority of their material increases substantially as we get down to those with more resources.
Perhaps heartening though, even in though even among those with the lowest resources of our members, we do see that bronze bar where they are conducting at least one source of digital preservation for for some of their material is happening.
So there is an awareness growing. Even among all types of publisher members that they should be doing digital preservation and that it's something that does matter for their membership with Crossreath.
You can also do this by number of member deposits, but I think the picture again is clear. You can see this silver bar grows substantially as we get down to the bigger members who are doing more deposits.
The bronze bar likewise, it stays Kind of goes down again at the end because the silver bars eating it up, which is a good sign.
But if you take bronze and silver collectively as a marker of who's doing preservation, apart from the very last bar of size, we've actually got a good level of growth as we go down there.
But clearly there are a number of smaller members here in this type of categorization who need us to to have a chat with them about what they're doing for digital preservation because at the moment their material is not particularly safe.
If you look at this on the works front, so that was looking at members as a whole rather than looking at the number of actual works though.
So we found that approximately 60% were preserved at least once. So 60% of the content in that sample had some kind of backup.
System in place that could rescue it if the site went down. But that left approximately 30% in our sample that seemed unreserved and at serious risk.
With an exclusion of 14% for being too recent. So they're in the current year so they weren't yet in digital preservation archives not being journalized calls or having insufficient date mess data for us to identify the source.
So what are we gonna do about this? We've got a forthcoming peer review paper that sets out the the data behind this and gives you the opportunity to read in more detail about the processes that we used to come to these conclusions.
We've got some lay and easy read summaries of the preservation situation coming out as well so that we can start to spread the word even among non-technical users or less technically minded publishers.
We'd like to conduct some direct member outreach. I think it's really important that we in a non-confrontational way get in touch with people who are not yet doing this well and work with them because it benefits everybody to have a secure and persistent linking.
And other types of work that use DOIs. We're also working on an experimental system group project opposite, which I've just started work on in the last couple of weeks.
This is a project that aims to integrate DOI deposit with ingestion into digital preservation archives.
And then tries to monitor an endpoint. So a DOI resolution point to see whether it's gone down.
And to tell people this is a problem, you can now view it at this archive instead if you like.
So if you Google for that project opposite and crossref, you should find the specification document and call for call for comment on that.
If you want to feedback on that, it's not too late. Still get in touch. And full charts data set behind these findings.
Are available at the website there. The hyphen vault.
Thank you very much. I hope that was of interest.
Excellent, thank you. So I think there's already interest in going to zoom in on the the charts and to be able to find more information on that.
So, Rhiannon, do, go and check the link that Lisa has just posted in the, in the chat and Again, if you've got for up questions.
That more? No.
Thank you.
So.
I'm Martin, are you hanging around for sort of the next sort of 1520 min? So if you're mulling over a question, he'll still be, he'll still be here if you want it and answered more, more live.
But yes, is it it's it's it's a really interesting analysis and there's definitely follow up options that we can see both as crossrefonds as community I expect.
And there's a question just jumped into the, into the QA as well.
I think the this is because of bad mess data on locations. So the question was that there are cities sometimes or regions in the location or what we've called country because it's supposed to be a country field but sometimes people put more detailed information in there.
So it's the best but you know i could spend a lot of time clearing up that data and going through but with with 7 million records filtering in you just have to basically take it with a pinch and salt the location fields in there and we've done our best on location.
Thank you. So final presentation struck demo for this session. Is the DI for for static sites.
So I should data from our, market. Labs team is going to give us a quick demo, an overview of the Cellic site.
Okay, identified generator using the Crossreft website as a test case. I appreciate a semi-live demo of stuff that is in active development, but again, I think you've put some see if you've got, you've put some safeguards in place as well.
So I'll hand over to you to talk us through it.
Alright, great. Thank you so much, Rachel. I'm gonna share my screen.
Can everyone see my screen?
Yeah, we can see the slide deck. Okay.
Yeah, that's great. Yeah, so I'm going to just kind of talk about this a little bit and then do a video demo because some of the processing takes a little time and that's boring.
So basically I am going to give a demo of this lovely wordy piece of software called static. Page ID generator because I couldn't think of a better name.
It is developed to help create pits such as DOIs for blogs or other research materials built by static site generators such as Hugo, Jackal or others.
Currently the software is being tested with Hugo. We decided to develop the software because increasingly at Crossraff we are seeing a need to develop permanent URLs for our own use on our website and could see a use for the wider community as well.
Our website is built using Hugo. The software
Can track a Git repository of the static site in question. And if a user specifies a path, it'll start focusing on any files under that path.
It'll generate unique IDs for the files that a user wants tracked and if the user is a cross roof member it'll build the XML deposit files.
Deposits the metadata for the files into Crossraff. Registers the DOIs and as the DOI back to the filing question.
If you're not across staff member for now, basically all it'll do is features one and 2, which is tracks the files and generates a unique ID.
But we are developing a plug-in architecture so that it can accommodate other use cases and other registration agencies and also other types of unique IDs that you'd want.
So I'm going to start with the end result of this, which is that here is an example page of the test version of Crossraf.
Of the Crossworth site. So I basically just created this very lovely test page and ran the script against it, which then deposited this, this metadata into Crossraff, which you can see in the API.
And here's the DOI for that. And it can also then be used in downstream applications set up such as the metadata search, or if you did a search on this, you would find the same.
Article and if I go back to it here's the file. So basically I'm going to show you how to show you 2 ways of how I use the script.
First is in the Get Lab repository, which contains the test version of the site. And then the second is the command line version.
In this repository, we run the script using Gitlab's continuous integration setup.
And if you use, if you have a repository that's in GitHub, it has a similar process.
So I first ran in the Get Lab repository a set of process that creates a config file which contains the information.
Let me. I have it queued up in my video. So here's the config file. So here you see that I'm asking the script to track everything in the repo.
I've put in a test to track everything in the repo. I've put in a test DOI prefix.
I have this crazy long domain, which is basically just the test version of the sites domain. I'm specifying that I want an ID type of DOI and the JSON file basically contains all of the information of the files that are being tracked.
And now I'm going to play the video. Not sure how much time we have, so I'll check in again after I've finished the get the repository part of the demo and if we have more time I can also show you the part of the demo.
Cool. Yeah, I think we're okay for time.
Yep.
Okay, cool. The JSON file contains. Start tracking a file, I created this file which is new and basically added some front matter so it includes some of, some basically labels that is recognized by the Hugo site and also crucially I added this X version tag which basically tells the script to start tracking it.
The value to the right of x version is in semantic version style. So basically the first 0 tells you that this is the major version.
The next one is the minor version and this is the patch. So, the script looks for a default version of 0 point 0 point 0 and it knows that this is a new file and to begin tracking it.
So once I push this file, I start to get a process that runs this job and it pulls down a darker image and runs the script essentially.
So we are mentioning giving some credentials for it and then we are also saying that it needs to check the repository.
Over here that the submission type is Crossraff. And that it needs some more submission information.
Our deposit information for the script to deposit the metadata that you will be using eventually. So what it does is it adds the file information to the JSON file, it deposits the XML.
And then it also creates a DOI and adds it back to this file. And you know that all as well because the job has succeeded.
It has run successfully. We can check the page back again. And we can see that a DOI has been added to this page.
And we can also see that once the site has been deployed to production, if we click on this, and go to this DI, it takes you to the site.
To the page that's been deployed to production. So this is basically how you would run it.
And continuous integration on a Getlab hosted repository. There is a similar operation that you would do if it was posted on GitHub as well because GitHub also has a continuous integration.
Setup.
We're running the script on the command line. I have cloned the static page ID script generator on to my computer.
And, I have installed all the requirements in a virtual environment and in another Terminal, I have my static site repository.
I first need to initialize the repository for the script, which generates 2 files that helps the script track the version files.
So to run the initialized script. I run this command, which requires a few arguments that tells the script a few things.
It tells the script that you to track this particular repository. To track the path in the repository which is content it gives it the production domain that it needs to generate a DOI.
And a DOI prefix. So once that is generated, it will create 2 files. This is to track the files.
And this is essentially contains all the values needed to generate DOIs. So I already ran this and so there is a config.
YAML and a PID. JSON which looks like this for the config. YAML file and it's just an empty pit dot JSON file because I haven't run anything.
So in order to let the script know that I want it to be tracked. I add this particular tag with its value in the front matter.
So I tell the script that I want this to be versioned and I add an expression tag to it and I follow the semantic versioning.
Style which is major, major version, minor version and patch version. So currently we have functionality that looks to see if the version number is 0 dot 0 and it's belonging to this tag and then it will start to run all that's seen run the script after telling, the script which files we want to be tagged.
And we do this by again specifying the repository of path. We're telling it that the submission type is going to be crossrapped because we're going to deposit this into Crossra.
And then we have another file that tells the submission scripts some more information to make the submissions successful.
So it's going to generate. Some, it's going to track the files. And it has now submitted the file and now it's going to check the DOI registration.
Once it runs it, it checks to DOIs and then it adds the DOI back to the file, which you can see.
Over here. So now we know that this file has a DOI and can be accessed once this once the website has been deployed.
Production. So that's how you would run it in the CLI. I wanted to show you what it would look like if I.
Check the DOI from the command line. So this is a page that has actually been deployed.
To production, which is the test. Website that I have so if I Copy and pay the paste this or just follow the link this will show me the page that has been deployed.
And as you can see, the DOI is listed on the page here and this is all of my test, of course.
So that's basically the demo for this. Thank you for bearing with the video and the extraneous sounds.
Future steps include that it's still an active development and we still have a ways to go including incorporating it into the.
Of website workflow. I'm planning on converting this into a library and allowing plug-in functionality so that it can accommodate.
Other types of use cases as well. And I'm happy to hear. More feedback from all of you and you know see how we can collaborate better on this.
Thank you so much.
That's great. Thank you very much. It all worked very, very smoothly. And there are a couple of, hopefully, quick questions in the Q&A.
If you want to call those up.
Or I can just
Cool. Yeah, is it the, do you have thoughts? Is that the one?
Yep, the thoughts about the, on the archiving of the final rendered static page with images, for example.
That's a really good question. I haven't really thought about it, Yeah, that is that is super interesting and also yeah we could also talk to Martin Eve about possibly looking into how to preserve this and so that's a that's a nice tie in into that for sure.
And then there's the second question about, or actually Martin, I will let you jump. I've seen you've unmuted.
Let's take a bit more time on that question.
I was just gonna draw attention actually to Martin Fenner's work that was mentioned earlier today I think in a different session.
That's been looking into the preservation of blogs and static sites. So there's already some experimentation around the preservation function here.
But you're right, we've already some experimentation around the preservation function here, but you're right, we've got an ideal point now if we're assigning a DI to these items to think about how we might get them into an archive so that we know that the stuff is preserved at that point.
Because obviously if we just open the floodgates and say everyone can have a DI for this content but then unaware that there's that preservation responsibility then we're in quite a difficult position.
And then there's another question. How does this tool handle multiple resolution?
Yeah, again, a very good question. Currently it's still very much an active development.
So we should definitely talk more about all of those types of use cases.
Great. And I will give people a few more minutes. The next session will start on the R.
I'll give people a few more minutes in case they're any final questions, but I don't want to miss the opportunity to say thanks to all of our fantastic presenters and also for for your really good questions.
I hope this gives a flavor of the the variety even just a small subset of the things that we are working about and thinking.
Across the various teams. Bruce has just posted the link to the next session which is a discussion among crossress staff and other members of our community on what we still need to build a research nexus.
So I hope we hope that you'll that you'll also join us for that.
Yeah, if you want to take a few minutes. Ask any further questions or go and get a cup of tea, coffee, glass of water. And this is your chance.
And thank you again.
Okay, I can see people starting to. Starting to head off and take up the opportunity of a of a coffee again, there are lots of mechanisms by which you can, you know, by which you can get in touch with Crossraff.
And our community forums a great, resource for that as well. So this isn't kind of the end of the opportunity to ask questions and and to find out more information.
And obviously we'll we'll follow up by posting the video and a link to the the slides in the coming days once we've pulled everything together.
So I'm going to leave it there. Thanks again and we hope to see you in the next session.