{"ID":109321,"post_author":"26","post_date":"2023-10-02 13:01:15","post_date_gmt":"2023-10-02 17:01:15","post_content":"","post_title":"LIMSjournal - Fall 2023","post_excerpt":"","post_status":"publish","comment_status":"closed","ping_status":"closed","post_password":"","post_name":"limsjournal-fall-2023","to_ping":"","pinged":"","post_modified":"2023-12-15 18:16:09","post_modified_gmt":"2023-12-15 23:16:09","post_content_filtered":"","post_parent":0,"guid":"https:\/\/www.limsforum.com\/?post_type=ebook&p=109321","menu_order":0,"post_type":"ebook","post_mime_type":"","comment_count":"0","filter":"","holland":null,"_ebook_metadata":{"enabled":"on","private":"0","guid":"1C7E2BE5-8039-488F-AB6C-665364184FAE","title":"LIMSjournal - Fall 2023","subtitle":"Volume 9, Issue 3","cover_theme":"nico_4","cover_image":"https:\/\/www.limsforum.com\/wp-content\/plugins\/rdp-ebook-builder\/pl\/cover.php?cover_style=nico_4&subtitle=Volume+9%2C+Issue+3&editor=Shawn+Douglas&title=LIMSjournal+-+Fall+2023&title_image=https%3A%2F%2Fs3.limsforum.com%2Fwww.limsforum.com%2Fwp-content%2Fuploads%2FFig1_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg&publisher=LabLynx+Press","editor":"Shawn Douglas","publisher":"LabLynx Press","author_id":"26","image_url":"","items":{"ab540fa1cc32f92043343fd9cf67ec0d_type":"article","ab540fa1cc32f92043343fd9cf67ec0d_title":"Registered data-centered lab management system based on data ownership security architecture (Zheng et al. 2023)","ab540fa1cc32f92043343fd9cf67ec0d_url":"https:\/\/www.limswiki.org\/index.php\/Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture","ab540fa1cc32f92043343fd9cf67ec0d_plaintext":"\n\nJournal:Registered data-centered lab management system based on data ownership security architectureFrom LIMSWikiJump to navigationJump to searchFull article title\n \nRegistered data-centered lab management system based on data ownership security architectureJournal\n \nElectronicsAuthor(s)\n \nZheng, Xuying; Miao, Fang; Udomwong, Piyachat; Chakpitak, NopasitAuthor affiliation(s)\n \nChiangmai University, Chengdu UniversityPrimary contact\n \nEmail: nopasit at cmuic dot netYear published\n \n2023Volume and issue\n \n12(8)Article #\n \n1817DOI\n \n10.3390\/electronics12081817ISSN\n \n2079-9292Distribution license\n \nCreative Commons Attribution 4.0 InternationalWebsite\n \nhttps:\/\/www.mdpi.com\/2079-9292\/12\/8\/1817Download\n \nhttps:\/\/www.mdpi.com\/2079-9292\/12\/8\/1817\/pdf?version=1681215589 (PDF)\n\nContents \n\n1 Abstract \n2 Introduction \n3 Related work \n\n3.1 Data ownership security architecture \n\n3.1.1 DRC \n3.1.2 DAC \n3.1.3 DEC \n\n\n3.2 Laboratory management technology \n3.3 Key system methods \n\n\n4 Methodology \n\n4.1 Laboratory-related data and persons \n4.2 Data encryption registration and application process \n4.3 Confirmation and authorization of the data transmission process \n\n4.3.1 Mutual trust - Single-layer encryption \n4.3.2 Data users do not trust the data source \u2014 Double encryption \n\n\n\n\n5 DOSA framework components - Managed laboratory data linked to rich soils \n\n5.1 Overall framework \n5.2 Laboratory accident searching based on DOSA \n\n\n6 Experiments \n\n6.1 Experimental environment \n6.2 Simulation key generation and transmission experiment \n6.3 Confirmation and authorization experiments \n6.4 Evaluate results \n\n\n7 Conclusions \n8 Abbreviations, acronyms, and initialisms \n9 Acknowledgements \n\n9.1 Author contributions \n9.2 Funding \n9.3 Conflicts of interest \n\n\n10 References \n11 Notes \n\n\n\nAbstract \nUniversity and college laboratories are important places to train professional and technical personnel. Various regulatory departments in colleges and universities still rely on traditional laboratory management in research projects, which are prone to problems such as untimely information and data transmission. The present study aimed to propose a new method to solve the problem of data islands, explicit ownership, conditional sharing, data security, and efficiency during laboratory data management. Hence, this study aimed to develop a data-centered lab management system that enhances the security of laboratory data management and allows the data owners of the labs to control data sharing with other users. The architecture ensures data privacy by binding data ownership with a person using a key management method. To achieve secure data flow, data ownership conversion through the process of authorization and confirmation was introduced. The designed lab management system enables laboratory regulatory departments to receive data in a secure form by using this platform, which could solve data sharing barriers. Finally, the proposed system was applied and run in different server environments by implementing data security registration, authorization, confirmation, and conditional sharing using SM2, SM4, RSA, and AES algorithms. The system was evaluated in terms of the execution time for several lab data with different sizes. The findings of this study indicate that the proposed strategy is secure and efficient for lab data sharing across domains.\nKeywords: university laboratory management, data sharing, data ownership safety architecture, conditional sharing, security and efficiency\n\nIntroduction \nThe importance of knowledge for organizations is now widely recognized, being one of the resources whose management influences the success of organizations through the exchange and sharing of information, knowledge, and experience among its members. Numerous challenges are associated with the management of laboratory data, as labs generate a lot of experimental and management data on a daily basis. Frequent experimental accidents and the leakage of hazardous chemicals is worthy of attention. Supervision departments, such as the Education Bureau, Emergency Bureau, and Public Security Department, need various types of laboratory management data to report while supervising laboratory safety.\nAt present, existing laboratory management systems used by each university, regulatory department, and even various laboratories are different, which leads to obvious problems when lab data collection or emergency response is needed. For example, which laboratory management data are classified confidential, and which data can be handed over to relevant departments? Who must claim the ownership of laboratory-related data? Which departments can view or use the data? Could lab data be transferred to other departments? These questions raised laboratory data ownership issues. Secondly, some data that relate to laboratory management need to be kept confidential. Where and how can we best secure this data? Is it safe for different departments to transmit data? How can we guarantee that there will be no data transfer to others? These can be summarized as data security storage and transmission questions. In recent years, frequent university laboratory accidents have occurred; in such cases, emergency responding units and other regulatory departments need to collect real-time lab-related data. If it is still a regular report retrieval process, it cannot meet the requirements of time efficiency. So, we need to find efficient and trustworthy methods to solve this problem of obtaining data in a timely manner.\nThe main reason for the above problems is that laboratory management data ownership is unclear, and the existing laboratory information management systems (LIMS) of each branch are independent. It is impossible to collect real-time, tamper-free, statistically accurate data when it is needed. To deal with these problems, we intend to adopt data ownership security architecture (DOSA) to securely circulate laboratory management data and cultivate an ecological and sustainable forest from the \u201cflowerpot\u201d of each independent system.[1]\nLaboratories are currently using LIMS frequently.[2] A university may also have multiple management systems, including an equipment management system, a chemical management system, a safety examination system, etc. Related systems comprise relevant departments of the laboratory, such as the experiment management department of the Education Bureau, the accident handling department of the Emergency Bureau, and the hazardous chemicals supervision department of Public Security. Such complicated systems make it difficult to obtain information accurately and quickly when the same report requires the cooperation of different departments or laboratories. Compared to traditional manual records, the LIMS used by a specific business unit or company can be easily outdated and has high maintenance costs. Once the database needs major updates related to its data source, the existing system cannot meet the demand, and a new system must be designed so that all data can be transferred, resulting in increased costs. The linkage between different systems is weak, the efficiency is relatively low, and the security is not uniform and difficult to guarantee when it is necessary to submit data.\nBlockchain poses one potential solution. The medical industry uses the blockchain system to manage medical imaging data, and the banking system uses blockchain as a new type of financial technology.[3] Blockchain has high energy consumption, expensive development costs, but relatively high security. At present, some scholars have produced a framework diagram of the blockchain management in university education, but there is no corresponding technical means.[4]\nConsidering the problems mentioned above, the lab industry would require a technology that allows data to flow securely and efficiently. This article\u2019s contributes discussion about potential technology to solve such problems. In our work, experiment-related data management is based on the data ownership security architecture. This security technology is built such that data ownership binds with a person and it can be safely registered on this platform; meanwhile, ownership of data can be determined, and data users and owners can conditionally share or trade using ownership conversion across domains and across borders efficiently and securely. Our proposed laboratory data management architecture is shown in Figure 1.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 1. Proposed data ownership security architecture (DOSA) to manage the laboratory.\n\n\n\nThe current study has been organized as follows. The next section explains related theories and key management. After that, we present the core strategies used, including laboratory data encrypted registration and an authorized transfer method for conditional sharing. We then detail the whole lab management framework and the incident searching case study based on DOSA using the above ownership conversion method. We then discuss and verify the effectiveness of the data flow process using key management based on proposed core strategies. Finally, we provide our conclusions.\n\nRelated work \nData ownership security architecture \nAs data has a tendency to not be securely circulated and shared, Miao et al. proposed a data ownership security architecture (DOSA) that contains one body with two wings.[5] The body represents data binding ownership, while one wing is the key and the other wing signifies value.\nTo solve the problem of data transfer barriers, DOSA proposes a global, cross-border, comprehensive, collaborative, secure, and efficient data architecture system. The whole field is no longer the cross-domain data of the head industry or a single system as previous, but all industry-related data are registered. Integration and collaboration mean that data are retrieved and used by different industry sources. This architecture has efficient security technologies for data security from source, registration, circulation, and use to supervision.\nDOSA is composed of a data registration center (DRC), a data authorization center (DAC), a data exception center (DEC), data application units (DAUs), a key system, and other components. Safety and efficiency are the core of DOSA. Similar DOSA-like framework applications have been implemented in tourism, lab education, and data product trading processes.[6][7]\n\nDRC \nThe DRC stores the data in an encrypted form, which is invisible even to the administrator. It not only confirms the ownership of the data but also keeps it safe and confidential. After the data are registered, the directory is automatically generated, which is convenient for data users to view and find relevant data sources. It can register the data of relevant units or laboratories to DRC or link to DRC through an application programming interface (API).[8]\n\nDAC \nTo ensure the relationship between people and data ownership, DAC is responsible for the process of data ownership confirmation and authorization.[9] By guaranteeing the interests of data owners, they are confident to put their data on the platform. The owner of the data divides data-related personnel into data owners, data producers, and data users. In the current study, a design was proposed to save all the related public keys to the DAC, and the private key is kept by their client, without placing it on any platform or cloud, to ensure security.\n\nDEC \nTo protect the interests of data owners, DEC needs to supervise during the data flow process. From the point of data register and data conditional sharing, even after the trading process, we took blockchain-related technical means, such as watermarks and timestamps, and added them to the data authorization process to ensure data users cannot transmit data again without the owner\u2019s permission.[10]\n\nLaboratory management technology \nExisting laboratory management usually uses a LIMS that collects related data through various laboratories. Some benefits of using a LIMS are the ability to track individual data items or samples from reported results.[11] Some limitations of a LIMS for this specific application exist, as they may not be specialized for specific equipment, or applicable to certain chemistries.[12] Some LIMS chemistry systems are only available to connect with the Public Safety department. It is rare to find a LIMS system for chemistry widely used in related departments. There is also a high price for many commercial LIMS systems. Various systems\u2019 costs are expensive for labs, and customization is limited to what the system offers.[13] Additionally, experimental data are private to external parties, yet it still needs to be shared among the research team.\nRecently, researchers have designed some LIMS for education labs or other specific applications. Focusing on education, there is Chaoqun and Lanlan's interactive LIMS, which incoporates experimental teaching management, experimental equipment management, laboratory open management, communication, and interactive management modules.[14] For specific disciplines, there are LIMS and other laboratory informatics solutions specifically designed for high-level biosafety labs or biobanks.[15][16] However, these systems only flow data inside the lab, in which data cannot flow quickly to supervision departments or related companies. Some researchers realized that the LIMS needs to be both secure and enhance efficiency in order to enable laboratory staff to conduct their research as freely as possible; therefore, the information on research, laboratory personnel, experimental materials, and experimental equipment is generally collected and utilized by only one system.[17] However, the current research mainly focuses on such a system's design, and there is no specific experimental technical means to verify it. Thus, we employed DOSA to rapidly and securely move data inside or outside the laboratory. In Table 1, we compare our proposed architecture to existing lab management systems.\n\n\n\n\n\n\n\nTable 1. Comparison of the proposed architecture and existing lab management.\n\n\nTopic\n\nProposed DOSA lab management system\n\nExisting lab management systems\n\n\nData ownership\n\nClear\n\nUndefined\n\n\nData security\n\nThe public key in DAC\n\nKey pair in systems\n\n\nData source\n\nGlobal\n\nSpecific service[18]\n\n\nData register\n\nEncrypted, fully automatic\n\nAdmin visible, semi-automatic entry\n\n\nData search\n\nAuthorization visible\n\nCollect systems to summarize\n\n\nApplication\n\nFor a variety of business\n\nFor a single item[19]\n\n\n\nHowever, DOSA-related studies have designed models for some domains such as tourism, which only have frameworks without practical implementation.[9] We propose to use technical methods to solve data ownership by binding persons that used a key management system to protect data privacy.\n\nKey system methods \nTo ensure data registration and sharing security, this work adopted RSA and AES algorithms, as well as Chinese domestic commercial keys SM2 and SM4, to compare the aspects of addressing efficiency and security of data. Table 2 summarizes the pros and cons of these key algorithms.\n\r\n\n\n\n\n\n\n\n\nTable 2. Comparison of the four key algorithms examined in this research.\n\n\nAlgorithm\n\nDescription\n\nAdvantage\n\nDisadvantage\n\n\nRSA\n\nAsymmetric encryption that encrypts with public key; decrypts with private key[20]\n\nIn the process of encryption and decryption, there is no need to transmit confidential keys through the network. The key management is better than the AES algorithm.\n\nThe speed of encryption and decryption is relatively slow; usually it is not suitable for the encryption of a large number of data files.\n\n\nAES\n\nSymmetric encryption algorithm; encryption and decryption processes use the same set of keys\n\nThe operation does not require a computer with very high processing power and significant memory. The operation resists attacks easily. It always maintains good performance in different operating environments; the encryption speed is relatively fast.\n\nIt is required to secretly distribute the key before communication. The decrypted private key must be transmitted to the receiver of the encrypted data through the network.\n\n\nSM2\n\nAsymmetric encryption algorithm which is an elliptic curve public key cryptography algorithm based on elliptic-curve cryptography (ECC)[21]\n\nCompared with RSA, the performance of SM2 is better and more secure. The password complexity is high, the processing speed is fast, and the computer performance consumption is comparatively small.\n\nEncryption and decryption take a relatively long time and are suitable for encrypting small amounts of data.\n\n\nSM4\n\nSymmetric encryption algorithm; the key length and block length are both 128 bits\n\nSM4 is efficient and secure, while remaining easy to implement across software and hardware. It usually has a fast computing speed.\n\nThe management and distribution of keys are relatively difficult and not secure enough.\n\n\n\nSM2 and SM4 are cryptographic standards authorized to be used in China. Relevant studies have shown that the SM2 and SM4 algorithms are more secure than ECDSA and AES.[22] During a controlled experiment, AES outperformed SM4 by a significant margin.[23] Symmetric encryption is usually used when the message sender needs to encrypt a large amount of data. It has the characteristics of an open algorithm, a small amount of calculation, and fast encryption speed. The advantage of the symmetric encryption algorithm lies in the high speed of encryption and decryption, and the difficulty of cracking when using a long key. The disadvantages of symmetric encryption are that key management and distribution are difficult and insecure. Before the data are transmitted, the sender and the receiver must agree on a secret key, and both parties must keep the key. If the key of one party is leaked, the encrypted information is insecure, and its security cannot be guaranteed.[24]\nThe advantage of asymmetric encryption is its higher security. The public key is made available, and the private key is kept by itself so there is no need to give the private key to others. The disadvantage of asymmetric encryption is that its speed is relatively slow, so it is only suitable for encrypting a small amount of data.\n\nMethodology \nLaboratory-related data and persons \nManaged laboratory information and data can be divided into unconditional and conditional sharing categories. For example, the opening hours of the laboratory represent unconditional sharing data. The transmission of the laboratory's inventory and usage of hazardous chemicals needs conditional sharing by the administrative department (data owners). Laboratory safety data, accident statistics, and related training data are transnational conditional sharing data. Lab-related data are also classified into a security level ranging from zero to five.[25] Class zero data is data that can be public, such as laboratory introductions and research group members\u2019 profiles which do not need any security protection. Class five data is critical national scientific research experiment data which must be kept strictly confidential. Classes one through four refer to data that the laboratory needs to share or that are trade-protected.\nLab data producers can be one or several persons, such as laboratory administrators, experimental instructors, or relevant teachers. A data owner is usually a person or a unit, such as a person in charge of the laboratory team or the emergency department. Data producers can input data, but they do not have the authority to share or trade data. After the data user has found targeted data by querying the DRC category, they need to contact the data owner. If the data owner considers that the data user can apply to view or use these data sources, the owner finds the user\u2019s public key from the DAC then encrypts these data with the user\u2019s public key to complete the authorization. The data user opens encrypted lab data with their private key to finish the conditional sharing of data (Figure 2a,b).\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 2. Illustrates the relationship of data-related persons or branches (a) and the relationship between data user, producer, and owner (b).\n\n\n\nData encryption registration and application process \nThe laboratory data are uploaded to the DRC by the owner, and the owner\u2019s public key is added to verify the ownership of the data. It can guarantee the privacy and security of the data at the source. The data are stored in the DRC, which the administrator cannot view without permission. The encrypted lab data are transmitted to the DRC, and the directory is automatically generated by keywords. The relevant users can apply for permission to obtain the data after searching in the data directory. Figure 3 shows the data owner encrypting the registration data, which automatically generates an index directory, then data users apply for the required data by viewing the catalog.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 3. Data encryption registration process.\n\n\n\nConfirmation and authorization of the data transmission process \nMutual trust - Single-layer encryption \nAfter the data user searches the DRC catalog and queries the data they need, the data user communicates with the data owner. If the user obtains permission, the data owner encrypts the data with the user\u2019s public key. Then, the data user can use their private key to decrypt data. Meanwhile, to ensure the interests of the data owner, the entire process has traceability technology in the DOSA lab management system. For example, time stamps and digital signatures are adopted, which can ensure that data users do not transmit data outside the authorized context.[26] If the mutual parties trust each other, the process of conditional sharing occurs, as shown in Figure 4. \n\r\n\n\n\n\n\n\n\n\n\n\nFigure 4. Single-layer encryption.\n\n\n\nAlgorithm 1 shows how the lab data user retrieves data in the DRC during the process of data ownership conversion. Table 3 shows the definition of symbols used in the algorithm.\n\n\n\n\n\n\n\nAlgorithm 1. Encryption_Once ( )\n\n\n\nInput: Lab data owner On with their lab data record Rn. Lab data user with their private key Uprkn.\r\n\nOutput: Boolean(True or False)\r\n\n1. #Function used to encrypt the lab data record.<br \/>\n2.\u2003<b>For<\/b> owner O query the user\u2019s public key <i>Upubk<sub>n<\/sub><\/i> from the DAC <i>Dacv<sub>n<\/sub><\/i><br \/>\n3.\u2003#Check the DAC\u2019s data<br \/>\n4.\u2003<b>if<\/b>(role == \u201cOwner\u201d) then<br \/>\n5.\u2003\u2003\u2003Encryption with the user\u2019s public key <i>Upubk<sub>n<\/sub><\/i><br \/>\n6.\u2003\u2003\u2003lab data record <i>R<sub>n<\/sub><\/i>\u2192encrypted data<br \/>\n7.\u2003\u2003\u2003encrypted data\u2192<i>Drc<sub>n<\/sub><\/i><br \/>\n8.\u2003\u2003\u2003<b>return<\/b> True<br \/>\n9. \u2003<b>else<\/b><br \/>\n10.\u2003\u2003\u2003<b>return<\/b> False<br \/>\n11.\u2003<b>end if<\/b><br \/>\n12. \u2003<b>end for<\/b><br \/>\n13.\u2003<b>For<\/b> user U query the data register center catalogue view <i>Drcv<sub>n<\/sub><\/i><br \/>\n14. \u2003<b>if<\/b>(role == \u201cUser\u201d) then<br \/>\n15.\u2003\u2003\u2003Decryption using the user\u2019s private key <i>Uprk<sub>n<\/sub><\/i><br \/>\n16.\u2003\u2003<i>Drc<sub>n<\/sub><\/i>\u2192encrypted data<br \/>\n17.\u2003\u2003\u2003encrypted data\u2192lab data record <i>R<sub>n<\/sub><\/i><br \/>\n18.\u2003\u2003\u2003<b>return<\/b> True<br \/>\n19.\u2003<b>else<\/b><br \/>\n20.\u2003\u2003<b>return<\/b> False<br \/>\n21.\u2003<b>end if<\/b><br \/>\n22.\u2003<b>end for<\/b><br \/>\n23.\u2003<b>end<\/b> function\n\n\n\n\n\n\n\n\n\n\nTable 3. The definition of symbols used in the algorith.\n\n\nSymbols\n\nDefinition\n\n\nUn\n\nnth Lab Data User\n\n\nOn\n\nnth Lab Data Owner\n\n\nRn\n\nnth Lab Data Record\n\n\nUpubkn\n\nnth User Public Key\n\n\nUprkn\n\nnth User Private Key\n\n\nOpubn\n\nnth Owner Public Key\n\n\nOprkn\n\nnth Owner Private Key\n\n\nDrcn\n\nnth Data Register Center\n\n\nDrcvn\n\nnth Data Register Center Catalogue View\n\n\nDacvn\n\nnth Data Authority Center View\n\n\n\n Data users do not trust the data source \u2014 Double encryption \nAfter checking the catalog in the DRC and communicating with the data owner about the data product\u2019s price, the data user may doubt the authenticity of the data source and ownership. In this situation, the data owner will add this data product to the owner\u2019s private key, then encrypt it by using the user\u2019s public key, which is found in the DAC. As the next step, data owners input this encrypted data to the DRC and then inform the users.\nFirstly, the data user finds the data owner\u2019s public key from the DAC, and they use it to open the double-encrypted data product. As the next step, the data user utilizes their private key to decrypt the data. Figure 5 shows the transaction process of experimental data products when the data user questions the data authenticity or the data source\u2019s ownership.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 5. Double encryption.\n\n\n\nUsing a person-binding data owner method to finish conditional data sharing can break data barriers. We also resolve issues of the laboratory data class and ownership conversion process.\n\nDOSA framework components - Managed laboratory data linked to rich soils \nOverall framework \nThis section briefly describes the entire laboratory data transfer framework using DOSA, designed by the core techniques. Figure 6 shows the overall structure of DOSA to enable laboratory-related data to break barriers. The existing LIMS is independent. We use the proposed data security protection technology such that data can be stored securely and conditionally. Every laboratory uses the public key algorithm to store encrypted data which can determine ownership. If any supervision departments (such as the Education Bureau) need to collect a certain type of data, they just need to apply for related laboratories. The data owners decide which data can be shared, then find the Education Bureau\u2019s public key in DAC that can encrypt related data to the DRC of the Education Bureau so data users can obtain real-time, non-tampered data from various university laboratories. Compared with the current information collected, the DOSA method using the ownership conversion technique can solve existing problems with isolated laboratory data islands.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 6. The whole structure of laboratory management based on DOSA.\n\n\n\nLaboratory accident searching based on DOSA \nWe selected the registration and inquiry of laboratory hazardous chemical materials as a case study. The full cycle of hazardous chemical materials includes application, acquirement, procurement, transportation, usage, recycling waste liquid, accidental leakage management, experimental operation accident, etc. The whole process needs to be supervised by multiple relevant units, data need to be circulated, and sharing is needed to break the information cocoon. There are many departments involved, and as such, routine security inspections need to be conducted and results should be submitted according to the protocols, which is not only inefficient but also error-prone and tamper-prone. In case there is an emergency leak or experiment-related explosion, an integrated and efficient system is necessary to query related data urgently. This workflow of searching for or collecting hazardous chemicals data among related branches, such as laboratories, the Education Bureau, the Public Security Bureau Inspection Department, and chemical companies, is shown in Figure 7.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 7. The workflow of querying hazardous chemicals.\n\n\n\nThe hazardous chemicals can be queried through the API specification interface (query by CAS or chemical name) directly among different branches. Hazardous chemical users, administrators, public security departments, emergency departments, and hazardous chemical suppliers can query in real time.\nIf an accident happens in the laboratory, we could obtain cross-domain and cross-branch information by each unit\u2019s authorization in real time. The proposed structure, which can search the corresponding hazardous chemical information, equipment details, relevant personnel profile, and experimental project proposal, etc., is shown in Figure 8.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 8. Structure of laboratory accident querying in DRC.\n\n\n\nExperiments \nIn this section, we discuss testing the efficiency of managed lab data based on DOSA by putting the proposed framework into practice. We investigated the time consumption of key generation and public key transmission to DRC by different algorithms using Python 3.10.9. Then, we tried to use four key algorithms\u2014SM2, SM4, RSA, and AES\u2014to compare which algorithm was faster in the proposed architecture regarding authorized encryption and decryption. Combining the security level of the lab data mentioned in the above chapters, we try to find the most suitable solution.\n\nExperimental environment \nTable 4 shows the details of the experiment environment. The algorithm suits the corresponding experimental data class, which can consider both efficiency and security.\n\n\n\n\n\n\n\nTable 4. Details of the experiment environment.\n\n\n\nEnvironment 1 HOST 1\r\n\nProcessor: Intel(R) Core (TM) i5-8300H CPU @ 2.30 GHz\r\n\nMemory: 8.0 GB\r\n\nMain hard disk: NVMe WDC PC SN520 SDA\r\n\nOperating system: Windows 11 \u00d764 Professional Edition Insider Preview\r\n\nProgramming software: Microsoft Visual Studio Code\r\n\nTest work data: lab data xlsx\n\n\n\n\nEnvironment 2 HOST 2\r\n\nProcessor: AMD Ryzen 5 3550H with Radeon Vega Mobile Gfx 2.10 GHz\r\n\nMemory: 8.0 GB\r\n\nMain hard disk: SAMSUNG MZVLB512HAJQ-00000\r\n\nOperating system: Windows 10 Home\r\n\nProgramming software: Pycharm Community Edition\r\n\nTest work data: lab data xlsx\n\n\n\n\nSimulation key generation and transmission experiment \nThe DAC module includes the right confirmation and authorization. The right confirmation determines the lab data ownership. The ownership of the experimental data belongs to the experimental team designer. After storing data in the DRC, the lab data product adds the research group public key to determine the ownership.\nDuring lab data transactions, the data user wants to purchase the experimental data product. After payment, the data user needs to find the data owner\u2019s public key in the DAC, then the user uses the owner\u2019s public key to open the lab data product. This is an authorization ownership process.\nIn this experiment, a key pair was generated in Chengdu, Sichuan Province, China. The public key was sent to Beijing, China (DAC). We carried out 12 experiments generating key pairs, and the tests were performed on Environment 1 (env1) and Environment 2 (env2), respectively. Figure 9 shows a speed test of the time required to generate a key pair. The results of this experiment show that using the AES algorithm is the fastest (1.232 \u00d7 10\u22125 s\/1.605 \u00d7 10\u22125 s) in two environments. Using SM4 (3.197 \u00d7 10\u22125 s\/6.882 \u00d7 10\u22125 s) to create a key pair is the second fastest, which is close to AES. Meanwhile, the SM2 algorithm is the most time-consuming way.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 9. Time consumption for generating key pairs using SM2, SM4, AES, and RSA algorithms in Environment 1 (env1) (a) and Environment 2 (env2) (b).\n\n\n\nFurther, under two different operating system environments, the experiment simulated the speed of public key transmission from Chengdu to Beijing, and the results are shown in Figure 10. Compared with the key generation experiment, the speed of AES transmission is still the fastest, and the average speed is 1.327 s\/1.400 s. In contrast, SM2 is the slowest, with an average speed of 1.485 s\/1.571 s.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 10. Time consumption for simulating transmission public keys to DRC using SM2, SM4, AES, and RSA in Environment 1 (env1) (a) and Environment 2 (env2) (b).\n\n\n\nConfirmation and authorization experiments \nIn the proposed system, we tried to test lab management digital records 50\u2013300 KB, which were encrypted, respectively, using the SM2, SM4, RSA, and AES algorithms, then stored in the DRC of HOST 1 and 2. The encryption time of these data texts is shown in Figure 11.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 11. Time consumption of authorization encryption of different experimental samples using SM2, SM4, AES, and RSA in Environment 1 (env1) (a) and Environment 2 (env2) (b).\n\n\n\nAfter the lab data user obtained the data owner\u2019s public key from DAC, the lab data products were decrypted with four different key management methods, and the time taken is shown in Figure 12. \n\r\n\n\n\n\n\n\n\n\n\n\nFigure 12. Time consumption of authorized decryption of different experimental samples using SM2, SM4, AES, and RSA in Environment 1 (env1) (a) and Environment 2 (env2) (b).\n\n\n\nThis experiment was performed 16 times, with different sizes of experimental products as variables, under HOST 1 and 2. The following is the logic Algorithm 2 for this experiment:\n\n\n\n\n\n\n\nAlgorithm 2. Calculate encryption and decryption time ( )\n\n\n\nData: Example excel file\r\n\nResult: Time consumption for encrypt and decrypt\r\n\n1. Data format conversion:<br \/>\n2.\u2003\u2003\u2003excel file -> json -> string -> bytes<br \/>\n3. \u2003<b>if<\/b> data.type == bytes <b>then<\/b><br \/>\n4.\u2003\u2003\u2003time.record<br \/>\n5.\u2003\u2003\u2003algorithm.encrypt(data)<br \/>\n6.\u2003\u2003\u2003time.record<br \/>\n7.\u2003\u2003\u2003algorithm.decrypt(data)<br \/>\n8.\u2003\u2003\u2003time. record<br \/>\n9.\u2003<b>end if<\/b><br \/>\n10.\u2003calc(time.consumption)\n\n\n\n\nAs shown in Figure 11 and Figure 12, AES was the fastest in performance to encrypt and decrypt lab data in the DOSA system. In particular, there are obvious advantages in decrypting data using AES. In the prior Table 2, we compared the security performance of these four algorithms. AES is a symmetric encryption algorithm, which has the conventional requirement to secretly distribute the key before communication, and the private key must be transmitted to the receiver through the network; thus, the key is not easy to keep secret and manage. However, as we proposed an ownership framework, the private key does not need to be transmitted to the data owner, so this shortcoming can be ignored. In summary, combined with security and speed performance, for lab data of class 1 to 4, it is recommended to use the AES algorithm to encrypt registration to determine ownership using the public key, and to decrypt to authorize with the private key based on DOSA.\nHowever, also need to take high-security measures (class 5 lab data) into account, such as special major laboratory explosion data, or state secret experiment plans. For these security class 5 lab data, we need to pay more attention to safety performance. Combined with Table 2, SM2 shows better safety performance. So, in the current experiment, class 5 was applied to samples to compare the authorization and confirmation time efficiency using SM2.\nHost 1\u2019s speed of encryption storage of a larger-volume data text (over 230 KB) is double that of Host 2. Additionally, the encryption storage speed of Host 1 for small and medium volume experimental data was three to four times faster than Host 2. However, it seems there is no difference in the data decryption time consumption between the two environments.\nAs shown in Figure 13, it is recommended to choose a larger experimental sample for class 5 data encryption and decryption, which could save time.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 13. Time consumption of encryption (a) and decryption (b) of different experimental samples using SM2.\n\n\n\nEvaluate results \nCombining the above experiments based on the proposed system, it is recommended to use the AES algorithm to encrypt the registration, authorization, and decryption of ordinary security-level data (class 1\u20134). For a high level of security experimental data (class 5), we recommend using the SM2 algorithm to encrypt larger experimental documents at one time, which can improve the efficiency.\nThe advantage of DOSA is that it binds data and ownership together, protects data with key technologies, and enables secure sharing. This study designed the whole conditional sharing and transaction process of laboratory-related data based on DOSA. In this process, class 1\u20135 data are protected by encryption technology to ensure their security. Through the key algorithm and the Gmssl (open-source toolbox), the problem of data leakage can be solved in the data transaction process. Using Python to encrypt the laboratory data with the tools provided by DOSA, it can be considered safe to share relevant laboratory data cross-domain and internally and externally.\n\nConclusions \nTo solve the problem of isolated islands of laboratory data, we proposed a data-ownership-based security architecture for confirming lab data rights, and we used a secret key algorithm to implement internal and external laboratory data conditional sharing. A data owner inputs encrypted data into the DRC, while confirming sharing rights. When a data user needs statistics or supervision, they can apply for targeted data by searching the directory. According to the different security levels of lab data, the appropriate key algorithm is selected for ownership authorization, which can realize efficient conditional sharing.\nLaboratory information and data necessarily need to be conditionally shared for global and cross-border purposes. Often, these data are dispersed across various information systems, posing a challenge in sharing data for a safe and efficient lab data management system. Meanwhile, it is not easy to summarize different sources of real-time data, and it is also difficult to search data urgently among the relevant departments of the independent systems. Moreover, the security performance of the different systems is also inconsistent. As such, we developed our data architecture to manage laboratory-related data by allowing relevant units, universities, and laboratories to encrypt data with public keys and upload them to the data register center to form a data directory. Data users can search \u201cDirectory\u201d or \u201cdata owner\u201d names to apply to the data owner for data viewing or use. Data can be shared conditionally through public key confirmation and private key authorization to break lab data barriers. Then, we used experiments to evaluate which algorithm is better for the efficiency of this system. We suggest AES for ordinary experimental data. For higher levels of experimental data, we can use SM2 to process larger data at one time, which addresses both efficiency and security of data. From these experiments, we verified the feasibility and efficient security of DOSA to manage lab-related data.\nThis ownership security architecture for laboratory-managed information and data can also be applied to other fields, such as smart city construction. In the future, we will focus on the prediction of laboratory accidents on this system. Our long-term work goal is to use this data architecture to process the artificial intelligence analysis of experimental management data. Through the entry of a large amount of data, under the premise of the authorization of the data user, we can use the registered massive data to make predictions, such as analyzing the probability of an experimental accident occurring.\n\n Abbreviations, acronyms, and initialisms \nAPI: application programming interface\nDAC: data authorization center\nDAU: data application unit\nDEC: data exception center\nDRC: data registration center\nDOSA: data ownership security architecture\nECC: elliptic-curve cryptography\nenv1: Environment 1\nenv2: Environment 2\nLIMS: laboratory information management system\nAcknowledgements \nAuthor contributions \nConceptualization, X.Z. and F.M.; methodology, X.Z. and N.C.; software, X.Z.; validation, P.U., X.Z. and F.M.; formal analysis, X.Z. and P.U.; data curation, X.Z. and P.U.; writing\u2014original draft preparation, X.Z.; writing\u2014review and editing, X.Z., P.U. and F.M; visualization, X.Z.; supervision, F.M., N.C. and P.U.; project administration, X.Z.; funding acquisition, X.Z. All authors have read and agreed to the published version of the manuscript.\n\nFunding \nThis research received no external funding.\n\nConflicts of interest \nThe authors declare no conflict of interest.\n\nReferences \n\n\n\u2191 Miao, Fang; Fan, Wenjie; Yang, Wenhui; Xie, Yan (19 January 2019). \"The study of data-oriented and ownership-based security architecture in open internet environment\" (in en). Proceedings of the 3rd International Conference on Cryptography, Security and Privacy (Kuala Lumpur Malaysia: ACM): 121\u2013129. doi:10.1145\/3309074.3309093. ISBN 978-1-4503-6618-2. https:\/\/dl.acm.org\/doi\/10.1145\/3309074.3309093 .   \n \n\n\u2191 Antes, Alison L.; Kuykendall, Ashley; DuBois, James M. (24 April 2019). Master, Zubin. ed. \"The lab management practices of \u201cResearch Exemplars\u201d that foster research rigor and regulatory compliance: A qualitative study of successful principal investigators\" (in en). PLOS ONE 14 (4): e0214595. doi:10.1371\/journal.pone.0214595. ISSN 1932-6203. PMC PMC6481787. PMID 31017929. https:\/\/dx.plos.org\/10.1371\/journal.pone.0214595 .   \n \n\n\u2191 Idrees, Sheikh Mohammad; Nowostawski, Mariusz; Jameel, Roshan; Mourya, Ashish Kumar (16 April 2021). \"Security Aspects of Blockchain Technology Intended for Industrial Applications\" (in en). Electronics 10 (8): 951. doi:10.3390\/electronics10080951. ISSN 2079-9292. https:\/\/www.mdpi.com\/2079-9292\/10\/8\/951 .   \n \n\n\u2191 Meyliana; Cassandra, Cadelina; Surjandy; Widjaja, Henry Antonius Eka; Prabowo, Harjanto; Fernando, Erick; Chandra, Yakob Utama (1 August 2020). \"A Blockchain Technology-Based for University Teaching and Learning Processes\". 2020 International Conference on Information Management and Technology (ICIMTech) (Bandung, Indonesia: IEEE): 244\u2013247. doi:10.1109\/ICIMTech50083.2020.9211209. ISBN 978-1-7281-7071-8. https:\/\/ieeexplore.ieee.org\/document\/9211209\/ .   \n \n\n\u2191 Miao, Fang; Fan, Wenjie; Yang, Wenhui; Xie, Yan (19 January 2019). \"The study of data-oriented and ownership-based security architecture in open internet environment\" (in en). Proceedings of the 3rd International Conference on Cryptography, Security and Privacy (Kuala Lumpur Malaysia: ACM): 121\u2013129. doi:10.1145\/3309074.3309093. ISBN 978-1-4503-6618-2. https:\/\/dl.acm.org\/doi\/10.1145\/3309074.3309093 .   \n \n\n\u2191 Miao, F.; Yang, W.; Ye, A. et al. (30 December 2015). \"A Data-Oriented Security Architecture Method and System - CN Patent 201511025657\".   \n \n\n\u2191 Zheng, Xuying; Miao, Fang; Chakpitak, Nopasit; Huang, Jin; Wang, Jing (17 December 2021). \"Discussion of University Chemistry Laboratory Management Using DOSA Platform and Safety Education Based on Blockchain\" (in en). 2021 4th International Conference on Education Technology Management (Tokyo Japan: ACM): 79\u201385. doi:10.1145\/3510309.3510322. ISBN 978-1-4503-8580-0. https:\/\/dl.acm.org\/doi\/10.1145\/3510309.3510322 .   \n \n\n\u2191 Panpeng, Vorada; Fang, Miao; Phaphuangwittayakul, Aniwat; Rattanadamrongaksorn, Tanarat (2021), Yang, Xin-She; Sherratt, Simon; Dey, Nilanjan et al.., eds., \"Preliminary Study and Implementation of Chiang Mai Tourism Platform Based on DOSA\" (in en), Proceedings of Fifth International Congress on Information and Communication Technology (Singapore: Springer Singapore) 1184: 511\u2013521, doi:10.1007\/978-981-15-5859-7_51, ISBN 978-981-15-5858-0, http:\/\/link.springer.com\/10.1007\/978-981-15-5859-7_51 . Retrieved 2023-07-31   \n \n\n\u2191 9.0 9.1 Miao, F.; Yang, W.; Li, Y. et al. (21 January 2016). \"WO2016008335 - DATA-ORIENTED INFORMATION TECHNOLOGY SYSTEM\". WIPO. https:\/\/patentscope.wipo.int\/search\/en\/detail.jsf?docId=WO2016008335 .   \n \n\n\u2191 Miao, F. (2016). \"Preliminary Study on Data-oriented Security Architecture\". ZTE Technology Journal 1: 19\u201322.   \n \n\n\u2191 Conti, Thomas J. (1 December 1992). \"LIMS and quality audits of a quality control laboratory\" (in en). Chemometrics and Intelligent Laboratory Systems 17 (3): 295\u2013300. doi:10.1016\/0169-7439(92)80066-D. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/016974399280066D .   \n \n\n\u2191 Xu, Yunbi; Crouch, Jonathan H. (1 March 2008). \"Marker\u2010Assisted Selection in Plant Breeding: From Publications to Practice\" (in en). Crop Science 48 (2): 391\u2013407. doi:10.2135\/cropsci2007.04.0191. ISSN 0011-183X. https:\/\/onlinelibrary.wiley.com\/doi\/10.2135\/cropsci2007.04.0191 .   \n \n\n\u2191 Jayashree, B; Reddy, Praveen T; Leeladevi, Y; Crouch, Jonathan H; Mahalakshmi, V; Buhariwalla, Hutokshi K; Eshwar, Ke; Mace, Emma et al. (1 December 2006). \"Laboratory Information Management Software for genotyping workflows: applications in high throughput crop genotyping\" (in en). BMC Bioinformatics 7 (1): 383. doi:10.1186\/1471-2105-7-383. ISSN 1471-2105. PMC PMC1559653. PMID 16914063. https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/1471-2105-7-383 .   \n \n\n\u2191 Chaoqun, Liu; Lanlan, Cai (27 March 2021). \"Interactive Laboratory Information Management System Based on WeChat\". 2021 7th International Conference on Information Management (ICIM) (London, United Kingdom: IEEE): 66\u201370. doi:10.1109\/ICIM52229.2021.9417148. ISBN 978-1-6654-4380-7. https:\/\/ieeexplore.ieee.org\/document\/9417148\/ .   \n \n\n\u2191 Sun, Dingzhong; Wu, Linhuan; Fan, Guomei (1 June 2021). \"Laboratory information management system for biosafety laboratory: Safety and efficiency\" (in en). Journal of Biosafety and Biosecurity 3 (1): 28\u201334. doi:10.1016\/j.jobb.2021.03.001. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2588933821000042 .   \n \n\n\u2191 D\u2019Amico, Gaspare; Szopik-Depczy\u0144ska, Katarzyna; Beltramo, Riccardo; D\u2019Adamo, Idiano; Ioppolo, Giuseppe (2 January 2022). \"Smart and Sustainable Bioeconomy Platform: A New Approach towards Sustainability\" (in en). Sustainability 14 (1): 466. doi:10.3390\/su14010466. ISSN 2071-1050. https:\/\/www.mdpi.com\/2071-1050\/14\/1\/466 .   \n \n\n\u2191 Kou, Yunpeng; Liu, Yuhang; Li, Guoqing; Hou, Jie; Luan, Liming; Wang, Hao (22 March 2022). Guo, Lijie. ed. \"Design and Implementation of an Integrated Management System for Backfill Experimental Data\" (in en). Advances in Civil Engineering 2022: 1\u20139. doi:10.1155\/2022\/1876435. ISSN 1687-8094. https:\/\/www.hindawi.com\/journals\/ace\/2022\/1876435\/ .   \n \n\n\u2191 Magzumova, Bibigul (1 November 2016). \"Efficient Data Integrity Through LIMS-Instrument Interface\". All Days (Astana, Kazakhstan: SPE): SPE\u2013182564\u2013MS. doi:10.2118\/182564-MS. https:\/\/onepetro.org\/SPECTCE\/proceedings\/16CTCE\/All-16CTCE\/Astana,%20Kazakhstan\/185619 .   \n \n\n\u2191 McIntosh, Leslie D.; Sharma, Mukesh K.; Mulvihill, David; Gupta, Snehil; Juehne, Anthony; George, Bijoy; Khot, Suhas B.; Kaushal, Atul et al. (1 October 2015). \"caTissue Suite to OpenSpecimen: Developing an extensible, open source, web-based biobanking management system\" (in en). Journal of Biomedical Informatics 57: 456\u2013464. doi:10.1016\/j.jbi.2015.08.020. PMC PMC4772150. PMID 26325296. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1532046415001884 .   \n \n\n\u2191 Zhou, Xin; Tang, Xiaofei (1 August 2011). \"Research and implementation of RSA algorithm for encryption and decryption\". Proceedings of 2011 6th International Forum on Strategic Technology 2: 1118\u20131121. doi:10.1109\/IFOST.2011.6021216. https:\/\/ieeexplore.ieee.org\/document\/6021216\/ .   \n \n\n\u2191 Zheng, Xin; Xu, Chongyao; Hu, Xianghong; Zhang, Yun; Xiong, Xiaoming (1 October 2020). \"The Software\/Hardware Co-Design and Implementation of SM2\/3\/4 Encryption\/Decryption and Digital Signature System\". IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 39 (10): 2055\u20132066. doi:10.1109\/TCAD.2019.2939330. ISSN 0278-0070. https:\/\/ieeexplore.ieee.org\/document\/8823969\/ .   \n \n\n\u2191 \"GB\/T 32918.3-2016 - Information security technology -- Public key cryptographic algorithm SM2 based on elliptic curves -- Part 3: Key exchange protocol\". ChineseStandard.net. 2016. https:\/\/www.chinesestandard.net\/PDF.aspx\/GBT32918.3-2016 .   \n \n\n\u2191 Saidi, Hafida; Labraoui, Nabila; Ari, Ado Adamou Abba; Maglaras, Leandros A.; Emati, Joel Herve Mboussam (2022). \"DSMAC: Privacy-Aware Decentralized Self-Management of Data Access Control Based on Blockchain for Health Data\". IEEE Access 10: 101011\u2013101028. doi:10.1109\/ACCESS.2022.3207803. ISSN 2169-3536. https:\/\/ieeexplore.ieee.org\/document\/9895264\/ .   \n \n\n\u2191 Aranha, Diego; Dahab, Ricardo; L\u00f3pez, Julio; Oliveira, Leonardo (1 May 2010). \"Efficient implementation of elliptic curve cryptography in wireless sensors\" (in en). Advances in Mathematics of Communications 4 (2): 169\u2013187. doi:10.3934\/amc.2010.4.169. ISSN 1930-5346. http:\/\/www.aimsciences.org\/journals\/displayArticles.jsp?paperID=5168 .   \n \n\n\u2191 Jintcharadze, Elza; Iavich, Maksim (1 September 2020). \"Hybrid Implementation of Twofish, AES, ElGamal and RSA Cryptosystems\". 2020 IEEE East-West Design & Test Symposium (EWDTS) (Varna, Bulgaria: IEEE): 1\u20135. doi:10.1109\/EWDTS50664.2020.9224901. ISBN 978-1-7281-9899-6. https:\/\/ieeexplore.ieee.org\/document\/9224901\/ .   \n \n\n\u2191 Abdolkhani, Robab; Gray, Kathleen; Borda, Ann; DeSouza, Ruth (1 December 2019). \"Patient-generated health data management and quality challenges in remote patient monitoring\" (in en). JAMIA Open 2 (4): 471\u2013478. doi:10.1093\/jamiaopen\/ooz036. ISSN 2574-2531. PMC PMC6993998. PMID 32025644. https:\/\/academic.oup.com\/jamiaopen\/article\/2\/4\/471\/5572201 .   \n \n\n\nNotes \nThis presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added.\n\n\n\n\n\nSource: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.limswiki.org\/index.php\/Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\">https:\/\/www.limswiki.org\/index.php\/Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture<\/a>\nCategories: LIMSwiki journal articles (added in 2023)LIMSwiki journal articles (all)LIMSwiki journal articles on data management and sharingLIMSwiki journal articles on laboratory informaticsLIMSwiki journal articles on laboratory managementNavigation menuPage actionsJournalDiscussionView sourceHistoryPage actionsJournalDiscussionMoreToolsIn other languagesPersonal toolsLog inNavigationMain pageEncyclopedic articlesRecent changesRandom pageHelp about MediaWikiSearch\u00a0 ToolsWhat links hereRelated changesSpecial pagesPermanent linkPage informationPopular publications\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\nPrint\/exportCreate a bookDownload as PDFDownload as PDFDownload as Plain textPrintable version This page was last edited on 31 July 2023, at 21:32.Content is available under a Creative Commons Attribution-ShareAlike 4.0 International License unless otherwise noted.This page has been accessed 1,129 times.Privacy policyAbout LIMSWikiDisclaimers\n\n\n\n","ab540fa1cc32f92043343fd9cf67ec0d_html":"<body class=\"mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-206 ns-subject page-Journal_Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture rootpage-Journal_Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture skin-monobook action-view skin--responsive\"><div id=\"rdp-ebb-globalWrapper\"><div id=\"rdp-ebb-column-content\"><div id=\"rdp-ebb-content\" class=\"mw-body\" role=\"main\"><a id=\"rdp-ebb-top\"><\/a>\n<h1 id=\"rdp-ebb-firstHeading\" class=\"firstHeading\" lang=\"en\">Journal:Registered data-centered lab management system based on data ownership security architecture<\/h1><div id=\"rdp-ebb-bodyContent\" class=\"mw-body-content\"><!-- start content --><div id=\"rdp-ebb-mw-content-text\" lang=\"en\" dir=\"ltr\" class=\"mw-content-ltr\"><div class=\"mw-parser-output\">\n\n\n<h2><span class=\"mw-headline\" id=\"Abstract\">Abstract<\/span><\/h2>\n<p>University and college <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory\" title=\"Laboratory\" class=\"wiki-link\" data-key=\"c57fc5aac9e4abf31dccae81df664c33\">laboratories<\/a> are important places to train professional and technical personnel. Various regulatory departments in colleges and universities still rely on traditional laboratory management in <a href=\"https:\/\/www.limswiki.org\/index.php\/Research\" title=\"Research\" class=\"wiki-link\" data-key=\"409634fd90113f119362927fe222f549\">research<\/a> projects, which are prone to problems such as untimely <a href=\"https:\/\/www.limswiki.org\/index.php\/Information\" title=\"Information\" class=\"wiki-link\" data-key=\"6300a14d9c2776dcca0999b5ed940e7d\">information<\/a> and data transmission. The present study aimed to propose a new method to solve the problem of data islands, explicit ownership, conditional sharing, <a href=\"https:\/\/www.limswiki.org\/index.php\/Information_security\" title=\"Information security\" class=\"wiki-link\" data-key=\"9eff362d944224ff1d4ffe3a149d7cff\">data security<\/a>, and efficiency during laboratory <a href=\"https:\/\/www.limswiki.org\/index.php\/Information_management\" title=\"Information management\" class=\"wiki-link\" data-key=\"f8672d270c0750a858ed940158ca0a73\">data management<\/a>. Hence, this study aimed to develop a data-centered lab management system that enhances the security of laboratory data management and allows the data owners of the labs to control <a href=\"https:\/\/www.limswiki.org\/index.php\/Data_sharing\" title=\"Data sharing\" class=\"wiki-link\" data-key=\"a99d5fda27f755c693c65864d9286130\">data sharing<\/a> with other users. The architecture ensures <a href=\"https:\/\/www.limswiki.org\/index.php\/Information_privacy\" title=\"Information privacy\" class=\"wiki-link\" data-key=\"185f6d9f874e48914b5789317408f782\">data privacy<\/a> by binding data ownership with a person using a key management method. To achieve secure data flow, data ownership conversion through the process of authorization and confirmation was introduced. The designed lab management system enables laboratory regulatory departments to receive data in a secure form by using this platform, which could solve data sharing barriers. Finally, the proposed system was applied and run in different server environments by implementing data security registration, authorization, confirmation, and conditional sharing using SM2, SM4, RSA, and AES algorithms. The system was evaluated in terms of the execution time for several lab data with different sizes. The findings of this study indicate that the proposed strategy is secure and efficient for lab data sharing across domains.\n<\/p><p><b>Keywords<\/b>: university laboratory management, data sharing, data ownership safety architecture, conditional sharing, security and efficiency\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Introduction\">Introduction<\/span><\/h2>\n<p>The importance of knowledge for organizations is now widely recognized, being one of the resources whose management influences the success of organizations through the exchange and sharing of <a href=\"https:\/\/www.limswiki.org\/index.php\/Information\" title=\"Information\" class=\"wiki-link\" data-key=\"6300a14d9c2776dcca0999b5ed940e7d\">information<\/a>, knowledge, and experience among its members. Numerous challenges are associated with the management of <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory\" title=\"Laboratory\" class=\"wiki-link\" data-key=\"c57fc5aac9e4abf31dccae81df664c33\">laboratory<\/a> data, as labs generate a lot of experimental and management data on a daily basis. Frequent experimental accidents and the leakage of hazardous chemicals is worthy of attention. Supervision departments, such as the Education Bureau, Emergency Bureau, and Public Security Department, need various types of laboratory management data to report while supervising laboratory safety.\n<\/p><p>At present, existing laboratory management systems used by each university, regulatory department, and even various laboratories are different, which leads to obvious problems when lab data collection or emergency response is needed. For example, which laboratory management data are classified confidential, and which data can be handed over to relevant departments? Who must claim the ownership of laboratory-related data? Which departments can view or use the data? Could lab data be transferred to other departments? These questions raised laboratory data ownership issues. Secondly, some data that relate to laboratory management need to be kept confidential. Where and how can we best secure this data? Is it safe for different departments to transmit data? How can we guarantee that there will be no data transfer to others? These can be summarized as data security storage and transmission questions. In recent years, frequent university laboratory accidents have occurred; in such cases, emergency responding units and other regulatory departments need to collect real-time lab-related data. If it is still a regular report retrieval process, it cannot meet the requirements of time efficiency. So, we need to find efficient and trustworthy methods to solve this problem of obtaining data in a timely manner.\n<\/p><p>The main reason for the above problems is that laboratory management data ownership is unclear, and the existing <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory_information_management_system\" title=\"Laboratory information management system\" class=\"wiki-link\" data-key=\"8ff56a51d34c9b1806fcebdcde634d00\">laboratory information management systems<\/a> (LIMS) of each branch are independent. It is impossible to collect real-time, tamper-free, statistically accurate data when it is needed. To deal with these problems, we intend to adopt data ownership security architecture (DOSA) to securely circulate laboratory management data and cultivate an ecological and sustainable forest from the \u201cflowerpot\u201d of each independent system.<sup id=\"rdp-ebb-cite_ref-1\" class=\"reference\"><a href=\"#cite_note-1\">[1]<\/a><\/sup>\n<\/p><p>Laboratories are currently using LIMS frequently.<sup id=\"rdp-ebb-cite_ref-2\" class=\"reference\"><a href=\"#cite_note-2\">[2]<\/a><\/sup> A university may also have multiple management systems, including an equipment management system, a chemical management system, a safety examination system, etc. Related systems comprise relevant departments of the laboratory, such as the experiment management department of the Education Bureau, the accident handling department of the Emergency Bureau, and the hazardous chemicals supervision department of Public Security. Such complicated systems make it difficult to obtain information accurately and quickly when the same report requires the cooperation of different departments or laboratories. Compared to traditional manual records, the LIMS used by a specific business unit or company can be easily outdated and has high maintenance costs. Once the database needs major updates related to its data source, the existing system cannot meet the demand, and a new system must be designed so that all data can be transferred, resulting in increased costs. The linkage between different systems is weak, the efficiency is relatively low, and the security is not uniform and difficult to guarantee when it is necessary to submit data.\n<\/p><p><a href=\"https:\/\/www.limswiki.org\/index.php\/Blockchain\" title=\"Blockchain\" class=\"wiki-link\" data-key=\"ae8b186c311716aca561aaee91944f8e\">Blockchain<\/a> poses one potential solution. The medical industry uses the blockchain system to manage <a href=\"https:\/\/www.limswiki.org\/index.php\/Medical_imaging\" title=\"Medical imaging\" class=\"wiki-link\" data-key=\"dddd7e2b5706415d7af0375386e6eafa\">medical imaging<\/a> data, and the banking system uses blockchain as a new type of financial technology.<sup id=\"rdp-ebb-cite_ref-3\" class=\"reference\"><a href=\"#cite_note-3\">[3]<\/a><\/sup> Blockchain has high energy consumption, expensive development costs, but relatively high security. At present, some scholars have produced a framework diagram of the blockchain management in university education, but there is no corresponding technical means.<sup id=\"rdp-ebb-cite_ref-4\" class=\"reference\"><a href=\"#cite_note-4\">[4]<\/a><\/sup>\n<\/p><p>Considering the problems mentioned above, the lab industry would require a technology that allows data to flow securely and efficiently. This article\u2019s contributes discussion about potential technology to solve such problems. In our work, experiment-related <a href=\"https:\/\/www.limswiki.org\/index.php\/Information_management\" title=\"Information management\" class=\"wiki-link\" data-key=\"f8672d270c0750a858ed940158ca0a73\">data management<\/a> is based on the data ownership security architecture. This security technology is built such that data ownership binds with a person and it can be safely registered on this platform; meanwhile, ownership of data can be determined, and data users and owners can conditionally share or trade using ownership conversion across domains and across borders efficiently and securely. Our proposed laboratory data management architecture is shown in Figure 1.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig1_Zheng_Electronics23_12-8.png\" class=\"image wiki-link\" data-key=\"57b66e0fa0b00ad57f1899348c70fd4c\"><img alt=\"Fig1 Zheng Electronics23 12-8.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/5\/50\/Fig1_Zheng_Electronics23_12-8.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 1.<\/b> Proposed data ownership security architecture (DOSA) to manage the laboratory.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>The current study has been organized as follows. The next section explains related theories and key management. After that, we present the core strategies used, including laboratory data <a href=\"https:\/\/www.limswiki.org\/index.php\/Encryption\" title=\"Encryption\" class=\"wiki-link\" data-key=\"86a503652ed5cc9d8e2b0252a480b5e1\">encrypted<\/a> registration and an authorized transfer method for conditional <a href=\"https:\/\/www.limswiki.org\/index.php\/Data_sharing\" title=\"Data sharing\" class=\"wiki-link\" data-key=\"a99d5fda27f755c693c65864d9286130\">sharing<\/a>. We then detail the whole lab management framework and the incident searching case study based on DOSA using the above ownership conversion method. We then discuss and verify the effectiveness of the data flow process using key management based on proposed core strategies. Finally, we provide our conclusions.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Related_work\">Related work<\/span><\/h2>\n<h3><span class=\"mw-headline\" id=\"Data_ownership_security_architecture\">Data ownership security architecture<\/span><\/h3>\n<p>As data has a tendency to not be securely circulated and shared, Miao <i>et al.<\/i> proposed a data ownership security architecture (DOSA) that contains one body with two wings.<sup id=\"rdp-ebb-cite_ref-5\" class=\"reference\"><a href=\"#cite_note-5\">[5]<\/a><\/sup> The body represents data binding ownership, while one wing is the key and the other wing signifies value.\n<\/p><p>To solve the problem of data transfer barriers, DOSA proposes a global, cross-border, comprehensive, collaborative, secure, and efficient data architecture system. The whole field is no longer the cross-domain data of the head industry or a single system as previous, but all industry-related data are registered. Integration and collaboration mean that data are retrieved and used by different industry sources. This architecture has efficient security technologies for data security from source, registration, circulation, and use to supervision.\n<\/p><p>DOSA is composed of a data registration center (DRC), a data authorization center (DAC), a data exception center (DEC), data application units (DAUs), a key system, and other components. Safety and efficiency are the core of DOSA. Similar DOSA-like framework applications have been implemented in tourism, lab education, and data product trading processes.<sup id=\"rdp-ebb-cite_ref-6\" class=\"reference\"><a href=\"#cite_note-6\">[6]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-7\" class=\"reference\"><a href=\"#cite_note-7\">[7]<\/a><\/sup>\n<\/p>\n<h4><span class=\"mw-headline\" id=\"DRC\">DRC<\/span><\/h4>\n<p>The DRC stores the data in an encrypted form, which is invisible even to the administrator. It not only confirms the ownership of the data but also keeps it safe and confidential. After the data are registered, the directory is automatically generated, which is convenient for data users to view and find relevant data sources. It can register the data of relevant units or laboratories to DRC or link to DRC through an <a href=\"https:\/\/www.limswiki.org\/index.php\/Application_programming_interface\" title=\"Application programming interface\" class=\"wiki-link\" data-key=\"36fc319869eba4613cb0854b421b0934\">application programming interface<\/a> (API).<sup id=\"rdp-ebb-cite_ref-8\" class=\"reference\"><a href=\"#cite_note-8\">[8]<\/a><\/sup>\n<\/p>\n<h4><span class=\"mw-headline\" id=\"DAC\">DAC<\/span><\/h4>\n<p>To ensure the relationship between people and data ownership, DAC is responsible for the process of data ownership confirmation and authorization.<sup id=\"rdp-ebb-cite_ref-:0_9-0\" class=\"reference\"><a href=\"#cite_note-:0-9\">[9]<\/a><\/sup> By guaranteeing the interests of data owners, they are confident to put their data on the platform. The owner of the data divides data-related personnel into data owners, data producers, and data users. In the current study, a design was proposed to save all the related public keys to the DAC, and the private key is kept by their client, without placing it on any platform or <a href=\"https:\/\/www.limswiki.org\/index.php\/Cloud_computing\" title=\"Cloud computing\" class=\"wiki-link\" data-key=\"fcfe5882eaa018d920cedb88398b604f\">cloud<\/a>, to ensure security.\n<\/p>\n<h4><span class=\"mw-headline\" id=\"DEC\">DEC<\/span><\/h4>\n<p>To protect the interests of data owners, DEC needs to supervise during the data flow process. From the point of data register and data conditional sharing, even after the trading process, we took blockchain-related technical means, such as watermarks and timestamps, and added them to the data authorization process to ensure data users cannot transmit data again without the owner\u2019s permission.<sup id=\"rdp-ebb-cite_ref-10\" class=\"reference\"><a href=\"#cite_note-10\">[10]<\/a><\/sup>\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Laboratory_management_technology\">Laboratory management technology<\/span><\/h3>\n<p>Existing laboratory management usually uses a LIMS that collects related data through various laboratories. Some benefits of using a LIMS are the ability to track individual data items or <a href=\"https:\/\/www.limswiki.org\/index.php\/Sample_(material)\" title=\"Sample (material)\" class=\"wiki-link\" data-key=\"7f8cd41a077a88d02370c02a3ba3d9d6\">samples<\/a> from reported results.<sup id=\"rdp-ebb-cite_ref-11\" class=\"reference\"><a href=\"#cite_note-11\">[11]<\/a><\/sup> Some limitations of a LIMS for this specific application exist, as they may not be specialized for specific equipment, or applicable to certain chemistries.<sup id=\"rdp-ebb-cite_ref-12\" class=\"reference\"><a href=\"#cite_note-12\">[12]<\/a><\/sup> Some LIMS chemistry systems are only available to connect with the Public Safety department. It is rare to find a LIMS system for chemistry widely used in related departments. There is also a high price for many commercial LIMS systems. Various systems\u2019 costs are expensive for labs, and customization is limited to what the system offers.<sup id=\"rdp-ebb-cite_ref-13\" class=\"reference\"><a href=\"#cite_note-13\">[13]<\/a><\/sup> Additionally, experimental data are private to external parties, yet it still needs to be shared among the research team.\n<\/p><p>Recently, researchers have designed some LIMS for education labs or other specific applications. Focusing on education, there is Chaoqun and Lanlan's interactive LIMS, which incoporates experimental teaching management, experimental equipment management, laboratory open management, communication, and interactive management modules.<sup id=\"rdp-ebb-cite_ref-14\" class=\"reference\"><a href=\"#cite_note-14\">[14]<\/a><\/sup> For specific disciplines, there are LIMS and other <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory_informatics\" title=\"Laboratory informatics\" class=\"wiki-link\" data-key=\"00edfa43edcde538a695f6d429280301\">laboratory informatics<\/a> solutions specifically designed for high-level <a href=\"https:\/\/www.limswiki.org\/index.php\/Biosafety\" title=\"Biosafety\" class=\"wiki-link\" data-key=\"c8cf3b113a435bf7559c8483312268be\">biosafety<\/a> labs or <a href=\"https:\/\/www.limswiki.org\/index.php\/Biobank\" title=\"Biobank\" class=\"wiki-link\" data-key=\"4e5f94a2b2036266701220c1fd724bd2\">biobanks<\/a>.<sup id=\"rdp-ebb-cite_ref-15\" class=\"reference\"><a href=\"#cite_note-15\">[15]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-16\" class=\"reference\"><a href=\"#cite_note-16\">[16]<\/a><\/sup> However, these systems only flow data inside the lab, in which data cannot flow quickly to supervision departments or related companies. Some researchers realized that the LIMS needs to be both secure and enhance efficiency in order to enable laboratory staff to conduct their research as freely as possible; therefore, the information on research, laboratory personnel, experimental materials, and experimental equipment is generally collected and utilized by only one system.<sup id=\"rdp-ebb-cite_ref-17\" class=\"reference\"><a href=\"#cite_note-17\">[17]<\/a><\/sup> However, the current research mainly focuses on such a system's design, and there is no specific experimental technical means to verify it. Thus, we employed DOSA to rapidly and securely move data inside or outside the laboratory. In Table 1, we compare our proposed architecture to existing lab management systems.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"3\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 1.<\/b> Comparison of the proposed architecture and existing lab management.\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Topic\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Proposed DOSA lab management system\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Existing lab management systems\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Data ownership\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Clear\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Undefined\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Data security\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">The public key in DAC\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Key pair in systems\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Data source\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Global\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Specific service<sup id=\"rdp-ebb-cite_ref-18\" class=\"reference\"><a href=\"#cite_note-18\">[18]<\/a><\/sup>\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Data register\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Encrypted, fully automatic\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Admin visible, semi-automatic entry\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Data search\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Authorization visible\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Collect systems to summarize\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Application\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">For a variety of business\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">For a single item<sup id=\"rdp-ebb-cite_ref-19\" class=\"reference\"><a href=\"#cite_note-19\">[19]<\/a><\/sup>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>However, DOSA-related studies have designed models for some domains such as tourism, which only have frameworks without practical implementation.<sup id=\"rdp-ebb-cite_ref-:0_9-1\" class=\"reference\"><a href=\"#cite_note-:0-9\">[9]<\/a><\/sup> We propose to use technical methods to solve data ownership by binding persons that used a key management system to protect <a href=\"https:\/\/www.limswiki.org\/index.php\/Information_privacy\" title=\"Information privacy\" class=\"wiki-link\" data-key=\"185f6d9f874e48914b5789317408f782\">data privacy<\/a>.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Key_system_methods\">Key system methods<\/span><\/h3>\n<p>To ensure data registration and sharing security, this work adopted RSA and AES algorithms, as well as Chinese domestic commercial keys SM2 and SM4, to compare the aspects of addressing efficiency and security of data. Table 2 summarizes the pros and cons of these key algorithms.\n<\/p><p><br \/>\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"4\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 2.<\/b> Comparison of the four key algorithms examined in this research.\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Algorithm\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Description\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Advantage\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Disadvantage\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>RSA<\/b>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Asymmetric encryption that encrypts with public key; decrypts with private key<sup id=\"rdp-ebb-cite_ref-20\" class=\"reference\"><a href=\"#cite_note-20\">[20]<\/a><\/sup>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">In the process of encryption and decryption, there is no need to transmit confidential keys through the network. The key management is better than the AES algorithm.\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">The speed of encryption and decryption is relatively slow; usually it is not suitable for the encryption of a large number of data files.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>AES<\/b>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Symmetric encryption algorithm; encryption and decryption processes use the same set of keys\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">The operation does not require a computer with very high processing power and significant memory. The operation resists attacks easily. It always maintains good performance in different operating environments; the encryption speed is relatively fast.\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">It is required to secretly distribute the key before communication. The decrypted private key must be transmitted to the receiver of the encrypted data through the network.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>SM2<\/b>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Asymmetric encryption algorithm which is an elliptic curve public key cryptography algorithm based on elliptic-curve cryptography (ECC)<sup id=\"rdp-ebb-cite_ref-21\" class=\"reference\"><a href=\"#cite_note-21\">[21]<\/a><\/sup>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Compared with RSA, the performance of SM2 is better and more secure. The password complexity is high, the processing speed is fast, and the computer performance consumption is comparatively small.\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Encryption and decryption take a relatively long time and are suitable for encrypting small amounts of data.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>SM4<\/b>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Symmetric encryption algorithm; the key length and block length are both 128 bits\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">SM4 is efficient and secure, while remaining easy to implement across software and hardware. It usually has a fast computing speed.\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">The management and distribution of keys are relatively difficult and not secure enough.\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>SM2 and SM4 are cryptographic standards authorized to be used in China. Relevant studies have shown that the SM2 and SM4 algorithms are more secure than ECDSA and AES.<sup id=\"rdp-ebb-cite_ref-22\" class=\"reference\"><a href=\"#cite_note-22\">[22]<\/a><\/sup> During a controlled experiment, AES outperformed SM4 by a significant margin.<sup id=\"rdp-ebb-cite_ref-23\" class=\"reference\"><a href=\"#cite_note-23\">[23]<\/a><\/sup> Symmetric encryption is usually used when the message sender needs to encrypt a large amount of data. It has the characteristics of an open algorithm, a small amount of calculation, and fast encryption speed. The advantage of the symmetric encryption algorithm lies in the high speed of encryption and decryption, and the difficulty of cracking when using a long key. The disadvantages of symmetric encryption are that key management and distribution are difficult and insecure. Before the data are transmitted, the sender and the receiver must agree on a secret key, and both parties must keep the key. If the key of one party is leaked, the encrypted information is insecure, and its security cannot be guaranteed.<sup id=\"rdp-ebb-cite_ref-24\" class=\"reference\"><a href=\"#cite_note-24\">[24]<\/a><\/sup>\n<\/p><p>The advantage of asymmetric encryption is its higher security. The public key is made available, and the private key is kept by itself so there is no need to give the private key to others. The disadvantage of asymmetric encryption is that its speed is relatively slow, so it is only suitable for encrypting a small amount of data.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Methodology\">Methodology<\/span><\/h2>\n<h3><span class=\"mw-headline\" id=\"Laboratory-related_data_and_persons\">Laboratory-related data and persons<\/span><\/h3>\n<p>Managed laboratory information and data can be divided into unconditional and conditional sharing categories. For example, the opening hours of the laboratory represent unconditional sharing data. The transmission of the laboratory's inventory and usage of hazardous chemicals needs conditional sharing by the administrative department (data owners). Laboratory safety data, accident statistics, and related training data are transnational conditional sharing data. Lab-related data are also classified into a security level ranging from zero to five.<sup id=\"rdp-ebb-cite_ref-25\" class=\"reference\"><a href=\"#cite_note-25\">[25]<\/a><\/sup> Class zero data is data that can be public, such as laboratory introductions and research group members\u2019 profiles which do not need any security protection. Class five data is critical national scientific research experiment data which must be kept strictly confidential. Classes one through four refer to data that the laboratory needs to share or that are trade-protected.\n<\/p><p>Lab data producers can be one or several persons, such as laboratory administrators, experimental instructors, or relevant teachers. A data owner is usually a person or a unit, such as a person in charge of the laboratory team or the emergency department. Data producers can input data, but they do not have the authority to share or trade data. After the data user has found targeted data by querying the DRC category, they need to contact the data owner. If the data owner considers that the data user can apply to view or use these data sources, the owner finds the user\u2019s public key from the DAC then encrypts these data with the user\u2019s public key to complete the authorization. The data user opens encrypted lab data with their private key to finish the conditional sharing of data (Figure 2a,b).\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig2_Zheng_Electronics23_12-8.png\" class=\"image wiki-link\" data-key=\"2f1d94ae261732e1b3394056da7aa71c\"><img alt=\"Fig2 Zheng Electronics23 12-8.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/0\/06\/Fig2_Zheng_Electronics23_12-8.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 2.<\/b> Illustrates the relationship of data-related persons or branches (a) and the relationship between data user, producer, and owner (b).<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h3><span class=\"mw-headline\" id=\"Data_encryption_registration_and_application_process\">Data encryption registration and application process<\/span><\/h3>\n<p>The laboratory data are uploaded to the DRC by the owner, and the owner\u2019s public key is added to verify the ownership of the data. It can guarantee the privacy and security of the data at the source. The data are stored in the DRC, which the administrator cannot view without permission. The encrypted lab data are transmitted to the DRC, and the directory is automatically generated by keywords. The relevant users can apply for permission to obtain the data after searching in the data directory. Figure 3 shows the data owner encrypting the registration data, which automatically generates an index directory, then data users apply for the required data by viewing the catalog.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig3_Zheng_Electronics23_12-8.png\" class=\"image wiki-link\" data-key=\"dcdb6277edbf94aedfcb7a178cbe3c5f\"><img alt=\"Fig3 Zheng Electronics23 12-8.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/6\/67\/Fig3_Zheng_Electronics23_12-8.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 3.<\/b> Data encryption registration process.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h3><span class=\"mw-headline\" id=\"Confirmation_and_authorization_of_the_data_transmission_process\">Confirmation and authorization of the data transmission process<\/span><\/h3>\n<h4><span class=\"mw-headline\" id=\"Mutual_trust_-_Single-layer_encryption\">Mutual trust - Single-layer encryption<\/span><\/h4>\n<p>After the data user searches the DRC catalog and queries the data they need, the data user communicates with the data owner. If the user obtains permission, the data owner encrypts the data with the user\u2019s public key. Then, the data user can use their private key to decrypt data. Meanwhile, to ensure the interests of the data owner, the entire process has traceability technology in the DOSA lab management system. For example, time stamps and digital signatures are adopted, which can ensure that data users do not transmit data outside the authorized context.<sup id=\"rdp-ebb-cite_ref-26\" class=\"reference\"><a href=\"#cite_note-26\">[26]<\/a><\/sup> If the mutual parties trust each other, the process of conditional sharing occurs, as shown in Figure 4. \n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig4_Zheng_Electronics23_12-8.png\" class=\"image wiki-link\" data-key=\"0d18760039540d3768cacd2438b59d18\"><img alt=\"Fig4 Zheng Electronics23 12-8.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/7\/76\/Fig4_Zheng_Electronics23_12-8.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 4.<\/b> Single-layer encryption.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>Algorithm 1 shows how the lab data user retrieves data in the DRC during the process of data ownership conversion. Table 3 shows the definition of symbols used in the algorithm.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"1\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Algorithm 1.<\/b> Encryption_Once ( )\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">\n<p><b>Input<\/b>: Lab data owner O<sub>n<\/sub> with their lab data record <i>R<sub>n<\/sub><\/i>. Lab data user with their private key <i>Uprk<sub>n<\/sub><\/i>.<br \/>\n<b>Output<\/b>: Boolean(True or False)<br \/>\n<code>1. #Function used to encrypt the lab data record.<br \/>\n2.\u2003<b>For<\/b> owner O query the user\u2019s public key <i>Upubk<sub>n<\/sub><\/i> from the DAC <i>Dacv<sub>n<\/sub><\/i><br \/>\n3.\u2003#Check the DAC\u2019s data<br \/>\n4.\u2003<b>if<\/b>(role == \u201cOwner\u201d) then<br \/>\n5.\u2003\u2003\u2003Encryption with the user\u2019s public key <i>Upubk<sub>n<\/sub><\/i><br \/>\n6.\u2003\u2003\u2003lab data record <i>R<sub>n<\/sub><\/i>\u2192encrypted data<br \/>\n7.\u2003\u2003\u2003encrypted data\u2192<i>Drc<sub>n<\/sub><\/i><br \/>\n8.\u2003\u2003\u2003<b>return<\/b> True<br \/>\n9. \u2003<b>else<\/b><br \/>\n10.\u2003\u2003\u2003<b>return<\/b> False<br \/>\n11.\u2003<b>end if<\/b><br \/>\n12. \u2003<b>end for<\/b><br \/>\n13.\u2003<b>For<\/b> user U query the data register center catalogue view <i>Drcv<sub>n<\/sub><\/i><br \/>\n14. \u2003<b>if<\/b>(role == \u201cUser\u201d) then<br \/>\n15.\u2003\u2003\u2003Decryption using the user\u2019s private key <i>Uprk<sub>n<\/sub><\/i><br \/>\n16.\u2003\u2003<i>Drc<sub>n<\/sub><\/i>\u2192encrypted data<br \/>\n17.\u2003\u2003\u2003encrypted data\u2192lab data record <i>R<sub>n<\/sub><\/i><br \/>\n18.\u2003\u2003\u2003<b>return<\/b> True<br \/>\n19.\u2003<b>else<\/b><br \/>\n20.\u2003\u2003<b>return<\/b> False<br \/>\n21.\u2003<b>end if<\/b><br \/>\n22.\u2003<b>end for<\/b><br \/>\n23.\u2003<b>end<\/b> function<\/code>\n<\/p>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"2\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 3.<\/b> The definition of symbols used in the algorith.\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Symbols\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Definition\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><i>U<sub>n<\/sub><\/i>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><i>n<\/i>th Lab Data User\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><i>O<sub>n<\/sub><\/i>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><i>n<\/i>th Lab Data Owner\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><i>R<sub>n<\/sub><\/i>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><i>n<\/i>th Lab Data Record\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><i>Upubk<sub>n<\/sub><\/i>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><i>n<\/i>th User Public Key\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><i>Uprk<sub>n<\/sub><\/i>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><i>n<\/i>th User Private Key\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><i>Opub<sub>n<\/sub><\/i>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><i>n<\/i>th Owner Public Key\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><i>Oprk<sub>n<\/sub><\/i>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><i>n<\/i>th Owner Private Key\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><i>Drc<sub>n<\/sub><\/i>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><i>n<\/i>th Data Register Center\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><i>Drcv<sub>n<\/sub><\/i>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><i>n<\/i>th Data Register Center Catalogue View\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><i>Dacv<sub>n<\/sub><\/i>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><i>n<\/i>th Data Authority Center View\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h4><span id=\"rdp-ebb-Data_users_do_not_trust_the_data_source_\u2014_Double_encryption\"><\/span><span class=\"mw-headline\" id=\"Data_users_do_not_trust_the_data_source_.E2.80.94_Double_encryption\">Data users do not trust the data source \u2014 Double encryption<\/span><\/h4>\n<p>After checking the catalog in the DRC and communicating with the data owner about the data product\u2019s price, the data user may doubt the authenticity of the data source and ownership. In this situation, the data owner will add this data product to the owner\u2019s private key, then encrypt it by using the user\u2019s public key, which is found in the DAC. As the next step, data owners input this encrypted data to the DRC and then inform the users.\n<\/p><p>Firstly, the data user finds the data owner\u2019s public key from the DAC, and they use it to open the double-encrypted data product. As the next step, the data user utilizes their private key to decrypt the data. Figure 5 shows the transaction process of experimental data products when the data user questions the data authenticity or the data source\u2019s ownership.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig5_Zheng_Electronics23_12-8.png\" class=\"image wiki-link\" data-key=\"501579875b8214a17dfb9849a125ea62\"><img alt=\"Fig5 Zheng Electronics23 12-8.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/d\/d9\/Fig5_Zheng_Electronics23_12-8.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 5.<\/b> Double encryption.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>Using a person-binding data owner method to finish conditional data sharing can break data barriers. We also resolve issues of the laboratory data class and ownership conversion process.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"DOSA_framework_components_-_Managed_laboratory_data_linked_to_rich_soils\">DOSA framework components - Managed laboratory data linked to rich soils<\/span><\/h2>\n<h3><span class=\"mw-headline\" id=\"Overall_framework\">Overall framework<\/span><\/h3>\n<p>This section briefly describes the entire laboratory data transfer framework using DOSA, designed by the core techniques. Figure 6 shows the overall structure of DOSA to enable laboratory-related data to break barriers. The existing LIMS is independent. We use the proposed data security protection technology such that data can be stored securely and conditionally. Every laboratory uses the public key algorithm to store encrypted data which can determine ownership. If any supervision departments (such as the Education Bureau) need to collect a certain type of data, they just need to apply for related laboratories. The data owners decide which data can be shared, then find the Education Bureau\u2019s public key in DAC that can encrypt related data to the DRC of the Education Bureau so data users can obtain real-time, non-tampered data from various university laboratories. Compared with the current information collected, the DOSA method using the ownership conversion technique can solve existing problems with isolated laboratory data islands.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig6_Zheng_Electronics23_12-8.png\" class=\"image wiki-link\" data-key=\"b80297ddbb28a682012265cc0e8a9dcb\"><img alt=\"Fig6 Zheng Electronics23 12-8.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/f\/f0\/Fig6_Zheng_Electronics23_12-8.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 6.<\/b> The whole structure of laboratory management based on DOSA.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h3><span class=\"mw-headline\" id=\"Laboratory_accident_searching_based_on_DOSA\">Laboratory accident searching based on DOSA<\/span><\/h3>\n<p>We selected the registration and inquiry of laboratory hazardous chemical materials as a case study. The full cycle of hazardous chemical materials includes application, acquirement, procurement, transportation, usage, recycling waste liquid, accidental leakage management, experimental operation accident, etc. The whole process needs to be supervised by multiple relevant units, data need to be circulated, and sharing is needed to break the information cocoon. There are many departments involved, and as such, routine security inspections need to be conducted and results should be submitted according to the protocols, which is not only inefficient but also error-prone and tamper-prone. In case there is an emergency leak or experiment-related explosion, an integrated and efficient system is necessary to query related data urgently. This <a href=\"https:\/\/www.limswiki.org\/index.php\/Workflow\" title=\"Workflow\" class=\"wiki-link\" data-key=\"92bd8748272e20d891008dcb8243e8a8\">workflow<\/a> of searching for or collecting hazardous chemicals data among related branches, such as laboratories, the Education Bureau, the Public Security Bureau Inspection Department, and chemical companies, is shown in Figure 7.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig7_Zheng_Electronics23_12-8.png\" class=\"image wiki-link\" data-key=\"fef9d18326b096065b108b6d91067679\"><img alt=\"Fig7 Zheng Electronics23 12-8.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/3\/3f\/Fig7_Zheng_Electronics23_12-8.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 7.<\/b> The workflow of querying hazardous chemicals.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>The hazardous chemicals can be queried through the API specification interface (query by CAS or chemical name) directly among different branches. Hazardous chemical users, administrators, public security departments, emergency departments, and hazardous chemical suppliers can query in real time.\n<\/p><p>If an accident happens in the laboratory, we could obtain cross-domain and cross-branch information by each unit\u2019s authorization in real time. The proposed structure, which can search the corresponding hazardous chemical information, equipment details, relevant personnel profile, and experimental project proposal, etc., is shown in Figure 8.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig8_Zheng_Electronics23_12-8.png\" class=\"image wiki-link\" data-key=\"ce9fba3e3e7a4f9073b62d4b1f26f634\"><img alt=\"Fig8 Zheng Electronics23 12-8.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/4\/4a\/Fig8_Zheng_Electronics23_12-8.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 8.<\/b> Structure of laboratory accident querying in DRC.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h2><span class=\"mw-headline\" id=\"Experiments\">Experiments<\/span><\/h2>\n<p>In this section, we discuss testing the efficiency of managed lab data based on DOSA by putting the proposed framework into practice. We investigated the time consumption of key generation and public key transmission to DRC by different algorithms using Python 3.10.9. Then, we tried to use four key algorithms\u2014SM2, SM4, RSA, and AES\u2014to compare which algorithm was faster in the proposed architecture regarding authorized encryption and decryption. Combining the security level of the lab data mentioned in the above chapters, we try to find the most suitable solution.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Experimental_environment\">Experimental environment<\/span><\/h3>\n<p>Table 4 shows the details of the experiment environment. The algorithm suits the corresponding experimental data class, which can consider both efficiency and security.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"1\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 4.<\/b> Details of the experiment environment.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">\n<p><i>Environment 1 HOST 1<\/i><br \/>\nProcessor: Intel(R) Core (TM) i5-8300H CPU @ 2.30 GHz<br \/>\nMemory: 8.0 GB<br \/>\nMain hard disk: NVMe WDC PC SN520 SDA<br \/>\nOperating system: Windows 11 \u00d764 Professional Edition Insider Preview<br \/>\nProgramming software: Microsoft Visual Studio Code<br \/>\nTest work data: lab data xlsx\n<\/p>\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">\n<p><i>Environment 2 HOST 2<\/i><br \/>\nProcessor: AMD Ryzen 5 3550H with Radeon Vega Mobile Gfx 2.10 GHz<br \/>\nMemory: 8.0 GB<br \/>\nMain hard disk: SAMSUNG MZVLB512HAJQ-00000<br \/>\nOperating system: Windows 10 Home<br \/>\nProgramming software: Pycharm Community Edition<br \/>\nTest work data: lab data xlsx\n<\/p>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h3><span class=\"mw-headline\" id=\"Simulation_key_generation_and_transmission_experiment\">Simulation key generation and transmission experiment<\/span><\/h3>\n<p>The DAC module includes the right confirmation and authorization. The right confirmation determines the lab data ownership. The ownership of the experimental data belongs to the experimental team designer. After storing data in the DRC, the lab data product adds the research group public key to determine the ownership.\n<\/p><p>During lab data transactions, the data user wants to purchase the experimental data product. After payment, the data user needs to find the data owner\u2019s public key in the DAC, then the user uses the owner\u2019s public key to open the lab data product. This is an authorization ownership process.\n<\/p><p>In this experiment, a key pair was generated in Chengdu, Sichuan Province, China. The public key was sent to Beijing, China (DAC). We carried out 12 experiments generating key pairs, and the tests were performed on Environment 1 (env1) and Environment 2 (env2), respectively. Figure 9 shows a speed test of the time required to generate a key pair. The results of this experiment show that using the AES algorithm is the fastest (1.232 \u00d7 10\u22125 s\/1.605 \u00d7 10\u22125 s) in two environments. Using SM4 (3.197 \u00d7 10\u22125 s\/6.882 \u00d7 10\u22125 s) to create a key pair is the second fastest, which is close to AES. Meanwhile, the SM2 algorithm is the most time-consuming way.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig10_Zheng_Electronics23_12-8.png\" class=\"image wiki-link\" data-key=\"d0a8ff6cd65b7843e0acdd35e7d6d981\"><img alt=\"Fig10 Zheng Electronics23 12-8.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/3\/38\/Fig10_Zheng_Electronics23_12-8.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 9.<\/b> Time consumption for generating key pairs using SM2, SM4, AES, and RSA algorithms in Environment 1 (env1) <b>(a)<\/b> and Environment 2 (env2) <b>(b)<\/b>.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>Further, under two different operating system environments, the experiment simulated the speed of public key transmission from Chengdu to Beijing, and the results are shown in Figure 10. Compared with the key generation experiment, the speed of AES transmission is still the fastest, and the average speed is 1.327 s\/1.400 s. In contrast, SM2 is the slowest, with an average speed of 1.485 s\/1.571 s.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig9_Zheng_Electronics23_12-8.png\" class=\"image wiki-link\" data-key=\"51bb0a338f16235daf56eed69085c15b\"><img alt=\"Fig9 Zheng Electronics23 12-8.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/d\/db\/Fig9_Zheng_Electronics23_12-8.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 10.<\/b> Time consumption for simulating transmission public keys to DRC using SM2, SM4, AES, and RSA in Environment 1 (env1) <b>(a)<\/b> and Environment 2 (env2) <b>(b)<\/b>.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h3><span class=\"mw-headline\" id=\"Confirmation_and_authorization_experiments\">Confirmation and authorization experiments<\/span><\/h3>\n<p>In the proposed system, we tried to test lab management digital records 50\u2013300 KB, which were encrypted, respectively, using the SM2, SM4, RSA, and AES algorithms, then stored in the DRC of HOST 1 and 2. The encryption time of these data texts is shown in Figure 11.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig11_Zheng_Electronics23_12-8.png\" class=\"image wiki-link\" data-key=\"5e312cbfbb98dd4a1cdaaa34b4095a6b\"><img alt=\"Fig11 Zheng Electronics23 12-8.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/6\/61\/Fig11_Zheng_Electronics23_12-8.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 11.<\/b> Time consumption of authorization encryption of different experimental samples using SM2, SM4, AES, and RSA in Environment 1 (env1) <b>(a)<\/b> and Environment 2 (env2) <b>(b)<\/b>.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>After the lab data user obtained the data owner\u2019s public key from DAC, the lab data products were decrypted with four different key management methods, and the time taken is shown in Figure 12. \n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig12_Zheng_Electronics23_12-8.png\" class=\"image wiki-link\" data-key=\"e74e95e807702d31cac972d13bc89e14\"><img alt=\"Fig12 Zheng Electronics23 12-8.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/b\/b2\/Fig12_Zheng_Electronics23_12-8.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 12.<\/b> Time consumption of authorized decryption of different experimental samples using SM2, SM4, AES, and RSA in Environment 1 (env1) <b>(a)<\/b> and Environment 2 (env2) <b>(b)<\/b>.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>This experiment was performed 16 times, with different sizes of experimental products as variables, under HOST 1 and 2. The following is the logic Algorithm 2 for this experiment:\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"1\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Algorithm 2.<\/b> Calculate encryption and decryption time ( )\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">\n<p><b>Data<\/b>: Example excel file<br \/>\n<b>Result<\/b>: Time consumption for encrypt and decrypt<br \/>\n<code>1. Data format conversion:<br \/>\n2.\u2003\u2003\u2003excel file -> json -> string -> bytes<br \/>\n3. \u2003<b>if<\/b> data.type == bytes <b>then<\/b><br \/>\n4.\u2003\u2003\u2003time.record<br \/>\n5.\u2003\u2003\u2003algorithm.encrypt(data)<br \/>\n6.\u2003\u2003\u2003time.record<br \/>\n7.\u2003\u2003\u2003algorithm.decrypt(data)<br \/>\n8.\u2003\u2003\u2003time. record<br \/>\n9.\u2003<b>end if<\/b><br \/>\n10.\u2003calc(time.consumption)<\/code>\n<\/p>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>As shown in Figure 11 and Figure 12, AES was the fastest in performance to encrypt and decrypt lab data in the DOSA system. In particular, there are obvious advantages in decrypting data using AES. In the prior Table 2, we compared the security performance of these four algorithms. AES is a symmetric encryption algorithm, which has the conventional requirement to secretly distribute the key before communication, and the private key must be transmitted to the receiver through the network; thus, the key is not easy to keep secret and manage. However, as we proposed an ownership framework, the private key does not need to be transmitted to the data owner, so this shortcoming can be ignored. In summary, combined with security and speed performance, for lab data of class 1 to 4, it is recommended to use the AES algorithm to encrypt registration to determine ownership using the public key, and to decrypt to authorize with the private key based on DOSA.\n<\/p><p>However, also need to take high-security measures (class 5 lab data) into account, such as special major laboratory explosion data, or state secret experiment plans. For these security class 5 lab data, we need to pay more attention to safety performance. Combined with Table 2, SM2 shows better safety performance. So, in the current experiment, class 5 was applied to samples to compare the authorization and confirmation time efficiency using SM2.\n<\/p><p>Host 1\u2019s speed of encryption storage of a larger-volume data text (over 230 KB) is double that of Host 2. Additionally, the encryption storage speed of Host 1 for small and medium volume experimental data was three to four times faster than Host 2. However, it seems there is no difference in the data decryption time consumption between the two environments.\n<\/p><p>As shown in Figure 13, it is recommended to choose a larger experimental sample for class 5 data encryption and decryption, which could save time.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig13_Zheng_Electronics23_12-8.png\" class=\"image wiki-link\" data-key=\"3cbca2ad6758b51aea62d587c4ea46d9\"><img alt=\"Fig13 Zheng Electronics23 12-8.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/f\/f3\/Fig13_Zheng_Electronics23_12-8.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 13.<\/b> Time consumption of encryption <b>(a)<\/b> and decryption <b>(b)<\/b> of different experimental samples using SM2.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h3><span class=\"mw-headline\" id=\"Evaluate_results\">Evaluate results<\/span><\/h3>\n<p>Combining the above experiments based on the proposed system, it is recommended to use the AES algorithm to encrypt the registration, authorization, and decryption of ordinary security-level data (class 1\u20134). For a high level of security experimental data (class 5), we recommend using the SM2 algorithm to encrypt larger experimental documents at one time, which can improve the efficiency.\n<\/p><p>The advantage of DOSA is that it binds data and ownership together, protects data with key technologies, and enables secure sharing. This study designed the whole conditional sharing and transaction process of laboratory-related data based on DOSA. In this process, class 1\u20135 data are protected by encryption technology to ensure their security. Through the key algorithm and the Gmssl (open-source toolbox), the problem of data leakage can be solved in the data transaction process. Using Python to encrypt the laboratory data with the tools provided by DOSA, it can be considered safe to share relevant laboratory data cross-domain and internally and externally.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Conclusions\">Conclusions<\/span><\/h2>\n<p>To solve the problem of isolated islands of laboratory data, we proposed a data-ownership-based security architecture for confirming lab data rights, and we used a secret key algorithm to implement internal and external laboratory data conditional sharing. A data owner inputs encrypted data into the DRC, while confirming sharing rights. When a data user needs statistics or supervision, they can apply for targeted data by searching the directory. According to the different security levels of lab data, the appropriate key algorithm is selected for ownership authorization, which can realize efficient conditional sharing.\n<\/p><p>Laboratory information and data necessarily need to be conditionally shared for global and cross-border purposes. Often, these data are dispersed across various information systems, posing a challenge in sharing data for a safe and efficient lab data management system. Meanwhile, it is not easy to summarize different sources of real-time data, and it is also difficult to search data urgently among the relevant departments of the independent systems. Moreover, the security performance of the different systems is also inconsistent. As such, we developed our data architecture to manage laboratory-related data by allowing relevant units, universities, and laboratories to encrypt data with public keys and upload them to the data register center to form a data directory. Data users can search \u201cDirectory\u201d or \u201cdata owner\u201d names to apply to the data owner for data viewing or use. Data can be shared conditionally through public key confirmation and private key authorization to break lab data barriers. Then, we used experiments to evaluate which algorithm is better for the efficiency of this system. We suggest AES for ordinary experimental data. For higher levels of experimental data, we can use SM2 to process larger data at one time, which addresses both efficiency and security of data. From these experiments, we verified the feasibility and efficient security of DOSA to manage lab-related data.\n<\/p><p>This ownership security architecture for laboratory-managed information and data can also be applied to other fields, such as smart city construction. In the future, we will focus on the prediction of laboratory accidents on this system. Our long-term work goal is to use this data architecture to process the artificial intelligence analysis of experimental management data. Through the entry of a large amount of data, under the premise of the authorization of the data user, we can use the registered massive data to make predictions, such as analyzing the probability of an experimental accident occurring.\n<\/p>\n<h2><span id=\"rdp-ebb-Abbreviations,_acronyms,_and_initialisms\"><\/span><span class=\"mw-headline\" id=\"Abbreviations.2C_acronyms.2C_and_initialisms\">Abbreviations, acronyms, and initialisms<\/span><\/h2>\n<ul><li><b>API<\/b>: application programming interface<\/li>\n<li><b>DAC<\/b>: data authorization center<\/li>\n<li><b>DAU<\/b>: data application unit<\/li>\n<li><b>DEC<\/b>: data exception center<\/li>\n<li><b>DRC<\/b>: data registration center<\/li>\n<li><b>DOSA<\/b>: data ownership security architecture<\/li>\n<li><b>ECC<\/b>: elliptic-curve cryptography<\/li>\n<li><b>env1<\/b>: Environment 1<\/li>\n<li><b>env2<\/b>: Environment 2<\/li>\n<li><b>LIMS<\/b>: laboratory information management system<\/li><\/ul>\n<h2><span class=\"mw-headline\" id=\"Acknowledgements\">Acknowledgements<\/span><\/h2>\n<h3><span class=\"mw-headline\" id=\"Author_contributions\">Author contributions<\/span><\/h3>\n<p>Conceptualization, X.Z. and F.M.; methodology, X.Z. and N.C.; software, X.Z.; validation, P.U., X.Z. and F.M.; formal analysis, X.Z. and P.U.; data curation, X.Z. and P.U.; writing\u2014original draft preparation, X.Z.; writing\u2014review and editing, X.Z., P.U. and F.M; visualization, X.Z.; supervision, F.M., N.C. and P.U.; project administration, X.Z.; funding acquisition, X.Z. All authors have read and agreed to the published version of the manuscript.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Funding\">Funding<\/span><\/h3>\n<p>This research received no external funding.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Conflicts_of_interest\">Conflicts of interest<\/span><\/h3>\n<p>The authors declare no conflict of interest.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"References\">References<\/span><\/h2>\n<div class=\"reflist references-column-width\" style=\"-moz-column-width: 30em; -webkit-column-width: 30em; column-width: 30em; list-style-type: decimal;\">\n<div class=\"mw-references-wrap mw-references-columns\"><ol class=\"references\">\n<li id=\"cite_note-1\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-1\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Miao, Fang; Fan, Wenjie; Yang, Wenhui; Xie, Yan (19 January 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/dl.acm.org\/doi\/10.1145\/3309074.3309093\" target=\"_blank\">\"The study of data-oriented and ownership-based security architecture in open internet environment\"<\/a> (in en). <i>Proceedings of the 3rd International Conference on Cryptography, Security and Privacy<\/i> (Kuala Lumpur Malaysia: ACM): 121\u2013129. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1145%2F3309074.3309093\" target=\"_blank\">10.1145\/3309074.3309093<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-4503-6618-2<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/dl.acm.org\/doi\/10.1145\/3309074.3309093\" target=\"_blank\">https:\/\/dl.acm.org\/doi\/10.1145\/3309074.3309093<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+study+of+data-oriented+and+ownership-based+security+architecture+in+open+internet+environment&rft.jtitle=Proceedings+of+the+3rd+International+Conference+on+Cryptography%2C+Security+and+Privacy&rft.aulast=Miao&rft.aufirst=Fang&rft.au=Miao%2C%26%2332%3BFang&rft.au=Fan%2C%26%2332%3BWenjie&rft.au=Yang%2C%26%2332%3BWenhui&rft.au=Xie%2C%26%2332%3BYan&rft.date=19+January+2019&rft.pages=121%E2%80%93129&rft.place=Kuala+Lumpur+Malaysia&rft.pub=ACM&rft_id=info:doi\/10.1145%2F3309074.3309093&rft.isbn=978-1-4503-6618-2&rft_id=https%3A%2F%2Fdl.acm.org%2Fdoi%2F10.1145%2F3309074.3309093&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-2\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-2\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Antes, Alison L.; Kuykendall, Ashley; DuBois, James M. (24 April 2019). Master, Zubin. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/dx.plos.org\/10.1371\/journal.pone.0214595\" target=\"_blank\">\"The lab management practices of \u201cResearch Exemplars\u201d that foster research rigor and regulatory compliance: A qualitative study of successful principal investigators\"<\/a> (in en). <i>PLOS ONE<\/i> <b>14<\/b> (4): e0214595. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1371%2Fjournal.pone.0214595\" target=\"_blank\">10.1371\/journal.pone.0214595<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1932-6203\" target=\"_blank\">1932-6203<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC6481787\/\" target=\"_blank\">PMC6481787<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/31017929\" target=\"_blank\">31017929<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/dx.plos.org\/10.1371\/journal.pone.0214595\" target=\"_blank\">https:\/\/dx.plos.org\/10.1371\/journal.pone.0214595<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+lab+management+practices+of+%E2%80%9CResearch+Exemplars%E2%80%9D+that+foster+research+rigor+and+regulatory+compliance%3A+A+qualitative+study+of+successful+principal+investigators&rft.jtitle=PLOS+ONE&rft.aulast=Antes&rft.aufirst=Alison+L.&rft.au=Antes%2C%26%2332%3BAlison+L.&rft.au=Kuykendall%2C%26%2332%3BAshley&rft.au=DuBois%2C%26%2332%3BJames+M.&rft.date=24+April+2019&rft.volume=14&rft.issue=4&rft.pages=e0214595&rft_id=info:doi\/10.1371%2Fjournal.pone.0214595&rft.issn=1932-6203&rft_id=info:pmc\/PMC6481787&rft_id=info:pmid\/31017929&rft_id=https%3A%2F%2Fdx.plos.org%2F10.1371%2Fjournal.pone.0214595&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-3\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-3\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Idrees, Sheikh Mohammad; Nowostawski, Mariusz; Jameel, Roshan; Mourya, Ashish Kumar (16 April 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.mdpi.com\/2079-9292\/10\/8\/951\" target=\"_blank\">\"Security Aspects of Blockchain Technology Intended for Industrial Applications\"<\/a> (in en). <i>Electronics<\/i> <b>10<\/b> (8): 951. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3390%2Felectronics10080951\" target=\"_blank\">10.3390\/electronics10080951<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2079-9292\" target=\"_blank\">2079-9292<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.mdpi.com\/2079-9292\/10\/8\/951\" target=\"_blank\">https:\/\/www.mdpi.com\/2079-9292\/10\/8\/951<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Security+Aspects+of+Blockchain+Technology+Intended+for+Industrial+Applications&rft.jtitle=Electronics&rft.aulast=Idrees&rft.aufirst=Sheikh+Mohammad&rft.au=Idrees%2C%26%2332%3BSheikh+Mohammad&rft.au=Nowostawski%2C%26%2332%3BMariusz&rft.au=Jameel%2C%26%2332%3BRoshan&rft.au=Mourya%2C%26%2332%3BAshish+Kumar&rft.date=16+April+2021&rft.volume=10&rft.issue=8&rft.pages=951&rft_id=info:doi\/10.3390%2Felectronics10080951&rft.issn=2079-9292&rft_id=https%3A%2F%2Fwww.mdpi.com%2F2079-9292%2F10%2F8%2F951&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-4\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-4\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Meyliana; Cassandra, Cadelina; Surjandy; Widjaja, Henry Antonius Eka; Prabowo, Harjanto; Fernando, Erick; Chandra, Yakob Utama (1 August 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9211209\/\" target=\"_blank\">\"A Blockchain Technology-Based for University Teaching and Learning Processes\"<\/a>. <i>2020 International Conference on Information Management and Technology (ICIMTech)<\/i> (Bandung, Indonesia: IEEE): 244\u2013247. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FICIMTech50083.2020.9211209\" target=\"_blank\">10.1109\/ICIMTech50083.2020.9211209<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-7281-7071-8<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9211209\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/9211209\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Blockchain+Technology-Based+for+University+Teaching+and+Learning+Processes&rft.jtitle=2020+International+Conference+on+Information+Management+and+Technology+%28ICIMTech%29&rft.aulast=Meyliana&rft.au=Meyliana&rft.au=Cassandra%2C%26%2332%3BCadelina&rft.au=Surjandy&rft.au=Widjaja%2C%26%2332%3BHenry+Antonius+Eka&rft.au=Prabowo%2C%26%2332%3BHarjanto&rft.au=Fernando%2C%26%2332%3BErick&rft.au=Chandra%2C%26%2332%3BYakob+Utama&rft.date=1+August+2020&rft.pages=244%E2%80%93247&rft.place=Bandung%2C+Indonesia&rft.pub=IEEE&rft_id=info:doi\/10.1109%2FICIMTech50083.2020.9211209&rft.isbn=978-1-7281-7071-8&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F9211209%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-5\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-5\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Miao, Fang; Fan, Wenjie; Yang, Wenhui; Xie, Yan (19 January 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/dl.acm.org\/doi\/10.1145\/3309074.3309093\" target=\"_blank\">\"The study of data-oriented and ownership-based security architecture in open internet environment\"<\/a> (in en). <i>Proceedings of the 3rd International Conference on Cryptography, Security and Privacy<\/i> (Kuala Lumpur Malaysia: ACM): 121\u2013129. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1145%2F3309074.3309093\" target=\"_blank\">10.1145\/3309074.3309093<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-4503-6618-2<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/dl.acm.org\/doi\/10.1145\/3309074.3309093\" target=\"_blank\">https:\/\/dl.acm.org\/doi\/10.1145\/3309074.3309093<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+study+of+data-oriented+and+ownership-based+security+architecture+in+open+internet+environment&rft.jtitle=Proceedings+of+the+3rd+International+Conference+on+Cryptography%2C+Security+and+Privacy&rft.aulast=Miao&rft.aufirst=Fang&rft.au=Miao%2C%26%2332%3BFang&rft.au=Fan%2C%26%2332%3BWenjie&rft.au=Yang%2C%26%2332%3BWenhui&rft.au=Xie%2C%26%2332%3BYan&rft.date=19+January+2019&rft.pages=121%E2%80%93129&rft.place=Kuala+Lumpur+Malaysia&rft.pub=ACM&rft_id=info:doi\/10.1145%2F3309074.3309093&rft.isbn=978-1-4503-6618-2&rft_id=https%3A%2F%2Fdl.acm.org%2Fdoi%2F10.1145%2F3309074.3309093&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-6\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-6\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Miao, F.; Yang, W.; Ye, A. et al. (30 December 2015). \"A Data-Oriented Security Architecture Method and System - CN Patent 201511025657\".<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=A+Data-Oriented+Security+Architecture+Method+and+System+-+CN+Patent+201511025657&rft.atitle=&rft.aulast=Miao%2C+F.%3B+Yang%2C+W.%3B+Ye%2C+A.+et+al.&rft.au=Miao%2C+F.%3B+Yang%2C+W.%3B+Ye%2C+A.+et+al.&rft.date=30+December+2015&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-7\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-7\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Zheng, Xuying; Miao, Fang; Chakpitak, Nopasit; Huang, Jin; Wang, Jing (17 December 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/dl.acm.org\/doi\/10.1145\/3510309.3510322\" target=\"_blank\">\"Discussion of University Chemistry Laboratory Management Using DOSA Platform and Safety Education Based on Blockchain\"<\/a> (in en). <i>2021 4th International Conference on Education Technology Management<\/i> (Tokyo Japan: ACM): 79\u201385. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1145%2F3510309.3510322\" target=\"_blank\">10.1145\/3510309.3510322<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-4503-8580-0<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/dl.acm.org\/doi\/10.1145\/3510309.3510322\" target=\"_blank\">https:\/\/dl.acm.org\/doi\/10.1145\/3510309.3510322<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Discussion+of+University+Chemistry+Laboratory+Management+Using+DOSA+Platform+and+Safety+Education+Based+on+Blockchain&rft.jtitle=2021+4th+International+Conference+on+Education+Technology+Management&rft.aulast=Zheng&rft.aufirst=Xuying&rft.au=Zheng%2C%26%2332%3BXuying&rft.au=Miao%2C%26%2332%3BFang&rft.au=Chakpitak%2C%26%2332%3BNopasit&rft.au=Huang%2C%26%2332%3BJin&rft.au=Wang%2C%26%2332%3BJing&rft.date=17+December+2021&rft.pages=79%E2%80%9385&rft.place=Tokyo+Japan&rft.pub=ACM&rft_id=info:doi\/10.1145%2F3510309.3510322&rft.isbn=978-1-4503-8580-0&rft_id=https%3A%2F%2Fdl.acm.org%2Fdoi%2F10.1145%2F3510309.3510322&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-8\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-8\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation\" id=\"rdp-ebb-CITEREFPanpengFangPhaphuangwittayakulRattanadamrongaksorn2021\">Panpeng, Vorada; Fang, Miao; Phaphuangwittayakul, Aniwat; Rattanadamrongaksorn, Tanarat (2021), Yang, Xin-She; Sherratt, Simon; Dey, Nilanjan <i>et al.<\/i>., eds., <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/978-981-15-5859-7_51\" target=\"_blank\">\"Preliminary Study and Implementation of Chiang Mai Tourism Platform Based on DOSA\"<\/a> (in en), <i>Proceedings of Fifth International Congress on Information and Communication Technology<\/i> (Singapore: Springer Singapore) <b>1184<\/b>: 511\u2013521, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2F978-981-15-5859-7_51\" target=\"_blank\">10.1007\/978-981-15-5859-7_51<\/a>, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-981-15-5858-0<span class=\"printonly\">, <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/978-981-15-5859-7_51\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/978-981-15-5859-7_51<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 2023-07-31<\/span><\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Preliminary+Study+and+Implementation+of+Chiang+Mai+Tourism+Platform+Based+on+DOSA&rft.jtitle=Proceedings+of+Fifth+International+Congress+on+Information+and+Communication+Technology&rft.aulast=Panpeng&rft.aufirst=Vorada&rft.au=Panpeng%2C%26%2332%3BVorada&rft.au=Fang%2C%26%2332%3BMiao&rft.au=Phaphuangwittayakul%2C%26%2332%3BAniwat&rft.au=Rattanadamrongaksorn%2C%26%2332%3BTanarat&rft.date=2021&rft.volume=1184&rft.pages=511%E2%80%93521&rft.place=Singapore&rft.pub=Springer+Singapore&rft_id=info:doi\/10.1007%2F978-981-15-5859-7_51&rft.isbn=978-981-15-5858-0&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2F978-981-15-5859-7_51&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:0-9\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:0_9-0\">9.0<\/a><\/sup> <sup><a href=\"#cite_ref-:0_9-1\">9.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation web\">Miao, F.; Yang, W.; Li, Y. et al. (21 January 2016). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/patentscope.wipo.int\/search\/en\/detail.jsf?docId=WO2016008335\" target=\"_blank\">\"WO2016008335 - DATA-ORIENTED INFORMATION TECHNOLOGY SYSTEM\"<\/a>. <i>WIPO<\/i><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/patentscope.wipo.int\/search\/en\/detail.jsf?docId=WO2016008335\" target=\"_blank\">https:\/\/patentscope.wipo.int\/search\/en\/detail.jsf?docId=WO2016008335<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=WO2016008335+-+DATA-ORIENTED+INFORMATION+TECHNOLOGY+SYSTEM&rft.atitle=WIPO&rft.aulast=Miao%2C+F.%3B+Yang%2C+W.%3B+Li%2C+Y.+et+al.&rft.au=Miao%2C+F.%3B+Yang%2C+W.%3B+Li%2C+Y.+et+al.&rft.date=21+January+2016&rft_id=https%3A%2F%2Fpatentscope.wipo.int%2Fsearch%2Fen%2Fdetail.jsf%3FdocId%3DWO2016008335&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-10\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-10\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Miao, F. (2016). \"Preliminary Study on Data-oriented Security Architecture\". <i>ZTE Technology Journal<\/i> <b>1<\/b>: 19\u201322.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Preliminary+Study+on+Data-oriented+Security+Architecture&rft.jtitle=ZTE+Technology+Journal&rft.aulast=Miao%2C+F.&rft.au=Miao%2C+F.&rft.date=2016&rft.volume=1&rft.pages=19%E2%80%9322&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-11\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-11\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Conti, Thomas J. (1 December 1992). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/016974399280066D\" target=\"_blank\">\"LIMS and quality audits of a quality control laboratory\"<\/a> (in en). <i>Chemometrics and Intelligent Laboratory Systems<\/i> <b>17<\/b> (3): 295\u2013300. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2F0169-7439%2892%2980066-D\" target=\"_blank\">10.1016\/0169-7439(92)80066-D<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/016974399280066D\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/016974399280066D<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=LIMS+and+quality+audits+of+a+quality+control+laboratory&rft.jtitle=Chemometrics+and+Intelligent+Laboratory+Systems&rft.aulast=Conti&rft.aufirst=Thomas+J.&rft.au=Conti%2C%26%2332%3BThomas+J.&rft.date=1+December+1992&rft.volume=17&rft.issue=3&rft.pages=295%E2%80%93300&rft_id=info:doi\/10.1016%2F0169-7439%2892%2980066-D&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2F016974399280066D&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-12\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-12\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Xu, Yunbi; Crouch, Jonathan H. (1 March 2008). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.2135\/cropsci2007.04.0191\" target=\"_blank\">\"Marker\u2010Assisted Selection in Plant Breeding: From Publications to Practice\"<\/a> (in en). <i>Crop Science<\/i> <b>48<\/b> (2): 391\u2013407. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.2135%2Fcropsci2007.04.0191\" target=\"_blank\">10.2135\/cropsci2007.04.0191<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0011-183X\" target=\"_blank\">0011-183X<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.2135\/cropsci2007.04.0191\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.2135\/cropsci2007.04.0191<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Marker%E2%80%90Assisted+Selection+in+Plant+Breeding%3A+From+Publications+to+Practice&rft.jtitle=Crop+Science&rft.aulast=Xu&rft.aufirst=Yunbi&rft.au=Xu%2C%26%2332%3BYunbi&rft.au=Crouch%2C%26%2332%3BJonathan+H.&rft.date=1+March+2008&rft.volume=48&rft.issue=2&rft.pages=391%E2%80%93407&rft_id=info:doi\/10.2135%2Fcropsci2007.04.0191&rft.issn=0011-183X&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.2135%2Fcropsci2007.04.0191&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-13\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-13\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Jayashree, B; Reddy, Praveen T; Leeladevi, Y; Crouch, Jonathan H; Mahalakshmi, V; Buhariwalla, Hutokshi K; Eshwar, Ke; Mace, Emma <i>et al.<\/i> (1 December 2006). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/1471-2105-7-383\" target=\"_blank\">\"Laboratory Information Management Software for genotyping workflows: applications in high throughput crop genotyping\"<\/a> (in en). <i>BMC Bioinformatics<\/i> <b>7<\/b> (1): 383. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1186%2F1471-2105-7-383\" target=\"_blank\">10.1186\/1471-2105-7-383<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1471-2105\" target=\"_blank\">1471-2105<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC1559653\/\" target=\"_blank\">PMC1559653<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/16914063\" target=\"_blank\">16914063<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/1471-2105-7-383\" target=\"_blank\">https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/1471-2105-7-383<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Laboratory+Information+Management+Software+for+genotyping+workflows%3A+applications+in+high+throughput+crop+genotyping&rft.jtitle=BMC+Bioinformatics&rft.aulast=Jayashree&rft.aufirst=B&rft.au=Jayashree%2C%26%2332%3BB&rft.au=Reddy%2C%26%2332%3BPraveen+T&rft.au=Leeladevi%2C%26%2332%3BY&rft.au=Crouch%2C%26%2332%3BJonathan+H&rft.au=Mahalakshmi%2C%26%2332%3BV&rft.au=Buhariwalla%2C%26%2332%3BHutokshi+K&rft.au=Eshwar%2C%26%2332%3BKe&rft.au=Mace%2C%26%2332%3BEmma&rft.au=Folksterma%2C%26%2332%3BRolf&rft.date=1+December+2006&rft.volume=7&rft.issue=1&rft.pages=383&rft_id=info:doi\/10.1186%2F1471-2105-7-383&rft.issn=1471-2105&rft_id=info:pmc\/PMC1559653&rft_id=info:pmid\/16914063&rft_id=https%3A%2F%2Fbmcbioinformatics.biomedcentral.com%2Farticles%2F10.1186%2F1471-2105-7-383&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-14\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-14\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Chaoqun, Liu; Lanlan, Cai (27 March 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9417148\/\" target=\"_blank\">\"Interactive Laboratory Information Management System Based on WeChat\"<\/a>. <i>2021 7th International Conference on Information Management (ICIM)<\/i> (London, United Kingdom: IEEE): 66\u201370. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FICIM52229.2021.9417148\" target=\"_blank\">10.1109\/ICIM52229.2021.9417148<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-6654-4380-7<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9417148\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/9417148\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Interactive+Laboratory+Information+Management+System+Based+on+WeChat&rft.jtitle=2021+7th+International+Conference+on+Information+Management+%28ICIM%29&rft.aulast=Chaoqun&rft.aufirst=Liu&rft.au=Chaoqun%2C%26%2332%3BLiu&rft.au=Lanlan%2C%26%2332%3BCai&rft.date=27+March+2021&rft.pages=66%E2%80%9370&rft.place=London%2C+United+Kingdom&rft.pub=IEEE&rft_id=info:doi\/10.1109%2FICIM52229.2021.9417148&rft.isbn=978-1-6654-4380-7&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F9417148%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-15\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-15\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Sun, Dingzhong; Wu, Linhuan; Fan, Guomei (1 June 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2588933821000042\" target=\"_blank\">\"Laboratory information management system for biosafety laboratory: Safety and efficiency\"<\/a> (in en). <i>Journal of Biosafety and Biosecurity<\/i> <b>3<\/b> (1): 28\u201334. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.jobb.2021.03.001\" target=\"_blank\">10.1016\/j.jobb.2021.03.001<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2588933821000042\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2588933821000042<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Laboratory+information+management+system+for+biosafety+laboratory%3A+Safety+and+efficiency&rft.jtitle=Journal+of+Biosafety+and+Biosecurity&rft.aulast=Sun&rft.aufirst=Dingzhong&rft.au=Sun%2C%26%2332%3BDingzhong&rft.au=Wu%2C%26%2332%3BLinhuan&rft.au=Fan%2C%26%2332%3BGuomei&rft.date=1+June+2021&rft.volume=3&rft.issue=1&rft.pages=28%E2%80%9334&rft_id=info:doi\/10.1016%2Fj.jobb.2021.03.001&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2588933821000042&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-16\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-16\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">D\u2019Amico, Gaspare; Szopik-Depczy\u0144ska, Katarzyna; Beltramo, Riccardo; D\u2019Adamo, Idiano; Ioppolo, Giuseppe (2 January 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.mdpi.com\/2071-1050\/14\/1\/466\" target=\"_blank\">\"Smart and Sustainable Bioeconomy Platform: A New Approach towards Sustainability\"<\/a> (in en). <i>Sustainability<\/i> <b>14<\/b> (1): 466. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3390%2Fsu14010466\" target=\"_blank\">10.3390\/su14010466<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2071-1050\" target=\"_blank\">2071-1050<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.mdpi.com\/2071-1050\/14\/1\/466\" target=\"_blank\">https:\/\/www.mdpi.com\/2071-1050\/14\/1\/466<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Smart+and+Sustainable+Bioeconomy+Platform%3A+A+New+Approach+towards+Sustainability&rft.jtitle=Sustainability&rft.aulast=D%E2%80%99Amico&rft.aufirst=Gaspare&rft.au=D%E2%80%99Amico%2C%26%2332%3BGaspare&rft.au=Szopik-Depczy%C5%84ska%2C%26%2332%3BKatarzyna&rft.au=Beltramo%2C%26%2332%3BRiccardo&rft.au=D%E2%80%99Adamo%2C%26%2332%3BIdiano&rft.au=Ioppolo%2C%26%2332%3BGiuseppe&rft.date=2+January+2022&rft.volume=14&rft.issue=1&rft.pages=466&rft_id=info:doi\/10.3390%2Fsu14010466&rft.issn=2071-1050&rft_id=https%3A%2F%2Fwww.mdpi.com%2F2071-1050%2F14%2F1%2F466&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-17\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-17\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Kou, Yunpeng; Liu, Yuhang; Li, Guoqing; Hou, Jie; Luan, Liming; Wang, Hao (22 March 2022). Guo, Lijie. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.hindawi.com\/journals\/ace\/2022\/1876435\/\" target=\"_blank\">\"Design and Implementation of an Integrated Management System for Backfill Experimental Data\"<\/a> (in en). <i>Advances in Civil Engineering<\/i> <b>2022<\/b>: 1\u20139. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1155%2F2022%2F1876435\" target=\"_blank\">10.1155\/2022\/1876435<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1687-8094\" target=\"_blank\">1687-8094<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.hindawi.com\/journals\/ace\/2022\/1876435\/\" target=\"_blank\">https:\/\/www.hindawi.com\/journals\/ace\/2022\/1876435\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Design+and+Implementation+of+an+Integrated+Management+System+for+Backfill+Experimental+Data&rft.jtitle=Advances+in+Civil+Engineering&rft.aulast=Kou&rft.aufirst=Yunpeng&rft.au=Kou%2C%26%2332%3BYunpeng&rft.au=Liu%2C%26%2332%3BYuhang&rft.au=Li%2C%26%2332%3BGuoqing&rft.au=Hou%2C%26%2332%3BJie&rft.au=Luan%2C%26%2332%3BLiming&rft.au=Wang%2C%26%2332%3BHao&rft.date=22+March+2022&rft.volume=2022&rft.pages=1%E2%80%939&rft_id=info:doi\/10.1155%2F2022%2F1876435&rft.issn=1687-8094&rft_id=https%3A%2F%2Fwww.hindawi.com%2Fjournals%2Face%2F2022%2F1876435%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-18\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-18\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Magzumova, Bibigul (1 November 2016). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onepetro.org\/SPECTCE\/proceedings\/16CTCE\/All-16CTCE\/Astana,%20Kazakhstan\/185619\" target=\"_blank\">\"Efficient Data Integrity Through LIMS-Instrument Interface\"<\/a>. <i>All Days<\/i> (Astana, Kazakhstan: SPE): SPE\u2013182564\u2013MS. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.2118%2F182564-MS\" target=\"_blank\">10.2118\/182564-MS<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onepetro.org\/SPECTCE\/proceedings\/16CTCE\/All-16CTCE\/Astana,%20Kazakhstan\/185619\" target=\"_blank\">https:\/\/onepetro.org\/SPECTCE\/proceedings\/16CTCE\/All-16CTCE\/Astana,%20Kazakhstan\/185619<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Efficient+Data+Integrity+Through+LIMS-Instrument+Interface&rft.jtitle=All+Days&rft.aulast=Magzumova&rft.aufirst=Bibigul&rft.au=Magzumova%2C%26%2332%3BBibigul&rft.date=1+November+2016&rft.pages=SPE%E2%80%93182564%E2%80%93MS&rft.place=Astana%2C+Kazakhstan&rft.pub=SPE&rft_id=info:doi\/10.2118%2F182564-MS&rft_id=https%3A%2F%2Fonepetro.org%2FSPECTCE%2Fproceedings%2F16CTCE%2FAll-16CTCE%2FAstana%2C%2520Kazakhstan%2F185619&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-19\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-19\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">McIntosh, Leslie D.; Sharma, Mukesh K.; Mulvihill, David; Gupta, Snehil; Juehne, Anthony; George, Bijoy; Khot, Suhas B.; Kaushal, Atul <i>et al.<\/i> (1 October 2015). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1532046415001884\" target=\"_blank\">\"caTissue Suite to OpenSpecimen: Developing an extensible, open source, web-based biobanking management system\"<\/a> (in en). <i>Journal of Biomedical Informatics<\/i> <b>57<\/b>: 456\u2013464. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.jbi.2015.08.020\" target=\"_blank\">10.1016\/j.jbi.2015.08.020<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC4772150\/\" target=\"_blank\">PMC4772150<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/26325296\" target=\"_blank\">26325296<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1532046415001884\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1532046415001884<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=caTissue+Suite+to+OpenSpecimen%3A+Developing+an+extensible%2C+open+source%2C+web-based+biobanking+management+system&rft.jtitle=Journal+of+Biomedical+Informatics&rft.aulast=McIntosh&rft.aufirst=Leslie+D.&rft.au=McIntosh%2C%26%2332%3BLeslie+D.&rft.au=Sharma%2C%26%2332%3BMukesh+K.&rft.au=Mulvihill%2C%26%2332%3BDavid&rft.au=Gupta%2C%26%2332%3BSnehil&rft.au=Juehne%2C%26%2332%3BAnthony&rft.au=George%2C%26%2332%3BBijoy&rft.au=Khot%2C%26%2332%3BSuhas+B.&rft.au=Kaushal%2C%26%2332%3BAtul&rft.au=Watson%2C%26%2332%3BMark+A.&rft.date=1+October+2015&rft.volume=57&rft.pages=456%E2%80%93464&rft_id=info:doi\/10.1016%2Fj.jbi.2015.08.020&rft_id=info:pmc\/PMC4772150&rft_id=info:pmid\/26325296&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS1532046415001884&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-20\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-20\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Zhou, Xin; Tang, Xiaofei (1 August 2011). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/6021216\/\" target=\"_blank\">\"Research and implementation of RSA algorithm for encryption and decryption\"<\/a>. <i>Proceedings of 2011 6th International Forum on Strategic Technology<\/i> <b>2<\/b>: 1118\u20131121. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FIFOST.2011.6021216\" target=\"_blank\">10.1109\/IFOST.2011.6021216<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/6021216\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/6021216\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Research+and+implementation+of+RSA+algorithm+for+encryption+and+decryption&rft.jtitle=Proceedings+of+2011+6th+International+Forum+on+Strategic+Technology&rft.aulast=Zhou&rft.aufirst=Xin&rft.au=Zhou%2C%26%2332%3BXin&rft.au=Tang%2C%26%2332%3BXiaofei&rft.date=1+August+2011&rft.volume=2&rft.pages=1118%E2%80%931121&rft_id=info:doi\/10.1109%2FIFOST.2011.6021216&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F6021216%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-21\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-21\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Zheng, Xin; Xu, Chongyao; Hu, Xianghong; Zhang, Yun; Xiong, Xiaoming (1 October 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/8823969\/\" target=\"_blank\">\"The Software\/Hardware Co-Design and Implementation of SM2\/3\/4 Encryption\/Decryption and Digital Signature System\"<\/a>. <i>IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems<\/i> <b>39<\/b> (10): 2055\u20132066. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FTCAD.2019.2939330\" target=\"_blank\">10.1109\/TCAD.2019.2939330<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0278-0070\" target=\"_blank\">0278-0070<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/8823969\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/8823969\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+Software%2FHardware+Co-Design+and+Implementation+of+SM2%2F3%2F4+Encryption%2FDecryption+and+Digital+Signature+System&rft.jtitle=IEEE+Transactions+on+Computer-Aided+Design+of+Integrated+Circuits+and+Systems&rft.aulast=Zheng&rft.aufirst=Xin&rft.au=Zheng%2C%26%2332%3BXin&rft.au=Xu%2C%26%2332%3BChongyao&rft.au=Hu%2C%26%2332%3BXianghong&rft.au=Zhang%2C%26%2332%3BYun&rft.au=Xiong%2C%26%2332%3BXiaoming&rft.date=1+October+2020&rft.volume=39&rft.issue=10&rft.pages=2055%E2%80%932066&rft_id=info:doi\/10.1109%2FTCAD.2019.2939330&rft.issn=0278-0070&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F8823969%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-22\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-22\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.chinesestandard.net\/PDF.aspx\/GBT32918.3-2016\" target=\"_blank\">\"GB\/T 32918.3-2016 - Information security technology -- Public key cryptographic algorithm SM2 based on elliptic curves -- Part 3: Key exchange protocol\"<\/a>. <i>ChineseStandard.net<\/i>. 2016<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.chinesestandard.net\/PDF.aspx\/GBT32918.3-2016\" target=\"_blank\">https:\/\/www.chinesestandard.net\/PDF.aspx\/GBT32918.3-2016<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=GB%2FT+32918.3-2016+-+Information+security+technology+--+Public+key+cryptographic+algorithm+SM2+based+on+elliptic+curves+--+Part+3%3A+Key+exchange+protocol&rft.atitle=ChineseStandard.net&rft.date=2016&rft_id=https%3A%2F%2Fwww.chinesestandard.net%2FPDF.aspx%2FGBT32918.3-2016&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-23\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-23\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Saidi, Hafida; Labraoui, Nabila; Ari, Ado Adamou Abba; Maglaras, Leandros A.; Emati, Joel Herve Mboussam (2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9895264\/\" target=\"_blank\">\"DSMAC: Privacy-Aware Decentralized Self-Management of Data Access Control Based on Blockchain for Health Data\"<\/a>. <i>IEEE Access<\/i> <b>10<\/b>: 101011\u2013101028. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FACCESS.2022.3207803\" target=\"_blank\">10.1109\/ACCESS.2022.3207803<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2169-3536\" target=\"_blank\">2169-3536<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9895264\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/9895264\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=DSMAC%3A+Privacy-Aware+Decentralized+Self-Management+of+Data+Access+Control+Based+on+Blockchain+for+Health+Data&rft.jtitle=IEEE+Access&rft.aulast=Saidi&rft.aufirst=Hafida&rft.au=Saidi%2C%26%2332%3BHafida&rft.au=Labraoui%2C%26%2332%3BNabila&rft.au=Ari%2C%26%2332%3BAdo+Adamou+Abba&rft.au=Maglaras%2C%26%2332%3BLeandros+A.&rft.au=Emati%2C%26%2332%3BJoel+Herve+Mboussam&rft.date=2022&rft.volume=10&rft.pages=101011%E2%80%93101028&rft_id=info:doi\/10.1109%2FACCESS.2022.3207803&rft.issn=2169-3536&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F9895264%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-24\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-24\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Aranha, Diego; Dahab, Ricardo; L\u00f3pez, Julio; Oliveira, Leonardo (1 May 2010). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.aimsciences.org\/journals\/displayArticles.jsp?paperID=5168\" target=\"_blank\">\"Efficient implementation of elliptic curve cryptography in wireless sensors\"<\/a> (in en). <i>Advances in Mathematics of Communications<\/i> <b>4<\/b> (2): 169\u2013187. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3934%2Famc.2010.4.169\" target=\"_blank\">10.3934\/amc.2010.4.169<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1930-5346\" target=\"_blank\">1930-5346<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.aimsciences.org\/journals\/displayArticles.jsp?paperID=5168\" target=\"_blank\">http:\/\/www.aimsciences.org\/journals\/displayArticles.jsp?paperID=5168<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Efficient+implementation+of+elliptic+curve+cryptography+in+wireless+sensors&rft.jtitle=Advances+in+Mathematics+of+Communications&rft.aulast=Aranha&rft.aufirst=Diego&rft.au=Aranha%2C%26%2332%3BDiego&rft.au=Dahab%2C%26%2332%3BRicardo&rft.au=L%C3%B3pez%2C%26%2332%3BJulio&rft.au=Oliveira%2C%26%2332%3BLeonardo&rft.date=1+May+2010&rft.volume=4&rft.issue=2&rft.pages=169%E2%80%93187&rft_id=info:doi\/10.3934%2Famc.2010.4.169&rft.issn=1930-5346&rft_id=http%3A%2F%2Fwww.aimsciences.org%2Fjournals%2FdisplayArticles.jsp%3FpaperID%3D5168&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-25\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-25\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Jintcharadze, Elza; Iavich, Maksim (1 September 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9224901\/\" target=\"_blank\">\"Hybrid Implementation of Twofish, AES, ElGamal and RSA Cryptosystems\"<\/a>. <i>2020 IEEE East-West Design & Test Symposium (EWDTS)<\/i> (Varna, Bulgaria: IEEE): 1\u20135. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FEWDTS50664.2020.9224901\" target=\"_blank\">10.1109\/EWDTS50664.2020.9224901<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-7281-9899-6<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9224901\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/9224901\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Hybrid+Implementation+of+Twofish%2C+AES%2C+ElGamal+and+RSA+Cryptosystems&rft.jtitle=2020+IEEE+East-West+Design+%26+Test+Symposium+%28EWDTS%29&rft.aulast=Jintcharadze&rft.aufirst=Elza&rft.au=Jintcharadze%2C%26%2332%3BElza&rft.au=Iavich%2C%26%2332%3BMaksim&rft.date=1+September+2020&rft.pages=1%E2%80%935&rft.place=Varna%2C+Bulgaria&rft.pub=IEEE&rft_id=info:doi\/10.1109%2FEWDTS50664.2020.9224901&rft.isbn=978-1-7281-9899-6&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F9224901%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-26\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-26\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Abdolkhani, Robab; Gray, Kathleen; Borda, Ann; DeSouza, Ruth (1 December 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/academic.oup.com\/jamiaopen\/article\/2\/4\/471\/5572201\" target=\"_blank\">\"Patient-generated health data management and quality challenges in remote patient monitoring\"<\/a> (in en). <i>JAMIA Open<\/i> <b>2<\/b> (4): 471\u2013478. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1093%2Fjamiaopen%2Fooz036\" target=\"_blank\">10.1093\/jamiaopen\/ooz036<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2574-2531\" target=\"_blank\">2574-2531<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC6993998\/\" target=\"_blank\">PMC6993998<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/32025644\" target=\"_blank\">32025644<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/academic.oup.com\/jamiaopen\/article\/2\/4\/471\/5572201\" target=\"_blank\">https:\/\/academic.oup.com\/jamiaopen\/article\/2\/4\/471\/5572201<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Patient-generated+health+data+management+and+quality+challenges+in+remote+patient+monitoring&rft.jtitle=JAMIA+Open&rft.aulast=Abdolkhani&rft.aufirst=Robab&rft.au=Abdolkhani%2C%26%2332%3BRobab&rft.au=Gray%2C%26%2332%3BKathleen&rft.au=Borda%2C%26%2332%3BAnn&rft.au=DeSouza%2C%26%2332%3BRuth&rft.date=1+December+2019&rft.volume=2&rft.issue=4&rft.pages=471%E2%80%93478&rft_id=info:doi\/10.1093%2Fjamiaopen%2Fooz036&rft.issn=2574-2531&rft_id=info:pmc\/PMC6993998&rft_id=info:pmid\/32025644&rft_id=https%3A%2F%2Facademic.oup.com%2Fjamiaopen%2Farticle%2F2%2F4%2F471%2F5572201&rfr_id=info:sid\/en.wikipedia.org:Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<\/ol><\/div><\/div>\n<h2><span class=\"mw-headline\" id=\"Notes\">Notes<\/span><\/h2>\n<p>This presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added.\n<\/p>\n<!-- \nNewPP limit report\nCached time: 20231215110721\nCache expiry: 86400\nDynamic content: false\nComplications: []\nCPU time usage: 0.959 seconds\nReal time usage: 1.484 seconds\nPreprocessor visited node count: 27357\/1000000\nPost\u2010expand include size: 229768\/2097152 bytes\nTemplate argument size: 73425\/2097152 bytes\nHighest expansion depth: 25\/40\nExpensive parser function count: 0\/100\nUnstrip recursion depth: 0\/20\nUnstrip post\u2010expand size: 61642\/5000000 bytes\n-->\n<!--\nTransclusion expansion time report (%,ms,calls,template)\n100.00% 632.015 1 -total\n 86.15% 544.510 1 Template:Reflist\n 66.04% 417.409 26 Template:Citation\/core\n 64.05% 404.829 22 Template:Cite_journal\n 12.01% 75.926 25 Template:Date\n 9.71% 61.392 47 Template:Citation\/identifier\n 8.48% 53.622 1 Template:Infobox_journal_article\n 7.30% 46.124 1 Template:Infobox\n 6.66% 42.065 3 Template:Cite_web\n 4.05% 25.601 94 Template:Hide_in_print\n-->\n\n<!-- Saved in parser cache with key limswiki:pcache:idhash:14330-0!canonical and timestamp 20231215110720 and revision id 52721. Serialized with JSON.\n -->\n<\/div><\/div><div class=\"printfooter\">Source: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.limswiki.org\/index.php\/Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture\">https:\/\/www.limswiki.org\/index.php\/Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture<\/a><\/div>\n<!-- end content --><div class=\"visualClear\"><\/div><\/div><\/div><div class=\"visualClear\"><\/div><\/div><!-- end of the left (by default at least) column --><div class=\"visualClear\"><\/div><\/div>\n\n\n\n<\/body>","ab540fa1cc32f92043343fd9cf67ec0d_images":["https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/5\/50\/Fig1_Zheng_Electronics23_12-8.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/0\/06\/Fig2_Zheng_Electronics23_12-8.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/6\/67\/Fig3_Zheng_Electronics23_12-8.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/7\/76\/Fig4_Zheng_Electronics23_12-8.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/d\/d9\/Fig5_Zheng_Electronics23_12-8.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/f\/f0\/Fig6_Zheng_Electronics23_12-8.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/3\/3f\/Fig7_Zheng_Electronics23_12-8.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/4\/4a\/Fig8_Zheng_Electronics23_12-8.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/3\/38\/Fig10_Zheng_Electronics23_12-8.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/d\/db\/Fig9_Zheng_Electronics23_12-8.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/6\/61\/Fig11_Zheng_Electronics23_12-8.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/b\/b2\/Fig12_Zheng_Electronics23_12-8.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/f\/f3\/Fig13_Zheng_Electronics23_12-8.png"],"ab540fa1cc32f92043343fd9cf67ec0d_timestamp":1702682173,"9e1d6433c962801d5a75756c1046599c_type":"article","9e1d6433c962801d5a75756c1046599c_title":"NIMS-OS: An automation software to implement a closed loop between artificial intelligence and robotic experiments in materials science (Tamura et al. 2023)","9e1d6433c962801d5a75756c1046599c_url":"https:\/\/www.limswiki.org\/index.php\/Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science","9e1d6433c962801d5a75756c1046599c_plaintext":"\n\nJournal:NIMS-OS: An automation software to implement a closed loop between artificial intelligence and robotic experiments in materials scienceFrom LIMSWikiJump to navigationJump to searchFull article title\n \nNIMS-OS: An automation software to implement a closed loop between artificial intelligence and robotic experiments in materials scienceJournal\n \nScience and Technology of Advanced Materials: MethodsAuthor(s)\n \nTamura, Ryo; Tsuda, Koji; Matsuda, ShoichiAuthor affiliation(s)\n \nThe University of Tokyo, National Institute for Materials SciencePrimary contact\n \nEmail: tamura dot ryo at nims dot go dot jpYear published\n \n2023Volume and issue\n \n3(1)Article #\n \n2232297DOI\n \n10.1080\/27660400.2023.2232297ISSN\n \n2766-0400Distribution license\n \nCreative Commons Attribution 4.0 InternationalWebsite\n \nhttps:\/\/www.tandfonline.com\/doi\/full\/10.1080\/27660400.2023.2232297Download\n \nhttps:\/\/www.tandfonline.com\/doi\/pdf\/10.1080\/27660400.2023.2232297 (PDF)\n\n\n\n\n \n This article contains rendered mathematical formulae. You may require the TeX All the Things plugin for Chrome or the Native MathML add-on and fonts for Firefox if they don't render properly for you. \n\n\nContents \n\n1 Abstract \n2 Introduction \n3 Preparation of candidates for experimental conditions \n4 Modules in NIMS-OS \n\n4.1 AI algorithms \n4.2 Bayesian optimization: PHYSBO \n\n4.2.1 Boundless objective-free exploration: BLOX \n\n\n4.3 Phase diagram construction: PDC \n\n4.3.1 Random exploration: RE \n\n\n4.4 Robotic experiments \n\n4.4.1 Standard module for robotic experiments: STAN \n4.4.2 NIMS automated robotic electrochemical experiments (NAREE) system: NAREE \n\n\n\n\n5 Usage of the NIMS-OS Python version \n\n5.1 Install \n5.2 Basic usage \n\n5.2.1 Assignment of parameters and candidates file \n5.2.2 Execution of AI \n5.2.3 Preparation of input files for robotic experiments and execution of experiments \n5.2.4 Analysis of output files from experiments and update of candidates file \n\n\n5.3 Visualization of the results \n\n\n6 Usage of the NIMS-OS GUI version \n7 Application \n8 Conclusion \n9 Supplemental material \n10 Abbreviations, acronyms, and initialisms \n11 Acknowledgements \n\n11.1 Funding \n11.2 Correction statement \n11.3 Supplementary data \n11.4 Conflict of interest \n\n\n12 References \n13 Notes \n\n\n\nAbstract \nNIMS-OS (NIMS Orchestration System) is a Python library created to realize a closed loop of robotic experiments and artificial intelligence (AI) without human intervention for automated materials exploration. It uses various combinations of modules to operate autonomously. Each module acts as an AI for materials exploration or a controller for a robotic experiments. As AI techniques, Optimization Tools for PHYSics Based on Bayesian Optimization (PHYSBO), BoundLess Objective-free eXploration (BLOX), phase diagram construction (PDC), and random exploration (RE) methods can be used. Moreover, a system called NIMS Automated Robotic Electrochemical Experiments (NAREE) is available as a set of robotic experimental equipment. Visualization tools for the results are also included, which allows users to check the optimization results in real time. Newly created modules for AI and robotic experiments can be added easily to extend the functionality of the system. In addition, we developed a graphical user interface (GUI)-driven application to control NIMS-OS. To demonstrate the operation of NIMS-OS, we consider an automated exploration for new electrolytes. NIMS-OS is available at https:\/\/github.com\/nimsos-dev\/nimsos.\nKeywords: NIMS-OS, robotic experiments, artificial intelligence, electrochemistry, materials informatics\n\nIntroduction \nThe integration of robotic experiments and artificial intelligence (AI) is essential to realize automated materials exploration. If an AI system can take on some information tasks conventionally performed by human researchers, robotic systems can then execute the required physical tasks and experiments for materials exploration can proceed automatically. Such a platform may be expected to discover many novel materials and lead to substantial innovation in materials science. In recent years, significant progress has been made in the development of AI techniques and robotic devices suitable for materials exploration.\nSince the launch of the Materials Genome Initiative[1], AI techniques have been actively used for materials exploration.[2][3][4] In general, materials exploration can be regarded as the problem of finding optimal materials from among a materials search space. The elements to be used in the search space must be configured, along with its composition range, process parameter range, and so forth. To solve this problem, black-box optimization methods are useful[5], and various methods have been developed and applied to fit various needs. Bayesian optimization (BO) is among the most frequently used methods in materials science.[6][7][8] In this method, promising materials can be selected in the materials search space using the predictions of their properties and the uncertainty of these predictions evaluated by Gaussian process regression. Using BO, various real materials, such as Li-ion conductive materials[9], multilayered metamaterials[10], halide perovskite[11], superalloys[12], and electrolytes[13] have been explored. BO is also used for the automated analysis of materials.[14][15] In addition, many methods have been proposed for black-box optimization in materials exploration, such as genetic algorithms[16][17], Monte Carlo tree search[18], rare event sampling[19], and algorithms using an Ising machine.[20][21][22] In the future, many more innovative methods are expected to be developed.\nRobotic experiments have progressed to realize laboratory automation of chemical analysis and high-throughput screening in the field of biology.[23][24][25][26] Various types of automated analyzers and pipetting devices have been developed, and robotic arms have been used as a transport system to connect these systems. Moreover, robotic technology has been used to explore novel materials, such as thin-film materials[27][28], battery electrolytes[13][29], and photocatalysts.[30] These studies used BO to automate the proposal of promising experimental conditions. This enables a closed loop of robotic experiments and AI that can perform automated materials exploration without human intervention. This approach involves some key advantages, such as the ability to generate materials data of uniform quality and the absence of human error. In contrast, at present, robotics systems are limited in their ability to perform complex material synthesis tasks that require the skills of experts. Thus, further innovation in robotic devices will be important.\nIn addition to AI and robotic technologies, the control systems and software used to interlink them are also an important element to realize a closed loop without human intervention. Generally, different AI algorithms should be used depending on the motivation of a materials exploration task. Furthermore, the procedure to control the devices should depends on the nature and characteristics of the robotic systems used. Therefore, control software has thus far been developed on a case-by-case basis for different AI algorithms and robotic systems.\nIn this study, we developed NIMS-OS (NIMS Orchestration System) to realize a closed loop between AI models and robotic experiments, with the aim of establishing a generic control software system. Although this software was written in the Python programming language, we also developed a graphical user interface (GUI)-driven version to improve operability after installation. NIMS-OS treats each AI algorithm and each robotic system as separate modules (see Figure 1). This enables the implementation of a closed loop with any combination of these modules. If modules for new AI algorithms or robotic systems are prepared, new closed-loop systems can be easily controlled via NIMS-OS. One of the advantages of developing such generic control software is the establishment of technical standards for automated materials exploration. For AI algorithms, we determined standard formats for the input and output. Algorithms created according to the standard format can be immediately tested using any currently available robotic system. Specifically, we developed a standard format in which all the experimental conditions to be explored are listed in advance, and the appropriate experimental conditions that have not yet been tested are selected from the list by AI algorithms. The advantage of this approach is that it enables automated materials exploration utilizing materials databases. When utilizing materials databases, the compositional and structural information needs to be converted into materials descriptors, which serve as the materials search space. However, this search space generated from the materials databases cannot be solely defined by a continuous parameter space, and it requires a selection from the pre-listed descriptors. Of course, optimization of continuous parameters can still be handled approximately by preparing a list of grid points that discretize the continuous parameters. Furthermore, we expect this work to contribute to the development of new AI algorithms for automated materials exploration. For robotic systems, we expect modules developed based on NIMS-OS to increase the commonality of operational procedures, leading to cost reductions as new robotic experimental devices are introduced. Note that ChemOS[31] is similar to NIMS-OS; it was developed as an automation system in the field of chemistry. ChemOS specializes in BO within a defined continuous or discretized parameter space and includes several default modules for various BO methods. On the other hand, NIMS-OS offers the capability to perform automated materials explorations not only within a defined parameter space but also utilizing materials databases. In addition to BO, NIMS-OS also incorporates several default implementations of black-box optimization methods to deal with different motivations in materials explorations.\n\n\n\n\n\n\n\n\n\nFigure 1. Image of the combinations of AI algorithms and robotic systems via NIMS-OS.\n\n\n\nLet us briefly introduce the specifications of NIMS-OS. First, a candidates file listing experimental conditions as a materials search space should be prepared in advance. A closed loop is formed according to the following three steps (see Figure 2):\n\nStep 1: Select promising experimental conditions from the candidates file using an AI model.\nStep 2: Create an input file for the robotic experiments and execute the experiments.\nStep 3: Analyze the output from the experiments and update the candidates file based on the experimental results.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 2. Procedures in NIMS-OS and roles of each Python scripts.\n\n\n\nCurrently, the following AI algorithms are used as modules, which are available for Step 1: (i) Optimization Tools for PHYSics Based on Bayesian Optimization (PHYSBO)[32], (ii) BoundLess Objective-free eXploration (BLOX)[33], and (iii) phase diagram construction (PDC)[34] methods, with the random exploration (RE) approach able to be selected according to the purpose of materials exploration effort. For Steps 2 and 3, a STANdard module (STAN) is provided for robotic experiments, which enables operation checks even without devices, along with a module for NIMS Automated Robotic Electrochemical Experiments (NAREE).[13][35] We plan to continue developing additional modules for this system.\nThe reminder of this study is organized as follows. The next section describes the preparation of a candidates file storing experimental conditions, followed by an introduction to the available modules for the AI and robotic experiments in NIMS-OS. Then the use of the Python code, and the usage of the GUI version is explained. As a demonstration, the results of an autonomous electrolyte exploration via a closed-loop approach using PHYSBO and NAREE in NIMS-OS are described. Finally, this work concludes with some discussion and suggests some important avenues for further research.\n\nPreparation of candidates for experimental conditions \nA major feature of NIMS-OS is that a data file listing candidate experimental conditions is prepared in advance (we refer to this data file a candidates file). In general, because there are many candidates, conducting experiments in all possible conditions is impractical. Thus, automated materials exploration proceeds by selecting promising experimental conditions from these listed candidates. This makes the closed-loop strategy more generalizable. That is, a variety of exploration motivations and robotic systems can be handled by NIMS-OS.\nThe experimental condition is expressed as a real-valued vector \n \n \n \n \n \n x\n \n \n i\n \n \n ∈\n \n \n R\n \n \n d\n \n \n \n \n {\\displaystyle \\mathbf {x} _{i}\\in \\mathbb {R} ^{d}}\n \n . This condition is prepared with information such as the compositions and structures of materials and the processes required to synthesize them. If the number of candidates for the experimental conditions is N, the dataset for candidates is defined as \n \n \n \n D\n =\n {\n \n \n x\n \n \n i\n \n \n \n }\n \n i\n =\n 1\n ,\n …\n ,\n N\n \n \n \n \n {\\displaystyle D=\\{\\mathbf {x} _{i}\\}_{i=1,\\ldots ,N}}\n \n . The initial candidates file is created by this dataset D. An example of a candidates file with l objective functions is presented in Figure 3. All the candidates of D are written in the first d columns. In this part, there should be no empty spaces. The next l columns are used for the objective function values. In this part, at the initial stage, all cells are empty because experiments have not been performed for all the experimental conditions.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 3. (Top panels) Examples of the candidates files of the initial stage and that after some experiments. Here, an example for the case that N=9 is shown. (Bottom panels) Examples for the list of descriptors depending on the types of search space. If the continuous parameter space is considered, \n \n \n \n D\n =\n {\n \n \n x\n \n \n i\n \n \n \n }\n \n i\n =\n 1\n ,\n …\n ,\n N\n \n \n \n \n {\\displaystyle D=\\{\\mathbf {x} _{i}\\}_{i=1,\\ldots ,N}}\n \n is the discretized parameters. When the combination of materials is the search space, the bit strings where the material used is represented by 1 and the material not used is represented by 0 are prepared in \n \n \n \n D\n =\n {\n \n \n x\n \n \n i\n \n \n \n }\n \n i\n =\n 1\n ,\n …\n ,\n N\n \n \n \n \n {\\displaystyle D=\\{\\mathbf {x} _{i}\\}_{i=1,\\ldots ,N}}\n \n . Furthermore, materials descriptors from compositions obtained by such as magpie[36][37] and fingerprint of molecules obtained by such as RDKit[38] would be used as \n \n \n \n D\n =\n {\n \n \n x\n \n \n i\n \n \n \n }\n \n i\n =\n 1\n ,\n …\n ,\n N\n \n \n \n \n {\\displaystyle D=\\{\\mathbf {x} _{i}\\}_{i=1,\\ldots ,N}}\n \n .\n\n\n\nIn NIMS-OS, some promising conditions are selected from among those listed in the candidates file using AI models (available algorithms are described in the next section). When the values of objective functions are obtained by performing experiments, the objective functions in the candidates file are updated accordingly. That is, when the experiments are completed for M experimental conditions, only results for M conditions are entered at the l columns for the objective functions. Thus, at the next step, the experimental conditions are selected from among \n \n \n \n N\n −\n M\n \n \n {\\displaystyle N-M}\n \n candidates.\n\nModules in NIMS-OS \nIn this section, we introduce the modules included in NIMS-OS for AI algorithms and robotic systems. In the present work, we prepared four and two types of modules as AI algorithms and robotic systems, respectively.\n\nAI algorithms \nTo select promising experimental conditions, three types of AI algorithms are implemented as standard in NIMS-OS. In addition, random exploration can be selected. Each algorithm is briefly explained in this subsection. In the future, more algorithms will be made available.\n\nBayesian optimization: PHYSBO \nBO is an optimization technique using machine learning (ML) prediction. In this method, by using Gaussian process regression, the value of an objective function is predicted when the experimental conditions are input. The next promising experimental conditions are then selected based on the prediction values. Here, because the Gaussian process can evaluate not only the mean value of the prediction but also its variance, an acquisition function defined by mean and variance can be used to make the selection. In NIMS-OS, BO can be performed using the Python package PHYSBO.[32] PHYSBO supports single- and multi-objective optimizations, and multiple proposals are calculated. Note that the number of objective functions is recommended to be no more than three due to excessively large computational time with higher values. In NIMS-OS, Thompson sampling is used to define the acquisition function for rapid calculation. The key point in using PHYSBO is that the exploration is performed to maximize the objective functions. Thus, if a material with the smaller properties is explored, we need to add a negative value to the objective functions.\n\nBoundless objective-free exploration: BLOX \nBLOX is a Python package that performs boundless objective-free exploration. It is based on an algorithm designed to select the next experimental conditions, to perform uniform sampling in the space of the objective functions. For materials science, curious materials can be found using BLOX. Specifically, BLOX trains ML models to predict objective functions from experimental conditions. Experimental conditions that realize uniform sampling in the space of objective functions are found based on the Stein discrepancy evaluated using the prediction results. In NIMS-OS, a modified version of the BLOX algorithm that can propose multiple candidates is implemented. To select multiple candidates, after the experimental condition with the largest Stein discrepancy is selected, another condition is selected when the predicted values of the selected condition are regarded as a correct value. This procedure is iterated, and we obtain multiple proposals. In NIMS-OS, random forest regression is used as a prediction model. Although BLOX can handle any number of objective functions, it is recommended that the number of the objective functions be limited to three or four, because exploration in more dimensions requires more time. BLOX has been used to search chemical spaces[33] and to explore superhard materials.[39]\n\nPhase diagram construction: PDC \nPDC is a Python package that can create a detailed phase diagram with a small number of experiments. To investigate a phase diagram, PDC proposes promising experimental conditions for the next experiment by using active learning. Specifically, uncertainty sampling based on the label propagation method finds uncertain points in the phase diagram, and these uncertain points are proposed for the next experiments. PDC was developed to propose multiple experimental conditions for batch experiments.[40] In NIMS-OS, the least confident score is used as an uncertainty score to evaluate uncertain points. Note that, for PDC, the objective function is the phase name or an index of phases, and thus only a one-dimensional objective function can be specified in the candidates file. PDC has been used to create new phase diagrams for the growth conditions of thin film[41] and to determine large and small areas of creep phenomena in polymer materials.[42]\n\nRandom exploration: RE \nIn RE, the next candidate experimental condition is selected randomly. This approach can be used to generate initial data before executing AI algorithms when no experimental data have yet been recorded. Furthermore, it can also be used to generate data for comparison as new AI algorithms can be developed.\n\nRobotic experiments \nThe module for robotic experiments comprises two Python scripts. The first script creates input files for robotic experiments according to the experimental conditions selected by the AI and commands a robot to begin the experiment. The second script analyzes the experimental results when the experiments are finished and updates the candidates file. At present, two types of modules are implemented in NIMS-OS: STAN and NAREE.\n\nStandard module for robotic experiments: STAN \nSTAN is a virtual implementation of the procedure for conducting robotic experiments. Thus, NIMS-OS can be run virtually using this module, even without a robotic device. In this module, the following steps are executed:\n\nCreate the input files for the robotic experiments in an appropriate folder according to the experimental conditions selected by the AI. In this standard module, we simply create a text file with a date as its name.\nSend a signal to the robotic system to begin the experiments. Depending on the machine, various cases can be considered, such as sending a start signal via serial communication. In this standard module, we assume that the experiments are begun by storing the inputend.txt file in the specified folder.\nWait until the robotic experiments are completed. This step includes various operations, such as receiving signals from the robot when the experiment is finished. This standard module assumes that the robot outputs outputend.txt file to indicate that the experiment is finished, and NIMS-OS continues waiting until this file appears.\nRead the files of experimental results and extract the values of objective functions. Here, the case of simply reading results.csv, which contains the objective function values, is implemented.\nUpdate the candidates file according to the values extracted in (4).\nSteps (1) and (2) are performed by preparation_input.py, and analysis_output.py conducts steps (3)-(5). In practice, for use with actual robotic systems, new modules can be created according to this standard module.\nAdditionally, this module can also facilitate closed-loop materials exploration between AI and experiments for processes that are time-consuming and cannot be partially automated. The procedure is as follows: When the proposals.csv file is generated, NIMS-OS automatically enters a sleep mode until experimental results are obtained. Based on the information in proposals.csv, the corresponding manual experiments are conducted. Once the objective function values are obtained through the experiments, a results.csv file is created, containing the objective function values corresponding to each line in proposals.csv. The results.csv file, along with an empty file named outputend.txt, is stored in the specified folder where the experimental results are output. Subsequently, NIMS-OS restarts and generates a new proposals.csv file.\n\n NIMS automated robotic electrochemical experiments (NAREE) system: NAREE \nAs a robotic system for materials science, the NIMS Automated Robotic Electrochemical Experiments (NAREE) system[13][35] can be used in NIMS-OS. NAREE comprises a liquid-handling dispenser, an electrochemical measurement unit, and a robotic arm. By using a microplate-based electrochemical cell equipped with electrodes, the performance of electrolytes prepared by mixing solution by a liquid handling dispenser is electrochemically evaluated in a high-throughput manner. This module was developed according to the procedures of the previously described STAN.\n\nUsage of the NIMS-OS Python version \nInstall \nNIMS-OS is written in Python3 programming language (version 3.6 or higher is required), and it can be installed via PyPI as follows:\n\n$ python3 -m pip install nimsos\nIf this installation is successful, the following packages are also installed or updated automatically:\n\nCython\nmatplotlib\nnumpy\nphysbo\nscikit-learn\nscipy\nBasic usage \nWe show a small example program (Program 1) in which PHYSBO is performed. In this program, assuming no experimental results in the candidates file, random exploration is performed in the first cycle.\n\nAssignment of parameters and candidates file \nFirst, the parameters for closed-loop experiments are defined. For example, when the number of objective functions is two, the number of proposals for each cycle is two, and the number of cycles is three. We define this in the code as follows:\n\nObjectivesNum\u2009=\u20092ProposalsNum\u2009=\u20092CyclesNum\u2009=\u20093\nNext, we specify a .csv file containing the candidates of experimental conditions, which is prepared as described under \"Preparation of candidates for experimental conditions.:\n\ncandidates_file\u2009=\u2009\u201c.\/candidates.csv\u201d\nThe name of the file that will contain the experimental conditions selected by the AI is as follows:\n\nproposals_file\u2009=\u2009\u201c.\/proposals.csv\u201d\nWe specify the folder name where the input files for the robotic experiments are stored and the folder name where the results from the experiments are output, respectively, as follows:\n\ninput_folder\u2009=\u2009\u201c.\/EXPInput\u201d\r\n\noutput_folder\u2009=\u2009\u201c.\/EXPOutput\u201d\nExecution of AI \nnimsos.selection is a class to select the next experimental conditions with the help of the AI. For example, nimsos.selection is used as follows:\n\nnimsos.selection(method\u2009=\u2009\u201cPHYSBO\u201d,\ninput_file\u2009=\u2009candidates_file,output_file\u2009=\u2009proposals_file,\u2003num_objectives\u2009=\u2009ObjectivesNum,\nnum_proposals\u2009=\u2009ProposalsNum)\nThe parameters of the method in this class (Program 1) indicate the module for AI algorithms. For the method, \"PHYSBO\" (Bayesian optimization), \"BLOX\" (objective free search), \"PDC\" (phase diagram construction), and \"RE\" (random exploration) are specified. The experimental conditions are selected from the data without the values of objective functions among input_file. In addition, selected conditions are outputted to output_file. For num_objectives, the number of objectives is input, and the number of proposals is specified as num_proposals. In general, although many hyperparameters should be considered to use the AI, they are determined automatically in NIMS-OS. Note that if there are no experimental results in the candidates file, only \"RE\" is used. For \"PHYSBO,\" \"BLOX,\" and \"PDC,\" some values of objective functions must be stored in the candidates file.\n\r\n\n\n\n\n\n\n\n\n\n\nProgram 1. Small example of NIMS-OS for Bayesian optimization.\n\n\n\nPreparation of input files for robotic experiments and execution of experiments \nnimsos.preparation_input is a class to prepare the input files for robotic experiments and send the start message to the robot. For example, nimsos.preparation_input is used as follows.\n\nnimsos.preparation_input(machine\u2009=\u2009\u201cSTAN\u201d,\ninput_file\u2009=\u2009proposals_file,\ninput_folder\u2009=\u2009input_folder)\nThe parameter of machine selects the module of robotic experiments. For machine, \"STAN,\" which is the standard module for this procedure, and \"NAREE\" (NIMS automated robotic electrochemical experiments) are used. For input_file, the experimental conditions selected by the AI are specified. In addition, the folder in the computer where the input files for robotic experiments are stored is referred to as the input_folder. In the nimsos.preparation_input module, the two functions make_machine_file() and send_message_machine() should be modified depending on the robotic systems used. The former creates the input files for robotic experiments from selected experimental conditions, whereas the latter sends the message to begin the robotic experiments.\n\nAnalysis of output files from experiments and update of candidates file \nnimsos.analysis_output is a class used to analyze the experimental results and update the candidates file. For example, nimsos.analysis_output is used as follows:\n\nnimsos.analysis_output(machine\u2009=\u2009\u201cSTAN\u201d,\ninput_file\u2009=\u2009proposals_file,\u2003\u2009output_file\u2009=\u2009candidates_file,\u2003\u2003num_objectives\u2009=\u2009ObjectivesNum,\n\u2003\u2009output_folder\u2009=\u2009output_folder)\nThe parameter of machine is the same as that found in the nimsos.preparation_input module, which selects the module for robotic experiments. Here, \"STAN\" and \"NAREE\" can be selected. For input_file, the experimental conditions selected by the AI are specified, and output_file is the name of the candidates file. The file specified by output_file is updated by this module. In addition, for num_objectives, the number of objectives is input. For output_folder, the folder in the computer where the results from robotic experiments are output is specified. In the nimsos.analysis_output module, two functions extract_objectives() and recieve_exit_message() should be modified depending on the robot systems. The former extracts the values of objective functions from the output files of robotic experiments, and the latter receives the message when the robotic experiments are finished. If \"NAREE\" is selected, objectives_info should be specified as a dictionary indicating which objective function is extracted from the experimental results.\n\nVisualization of the results \nBy using nimsos.visualization, the figures of the results are obtained. When this module is used, the new folder named \"fig\" is prepared in advance in the same folder where the main script is stored. The figures are output to this folder. nimsos.visualization.plot_history and nimsos.visualization.plot_distribution.plot create figures for the history and distributions of objective functions, respectively. These modules are useful when using AI algorithms other than PDC. In contrast, nimsos.visualization.plot_phase_diagram.plot creates the predicted phase diagram when PDC is used as an AI algorithm.\n\nUsage of the NIMS-OS GUI version \nA GUI version of NIMS-OS has been developed for easy execution, which is available at https:\/\/github.com\/nimsos-dev\/nimsos-gui. This can be used after installing the required Python version, as described in the prior section, and performing the installation as described in the manual (https:\/\/nimsos-dev.github.io\/nimsos\/docs\/en\/index.html). Figure 4 shows the operation screen of the NIMS-OS GUI version. In this GUI version, the name of the candidates file is fixed to candidates.csv, and the name of the proposals file is fixed to proposals.csv. \n\r\n\n\n\n\n\n\n\n\n\n\nFigure 4. Operation screen of the NIMS-OS GUI version.\n\n\n\nThe execution procedure is as follows:\n\nSpecify the number of objectives, proposals, and cycles in the \"Parameters\" section.\nSelect the method to be used in the AI algorithm section. If we use a newly created module for AI method named \"ai_tool_original.py,\" click on Original.\nSelect the robotic system in the Robotic system section. If we use a newly created module for robotic systems named \"preparation_input_original.py\" and \"analysis_output_original.py,\" click on Original.\nPress the \"Run\" button on the \"Controller\" section to begin automated materials exploration.\nWhen NIMS-OS is started, the Cycle counter begins to operate. Furthermore, in the \"Time\" section, the amount of time required to execute the AI algorithm and a single cycle are measured, and the remaining time is also output. The standard output of the Python version is displayed in real time in the \"Results\" section, and these output results can be saved as a file by pressing the Output button. In addition, to pause the automated exploration, the user can press the \"Stop\" button of the \"Controller\" section. Note that pressing this button does not stop the process immediately, but when the candidates file is updated, NIMS-OS is stopped. To reset the settings, press the \"Reset\" button on the Controller. The operation with NAREE is shown as a video (Supplemental Movie 1). Note that even for processes that cannot be partially automated, the closed loop between AI and manual experiments can still be achieved by selecting STAN in the \"Robotic system\" section, as explained prior.\n\nApplication \nTo demonstrate the effectiveness of NIMS-OS for the application of automated robotic experiments, we applied the NIMS-OS for NAREE system and performed an exploration for multi-component electrolytes that maximize the performance of lithium metal electrode. The anode-free type microplate based electrochemical cells were fabricated using LiFePO4 as positive electrode and Cu foil as negative electrode. The cells were subjected to a charging process with capacity limitation of 0.05 mAh. After that, the cells were subjected to a discharge process. Here, we defined the discharge time as a one-dimensional objective function. In this case, the longer discharge time represents the better battery performance (higher capacity). Using such an experimental setup, a combination of electrolyte additives was optimized to maximize the discharge time. Five different additives were selected from a list of 16 compounds (Table 1) and injected into an electrochemical cell containing 1\u2009M LiTFSI in TEGDME. In this case, the number of candidates for combination of electrolyte additives is \n \n \n \n \n  \n \n 16\n \n \n \n C\n \n 5\n \n \n =\n 4\n ,\n \n 368\n \n \n {\\displaystyle ~_{16}C_{5}=4,\\!368}\n \n . The candidate files for this experiment were prepared in a similar manner as shown in Figure 3 (combination of materials). In our experiment, 32 electrochemical cells were prepared in one microplate and 32 experiments were performed in parallel for two\u2009hours.\n\n\n\n\n\n\n\nTable 1. List of 16 types of additives used in an automated exploration for new electrolytes using the NAREE system. For all additives, the solvent is fixed as TEGDME.\n\n\nID\n\nAdditive\n\nConcentration\n\n\n1\n\nlithium bis(pentafluoroethanesulfonyl)imide (LiBETI)\n\n100\u2009mM\n\n\n2\n\nLiPF6\n\n100\u2009mM\n\n\n3\n\nLiBF4\n\n100\u2009mM\n\n\n4\n\nlithium bis(trifluoro methanesulfonyl)imide (LiTFSI)\n\n100\u2009mM\n\n\n5\n\nLiTfO\n\n100\u2009mM\n\n\n6\n\nLiClO4\n\n100\u2009mM\n\n\n7\n\nlithium bis(oxalate)borate (LiBOB)\n\n10\u2009mM\n\n\n8\n\nLiAsF6\n\n10\u2009mM\n\n\n9\n\nLiF\n\n10\u2009mM\n\n\n10\n\nN-methyl-2-pyrrodione (NMP)\n\n2 vol.%\n\n\n11\n\nsulfolane\n\n2 vol.%\n\n\n12\n\ndimethyl sulfoxide (DMSO)\n\n2 vol.%\n\n\n13\n\npropylene carbonate (PC)\n\n2 vol.%\n\n\n14\n\nethylene carbonate (EC)\n\n2 vol.%\n\n\n15\n\nfluoroethylene carbonate (FEC)\n\n2 vol.%\n\n\n16\n\nvinylene carbonate (VC)\n\n2 vol.%\n\n\n\nFor the autonomous experiments for searching multi-component electrolytes using the NAREE system operated by NIMS-OS, at first, 32 parallel experiments (one microplate) were performed by random exploration using RE because we do not have initial data at this stage. After obtaining the initial data by RE, the next five cycles of experiments (five microplates) were performed by BO using PHYSBO. Notably, a fully automated experiment was continuously conducted without any human intervention for 10\u2009hours. After that, addition six cycles of experiments (six microplates) were also performed by BO. In total, 384 experiments were performed. The obtained results can be visualized by using nimsos.visualization in the Python version of NIMS-OS, and the time course of the objective function and the histogram distribution of the results in the total 384 experiments were summarized in Figure 5. The results clearly revealed that the best electrolyte composition was discovered at the seventh experimental cycle. In Table 2, the details of electrolyte composition for the top 10 samples that enhanced the discharge time were summarized. The electrolyte\u2014containing 100\u2009mM LiPF6, 100\u2009mM LiTFSI, 2 vol.% PC, 2 vol.% FEC, and 2 vol.% VC\u2014exhibits the highest discharge time of 1,439.09 seconds. It should be noted that the possible maximum discharge time is 1,800 seconds since the current density during discharge was set to 0.1\u2009mA. Thus, there is still much room for improvement of battery performance. In addition, there can be seen that most of the top 10 samples contain VC and\/or FEC. These results are essentially consistent with the knowledge in this field that VC and FEC have a positive effect for improving the performance of the lithium metal electrode.[43][44]\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 5. Output results from NIMS-OS for automated exploration for electrolytes using the NAREE system: (a) history_step.png and (b) history_best.png by nimsos.Visualization.plot_history and (c) distribution.png by nimsos.Visualization.plot_distribution.plot. The target property is the discharge time and its unit is seconds. In the first cycle, RE is used to generate initial states. After the second cycle, PHYSBO is used.\n\n\n\n\n\n\n\n\n\nTable 2. Top 10 compositions that enhanced the discharge time. The found cycle number is also shown.\n\n\nRanking\n\nAdditive 1\n\nAdditive 2\n\nAdditive 3\n\nAdditive 4\n\nAdditive 5\n\nDischarge time\r\n(seconds)\n\nFound cycle\n\n\n1\n\n100\u2009mM LiPF6\n\n100\u2009mM LiTFSI\n\n2 vol.% PC\n\n2 vol.% FEC\n\n2 vol.% VC\n\n1,439.09\n\nSeventh\n\n\n2\n\n100\u2009mM LiBETI\n\n100\u2009mM LiTfO\n\n10\u2009mM LiBOB\n\n2 vol.% EC\n\n2 vol.% FEC\n\n1,401.97\n\nTwelfth\n\n\n3\n\n2 vol.% NMP\n\n2 vol.% sulfolane\n\n2 vol.% DMSO\n\n2 vol.% PC\n\n2 vol.% FEC\n\n1,374.86\n\nNinth\n\n\n4\n\n100\u2009mM LiBF4\n\n100\u2009mM LiTFSI\n\n100\u2009mM LiTfO\n\n10\u2009mM LiF\n\n2 vol.% FEC\n\n1,365.57\n\nNinth\n\n\n5\n\n100\u2009mM LiBETI\n\n100\u2009mM LiBF44\n\n10\u2009mM LiBOB\n\n2 vol.% PC\n\n2 vol.% FEC\n\n1,364.32\n\nTwelfth\n\n\n6\n\n100\u2009mM LiTFSI\n\n100\u2009mM LiTfO\n\n10\u2009mM LiAsF6\n\n2 vol.% FEC\n\n2 vol.% VC\n\n1,358.99\n\nTenth\n\n\n7\n\n100\u2009mM LiBETI\n\n10\u2009mM LiBOB\n\n10\u2009mM LiF\n\n2 vol.% EC\n\n2 vol.% VC\n\n1,357.43\n\nTenth\n\n\n8\n\n100\u2009mM LiBETI\n\n100\u2009mM LiTFSI\n\n10\u2009mM LiBOB\n\n10\u2009mM LiF\n\n2 vol.% FEC\n\n1,356.39\n\nNinth\n\n\n9\n\n100\u2009mM LiBF4\n\n100\u2009mM LiTFSI\n\n100\u2009mM LiClO4\n\n2 vol.% sulfolane\n\n2 vol.% VC\n\n1,347.23\n\nEleventh\n\n\n10\n\n100\u2009mM LiBF6\n\n100\u2009mM LiTfO\n\n10\u2009mM LiAsF6\n\n10\u2009mM LiF\n\n2 vol.% EC\n\n1,346.72\n\nSeventh\n\n\n\nConclusion \nIn this study, we developed NIMS-OS to implement a closed loop of AI and robotic experiments for automated materials exploration. We anticipate that this software can serve as a generic control system. To use NIMS-OS, a candidates file listing experimental conditions as a materials search space should be prepared in advance. This allows various problems for automated materials exploration to be commonly performed in NIMS-OS. Establishing standards for automated materials exploration is a key advantages of such generic control software. Using NIMS-OS and the NAREE system, we also demonstrated an example of automatic exploration for electrolytes.\nThe compatibility with original robotic systems other than NAREE is discussed. We believe that the most crucial aspects of integrating other robotic systems lie in providing instructions to initiate the robot and determining the completion of the robotic experiment. Regarding the former, the current NAREE system is designed to automatically start an experiment when an input file is stored in a specified folder. Therefore, by making slight modifications to the existing Python script, original robotic systems with this functionality can be easily integrated into NIMS-OS. Even if the PC controlling the experimental system and the PC running the NIMS-OS are different, the robot can be started by sharing the specified folder using a file-sharing service or network-attached storage (NAS). However, if the experimental systems require voltage signal control or application programming interface (API) control, specific Python code needs to be developed. The development of Python code for voltage signal control or API control is considered a future prospect. Regarding the latter, the experimental results will always be output in the specified folder. Therefore, it is sufficient to determine whether the result files have been generated or not, even if the robotic systems are changed. Furthermore, there may be cases where the experimental system can only be fully controlled by the GUI-driven software. In such cases, it is necessary to manually press a button on the GUI control screen to initiate the robot. However, an automated closed loop can be achieved by installing Robotic Process Automation (RPA) on the PC that controls the experimental system.[15] For example, with RPA, the following operations can be performed: (i) identifying the presence of an input file in the specified folder, (ii) providing instructions to initiate the experiments on the GUI operation screen that controls the experimental system, and (iii) deleting the input file once it is confirmed that the experimental results have been generated. Thus, we believe that the current NIMS-OS is designed to be easily adaptable to a variety of original robotic systems in materials science.\nAt present, NIMS-OS does not include a sufficient set of available AI algorithms and robotic experimental systems. For the further growth of this OS, developing and releasing more modules for various AI algorithms and robotic systems will be essential. The NAREE system used in this study can perform sequential operations since all evaluations of proposed experimental conditions by robotic experiments are completed within the same timeframe. However, in realistic experiments, the costs associated with synthesis, device fabrication, and evaluation strongly depend on the specific experimental conditions. In such cases, waiting for all experiments to be completed would be inefficient. To address this issue, BO introduces the concept of asynchronous parallel global optimization.[45][46] Therefore, it is necessary to develop a module within NIMS-OS that can facilitate asynchronous parallel optimization. Furthermore, in automated materials exploration, a greater amount of experimental data is generated compared to human experiments. Thus, the ability to store, share, and utilize experimental data for secondary purposes should be implemented as extensions in NIMS-OS. Specifically, a module that facilitates the automatic transfer of data to external storage or data repositories will be essential in enhancing data sharing and utilization. We will continue to enhance the extensions available for NIMS-OS to develop it as a game changer for digital transformation (DX) in materials science.\n\nSupplemental material \nSupplemental Movie 1 (.mp4; 12,302 KB)\n Abbreviations, acronyms, and initialisms \nAI: artificial intelligence\nBO: Bayesian optimization\nBLOX: BoundLess Objective-free eXploration\nDMSO: dimethyl sulfoxide\nEC: ethylene carbonate\nFEC: fluoroethylene carbonate\nGUI: graphical user interface\nLiBETI: lithium bis(pentafluoroethanesulfonyl)imide\nLiBOB: lithium bis(oxalate)borate\nLiTFSI: lithium bis(trifluoro methanesulfonyl)imide\nML: machine learning\nNAREE: NIMS Automated Robotic Electrochemical Experiments\nNIMS-OS: NIMS Orchestration System\nNMP: N-methyl-2-pyrrodione\nPC: propylene carbonate\nPDC: phase diagram construction\nPHYSBO: Optimization Tools for PHYSics Based on Bayesian Optimization\nRE: random exploration\nSTAN: STANdard module\nVC: vinylene carbonate\nAcknowledgements \nThe authors thank Masahiko Demura, Hideki Yoshikawa, and Masanobu Naito for valuable discussions. The authors also thank Kazuha Nakamura for experimental contributions, and thank Satoshi Murata, Daisuke Ryuno, and Hiromichi Taketa for the development of NIMS-OS.\n\nFunding \nThe work was supported by the MEXT Program: Data Creation and Utilization-Type Material Research and Development Project [JPMXP1122712807].\n\nCorrection statement \nThis article has been republished with minor changes. These changes do not impact the academic content of the article.\n\nSupplementary data \nSupplemental data for this article can be accessed online at https:\/\/doi.org\/10.1080\/27660400.2023.2232297.\n\nConflict of interest \nNo potential conflict of interest was reported by the author(s).\n\nReferences \n\n\n\u2191 White, Ashley (1 August 2012). \"The Materials Genome Initiative: One year on\" (in en). MRS Bulletin 37 (8): 715\u2013716. doi:10.1557\/mrs.2012.194. ISSN 0883-7694. http:\/\/link.springer.com\/10.1557\/mrs.2012.194 .   \n \n\n\u2191 Ramprasad, Rampi; Batra, Rohit; Pilania, Ghanshyam; Mannodi-Kanakkithodi, Arun; Kim, Chiho (13 December 2017). \"Machine learning in materials informatics: recent applications and prospects\" (in en). npj Computational Materials 3 (1): 54. doi:10.1038\/s41524-017-0056-5. ISSN 2057-3960. https:\/\/www.nature.com\/articles\/s41524-017-0056-5 .   \n \n\n\u2191 Schmidt, Jonathan; Marques, M\u00e1rio R. G.; Botti, Silvana; Marques, Miguel A. L. (8 August 2019). \"Recent advances and applications of machine learning in solid-state materials science\" (in en). npj Computational Materials 5 (1): 83. doi:10.1038\/s41524-019-0221-0. ISSN 2057-3960. https:\/\/www.nature.com\/articles\/s41524-019-0221-0 .   \n \n\n\u2191 Ling, Chen (18 February 2022). \"A review of the recent progress in battery informatics\" (in en). npj Computational Materials 8 (1): 33. doi:10.1038\/s41524-022-00713-x. ISSN 2057-3960. https:\/\/www.nature.com\/articles\/s41524-022-00713-x .   \n \n\n\u2191 Terayama, Kei; Sumita, Masato; Tamura, Ryo; Tsuda, Koji (16 March 2021). \"Black-Box Optimization for Automated Discovery\" (in en). Accounts of Chemical Research 54 (6): 1334\u20131346. doi:10.1021\/acs.accounts.0c00713. ISSN 0001-4842. https:\/\/pubs.acs.org\/doi\/10.1021\/acs.accounts.0c00713 .   \n \n\n\u2191 Ueno, Tsuyoshi; Rhone, Trevor David; Hou, Zhufeng; Mizoguchi, Teruyasu; Tsuda, Koji (1 June 2016). \"COMBO: An efficient Bayesian optimization library for materials science\" (in en). Materials Discovery 4: 18\u201321. doi:10.1016\/j.md.2016.04.001. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2352924516300035 .   \n \n\n\u2191 Packwood, Daniel (2017). Bayesian Optimization for Materials Science. SpringerBriefs in the Mathematics of Materials. 3. Singapore: Springer Singapore. doi:10.1007\/978-981-10-6781-5. ISBN 978-981-10-6780-8. http:\/\/link.springer.com\/10.1007\/978-981-10-6781-5 .   \n \n\n\u2191 Jin, Yimeng; Kumar, Priyank V. (2023). \"Bayesian optimisation for efficient material discovery: a mini review\" (in en). Nanoscale 15 (26): 10975\u201310984. doi:10.1039\/D2NR07147A. ISSN 2040-3364. http:\/\/xlink.rsc.org\/?DOI=D2NR07147A .   \n \n\n\u2191 Homma, Kenji; Liu, Yu; Sumita, Masato; Tamura, Ryo; Fushimi, Naoki; Iwata, Junichi; Tsuda, Koji; Kaneta, Chioko (18 June 2020). \"Optimization of a Heterogeneous Ternary Li 3 PO 4 \u2013Li 3 BO 3 \u2013Li 2 SO 4 Mixture for Li-Ion Conductivity by Machine Learning\" (in en). The Journal of Physical Chemistry C 124 (24): 12865\u201312870. doi:10.1021\/acs.jpcc.9b11654. ISSN 1932-7447. https:\/\/pubs.acs.org\/doi\/10.1021\/acs.jpcc.9b11654 .   \n \n\n\u2191 Sakurai, Atsushi; Yada, Kyohei; Simomura, Tetsushi; Ju, Shenghong; Kashiwagi, Makoto; Okada, Hideyuki; Nagao, Tadaaki; Tsuda, Koji et al. (27 February 2019). \"Ultranarrow-Band Wavelength-Selective Thermal Emission with Aperiodic Multilayered Metamaterials Designed by Bayesian Optimization\" (in en). ACS Central Science 5 (2): 319\u2013326. doi:10.1021\/acscentsci.8b00802. ISSN 2374-7943. PMC PMC6396383. PMID 30834320. https:\/\/pubs.acs.org\/doi\/10.1021\/acscentsci.8b00802 .   \n \n\n\u2191 Sun, Shijing; Tiihonen, Armi; Oviedo, Felipe; Liu, Zhe; Thapa, Janak; Zhao, Yicheng; Hartono, Noor Titan P.; Goyal, Anuj et al. (1 April 2021). \"A data fusion approach to optimize compositional stability of halide perovskites\" (in en). Matter 4 (4): 1305\u20131322. doi:10.1016\/j.matt.2021.01.008. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2590238521000084 .   \n \n\n\u2191 Tamura, Ryo; Osada, Toshio; Minagawa, Kazumi; Kohata, Takuma; Hirosawa, Masashi; Tsuda, Koji; Kawagishi, Kyoko (1 January 2021). \"Machine learning-driven optimization in powder manufacturing of Ni-Co based superalloy\" (in en). Materials & Design 198: 109290. doi:10.1016\/j.matdes.2020.109290. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0264127520308261 .   \n \n\n\u2191 13.0 13.1 13.2 13.3 Matsuda, Shoichi; Lambard, Guillaume; Sodeyama, Keitaro (1 April 2022). \"Data-driven automated robotic experiments accelerate discovery of multi-component electrolyte for rechargeable Li\u2013O2 batteries\" (in en). Cell Reports Physical Science 3 (4): 100832. doi:10.1016\/j.xcrp.2022.100832. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2666386422001023 .   \n \n\n\u2191 Ozaki, Yoshihiko; Suzuki, Yuta; Hawai, Takafumi; Saito, Kotaro; Onishi, Masaki; Ono, Kanta (5 June 2020). \"Automated crystal structure analysis based on blackbox optimisation\" (in en). npj Computational Materials 6 (1): 75. doi:10.1038\/s41524-020-0330-9. ISSN 2057-3960. https:\/\/www.nature.com\/articles\/s41524-020-0330-9 .   \n \n\n\u2191 15.0 15.1 Tamura, Ryo; Sumita, Masato; Terayama, Kei; Tsuda, Koji; Izumi, Fujio; Matsushita, Yoshitaka (31 December 2022). \"Automatic Rietveld refinement by robotic process automation with RIETAN-FP\" (in en). Science and Technology of Advanced Materials: Methods 2 (1): 435\u2013444. doi:10.1080\/27660400.2022.2146470. ISSN 2766-0400. https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/27660400.2022.2146470 .   \n \n\n\u2191 Chakraborti, N. (1 June 2004). \"Genetic algorithms in materials design and processing\" (in en). International Materials Reviews 49 (3-4): 246\u2013260. doi:10.1179\/095066004225021909. ISSN 0950-6608. http:\/\/www.tandfonline.com\/doi\/full\/10.1179\/095066004225021909 .   \n \n\n\u2191 Patra, Tarak K.; Meenakshisundaram, Venkatesh; Hung, Jui-Hsiang; Simmons, David S. (13 February 2017). \"Neural-Network-Biased Genetic Algorithms for Materials Design: Evolutionary Algorithms That Learn\" (in en). ACS Combinatorial Science 19 (2): 96\u2013107. doi:10.1021\/acscombsci.6b00136. ISSN 2156-8952. https:\/\/pubs.acs.org\/doi\/10.1021\/acscombsci.6b00136 .   \n \n\n\u2191 M. Dieb, Thaer; Ju, Shenghong; Yoshizoe, Kazuki; Hou, Zhufeng; Shiomi, Junichiro; Tsuda, Koji (31 December 2017). \"MDTS: automatic complex materials design using Monte Carlo tree search\" (in en). Science and Technology of Advanced Materials 18 (1): 498\u2013503. doi:10.1080\/14686996.2017.1344083. ISSN 1468-6996. PMC PMC5532970. PMID 28804525. https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/14686996.2017.1344083 .   \n \n\n\u2191 Li, Jiawen; Zhang, Jinzhe; Tamura, Ryo; Tsuda, Koji (2022). \"Self-learning entropic population annealing for interpretable materials design\" (in en). Digital Discovery 1 (3): 295\u2013302. doi:10.1039\/D1DD00043H. ISSN 2635-098X. http:\/\/xlink.rsc.org\/?DOI=D1DD00043H .   \n \n\n\u2191 Kitai, Koki; Guo, Jiang; Ju, Shenghong; Tanaka, Shu; Tsuda, Koji; Shiomi, Junichiro; Tamura, Ryo (16 March 2020). \"Designing metamaterials with quantum annealing and factorization machines\" (in en). Physical Review Research 2 (1): 013319. doi:10.1103\/PhysRevResearch.2.013319. ISSN 2643-1564. https:\/\/link.aps.org\/doi\/10.1103\/PhysRevResearch.2.013319 .   \n \n\n\u2191 Koshikawa, Ami S.; Ohzeki, Masayuki; Kadowaki, Tadashi; Tanaka, Kazuyuki (15 June 2021). \"Benchmark Test of Black-box Optimization Using D-Wave Quantum Annealer\" (in en). Journal of the Physical Society of Japan 90 (6): 064001. doi:10.7566\/JPSJ.90.064001. ISSN 0031-9015. https:\/\/journals.jps.jp\/doi\/10.7566\/JPSJ.90.064001 .   \n \n\n\u2191 Izawa, Syun; Kitai, Koki; Tanaka, Shu; Tamura, Ryo; Tsuda, Koji (21 April 2022). \"Continuous black-box optimization with an Ising machine and random subspace coding\" (in en). Physical Review Research 4 (2): 023062. doi:10.1103\/PhysRevResearch.4.023062. ISSN 2643-1564. https:\/\/link.aps.org\/doi\/10.1103\/PhysRevResearch.4.023062 .   \n \n\n\u2191 King, Ross D.; Rowland, Jem; Oliver, Stephen G.; Young, Michael; Aubrey, Wayne; Byrne, Emma; Liakata, Maria; Markham, Magdalena et al. (3 April 2009). \"The Automation of Science\" (in en). Science 324 (5923): 85\u201389. doi:10.1126\/science.1165620. ISSN 0036-8075. https:\/\/www.science.org\/doi\/10.1126\/science.1165620 .   \n \n\n\u2191 Boyd, James (18 January 2002). \"Robotic Laboratory Automation\" (in en). Science 295 (5554): 517\u2013518. doi:10.1126\/science.295.5554.517. ISSN 0036-8075. https:\/\/www.science.org\/doi\/10.1126\/science.295.5554.517 .   \n \n\n\u2191 Olsen, Kevin (1 December 2012). \"The First 110 Years of Laboratory Automation: Technologies, Applications, and the Creative Scientist\" (in en). SLAS Technology 17 (6): 469\u2013480. doi:10.1177\/2211068212455631. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S247263032201706X .   \n \n\n\u2191 Macarron, Ricardo; Banks, Martyn N.; Bojanic, Dejan; Burns, David J.; Cirovic, Dragan A.; Garyantes, Tina; Green, Darren V. S.; Hertzberg, Robert P. et al. (1 March 2011). \"Impact of high-throughput screening in biomedical research\" (in en). Nature Reviews Drug Discovery 10 (3): 188\u2013195. doi:10.1038\/nrd3368. ISSN 1474-1776. https:\/\/www.nature.com\/articles\/nrd3368 .   \n \n\n\u2191 MacLeod, B. P.; Parlane, F. G. L.; Morrissey, T. D.; H\u00e4se, F.; Roch, L. M.; Dettelbach, K. E.; Moreira, R.; Yunker, L. P. E. et al. (15 May 2020). \"Self-driving laboratory for accelerated discovery of thin-film materials\" (in en). Science Advances 6 (20): eaaz8867. doi:10.1126\/sciadv.aaz8867. ISSN 2375-2548. PMC PMC7220369. PMID 32426501. https:\/\/www.science.org\/doi\/10.1126\/sciadv.aaz8867 .   \n \n\n\u2191 Shimizu, Ryota; Kobayashi, Shigeru; Watanabe, Yuki; Ando, Yasunobu; Hitosugi, Taro (1 November 2020). \"Autonomous materials synthesis by machine learning and robotics\" (in en). APL Materials 8 (11): 111110. doi:10.1063\/5.0020370. ISSN 2166-532X. https:\/\/pubs.aip.org\/apm\/article\/8\/11\/111110\/569817\/Autonomous-materials-synthesis-by-machine-learning .   \n \n\n\u2191 Dave, Adarsh; Mitchell, Jared; Kandasamy, Kirthevasan; Wang, Han; Burke, Sven; Paria, Biswajit; P\u00f3czos, Barnab\u00e1s; Whitacre, Jay et al. (1 December 2020). \"Autonomous Discovery of Battery Electrolytes with Robotic Experimentation and Machine Learning\" (in en). Cell Reports Physical Science 1 (12): 100264. doi:10.1016\/j.xcrp.2020.100264. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2666386420302861 .   \n \n\n\u2191 Burger, Benjamin; Maffettone, Phillip M.; Gusev, Vladimir V.; Aitchison, Catherine M.; Bai, Yang; Wang, Xiaoyan; Li, Xiaobo; Alston, Ben M. et al. (9 July 2020). \"A mobile robotic chemist\" (in en). Nature 583 (7815): 237\u2013241. doi:10.1038\/s41586-020-2442-2. ISSN 0028-0836. https:\/\/www.nature.com\/articles\/s41586-020-2442-2 .   \n \n\n\u2191 Roch, Lo\u00efc M.; H\u00e4se, Florian; Kreisbeck, Christoph; Tamayo-Mendoza, Teresa; Yunker, Lars P. E.; Hein, Jason E.; Aspuru-Guzik, Al\u00e1n (16 April 2020). Hu, Jianjun. ed. \"ChemOS: An orchestration software to democratize autonomous discovery\" (in en). PLOS ONE 15 (4): e0229862. doi:10.1371\/journal.pone.0229862. ISSN 1932-6203. PMC PMC7161969. PMID 32298284. https:\/\/dx.plos.org\/10.1371\/journal.pone.0229862 .   \n \n\n\u2191 32.0 32.1 Motoyama, Yuichi; Tamura, Ryo; Yoshimi, Kazuyoshi; Terayama, Kei; Ueno, Tsuyoshi; Tsuda, Koji (1 September 2022). \"Bayesian optimization package: PHYSBO\" (in en). Computer Physics Communications 278: 108405. doi:10.1016\/j.cpc.2022.108405. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0010465522001242 .   \n \n\n\u2191 33.0 33.1 Terayama, Kei; Sumita, Masato; Tamura, Ryo; Payne, Daniel T.; Chahal, Mandeep K.; Ishihara, Shinsuke; Tsuda, Koji (2020). \"Pushing property limits in materials discovery via boundless objective-free exploration\" (in en). Chemical Science 11 (23): 5959\u20135968. doi:10.1039\/D0SC00982B. ISSN 2041-6520. PMC PMC7409358. PMID 32832058. http:\/\/xlink.rsc.org\/?DOI=D0SC00982B .   \n \n\n\u2191 Terayama, Kei; Tamura, Ryo; Nose, Yoshitaro; Hiramatsu, Hidenori; Hosono, Hideo; Okuno, Yasushi; Tsuda, Koji (8 March 2019). \"Efficient construction method for phase diagrams using uncertainty sampling\" (in en). Physical Review Materials 3 (3): 033802. doi:10.1103\/PhysRevMaterials.3.033802. ISSN 2475-9953. https:\/\/link.aps.org\/doi\/10.1103\/PhysRevMaterials.3.033802 .   \n \n\n\u2191 35.0 35.1 Matsuda, Shoichi; Nishioka, Kiho; Nakanishi, Shuji (17 April 2019). \"High-throughput combinatorial screening of multi-component electrolyte additives to improve the performance of Li metal secondary batteries\" (in en). Scientific Reports 9 (1): 6211. doi:10.1038\/s41598-019-42766-x. ISSN 2045-2322. PMC PMC6470175. PMID 30996343. https:\/\/www.nature.com\/articles\/s41598-019-42766-x .   \n \n\n\u2191 Ong, Shyue Ping; Richards, William Davidson; Jain, Anubhav; Hautier, Geoffroy; Kocher, Michael; Cholia, Shreyas; Gunter, Dan; Chevrier, Vincent L. et al. (1 February 2013). \"Python Materials Genomics (pymatgen): A robust, open-source python library for materials analysis\" (in en). Computational Materials Science 68: 314\u2013319. doi:10.1016\/j.commatsci.2012.10.028. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0927025612006295 .   \n \n\n\u2191 Ward, Logan; Agrawal, Ankit; Choudhary, Alok; Wolverton, Christopher (26 August 2016). \"A general-purpose machine learning framework for predicting properties of inorganic materials\" (in en). npj Computational Materials 2 (1): 16028. doi:10.1038\/npjcompumats.2016.28. ISSN 2057-3960. https:\/\/www.nature.com\/articles\/npjcompumats201628 .   \n \n\n\u2191 \"RDKit: Open-Source Cheminformatics Software\". 2023. https:\/\/www.rdkit.org\/ .   \n \n\n\u2191 Ojih, Joshua; Al-Fahdi, Mohammed; Rodriguez, Alejandro David; Choudhary, Kamal; Hu, Ming (4 July 2022). \"Efficiently searching extreme mechanical properties via boundless objective-free exploration and minimal first-principles calculations\" (in en). npj Computational Materials 8 (1): 143. doi:10.1038\/s41524-022-00836-1. ISSN 2057-3960. https:\/\/www.nature.com\/articles\/s41524-022-00836-1 .   \n \n\n\u2191 Tamura, Ryo; Deffrennes, Guillaume; Han, Kwangsik; Abe, Taichi; Morito, Haruhiko; Nakamura, Yasuyuki; Naito, Masanobu; Katsube, Ryoji et al. (31 December 2022). \"Machine-Learning-Based phase diagram construction for high-throughput batch experiments\" (in en). Science and Technology of Advanced Materials: Methods 2 (1): 153\u2013161. doi:10.1080\/27660400.2022.2076548. ISSN 2766-0400. https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/27660400.2022.2076548 .   \n \n\n\u2191 Katsube, Ryoji; Terayama, Kei; Tamura, Ryo; Nose, Yoshitaro (1 June 2020). \"Experimental Establishment of Phase Diagrams Guided by Uncertainty Sampling: An Application to the Deposition of Zn\u2013Sn\u2013P Films by Molecular Beam Epitaxy\" (in en). ACS Materials Letters 2 (6): 571\u2013575. doi:10.1021\/acsmaterialslett.0c00104. ISSN 2639-4979. https:\/\/pubs.acs.org\/doi\/10.1021\/acsmaterialslett.0c00104 .   \n \n\n\u2191 Hu, Wei-Hsun; Chen, Ta-Te; Tamura, Ryo; Terayama, Kei; Wang, Siqian; Watanabe, Ikumu; Naito, Masanobu (31 December 2022). \"Topological alternation from structurally adaptable to mechanically stable crosslinked polymer\" (in en). Science and Technology of Advanced Materials 23 (1): 66\u201375. doi:10.1080\/14686996.2021.2025426. ISSN 1468-6996. PMC PMC8812728. PMID 35125966. https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/14686996.2021.2025426 .   \n \n\n\u2191 Ota, Hitoshi; Shima, Kunihisa; Ue, Makoto; Yamaki, Jun-ichi (1 February 2004). \"Effect of vinylene carbonate as additive to electrolyte for lithium metal anode\" (in en). Electrochimica Acta 49 (4): 565\u2013572. doi:10.1016\/j.electacta.2003.09.010. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0013468603007266 .   \n \n\n\u2191 Zhang, Xue-Qiang; Cheng, Xin-Bing; Chen, Xiang; Yan, Chong; Zhang, Qiang (1 March 2017). \"Fluoroethylene Carbonate Additives to Render Uniform Li Deposits in Lithium Metal Batteries\" (in en). Advanced Functional Materials 27 (10): 1605989. doi:10.1002\/adfm.201605989. https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/adfm.201605989 .   \n \n\n\u2191 Ginsbourger, D.; Janusevskis, J.; Le Riche, R. (2011). \"Dealing with asynchronicity in parallel Gaussian Process based global optimization\". Mines Saint-Etienne. https:\/\/hal.science\/hal-00507632 .   \n \n\n\u2191 Janusevskis, Janis; Le Riche, Rodolphe; Ginsbourger, David; Girdziusas, Ramunas (2012), Hamadi, Youssef; Schoenauer, Marc, eds., \"Expected Improvements for the Asynchronous Parallel Global Optimization of Expensive Functions: Potentials and Challenges\", Learning and Intelligent Optimization (Berlin, Heidelberg: Springer Berlin Heidelberg) 7219: 413\u2013418, doi:10.1007\/978-3-642-34413-8_37, ISBN 978-3-642-34412-1, http:\/\/link.springer.com\/10.1007\/978-3-642-34413-8_37 . Retrieved 2023-09-18   \n \n\n\nNotes \nThis presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added. In the original, there are multiple instances of citing research work using the last name of the last author listed, rather than the last name of the first author listed; this may have been a product of Japanese culture tending to read text from right to left. For this version, the last name of the first author was used to be consistent with research norms.\n\n\n\n\n\nSource: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.limswiki.org\/index.php\/Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\">https:\/\/www.limswiki.org\/index.php\/Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science<\/a>\nCategories: LIMSwiki journal articles (added in 2023)LIMSwiki journal articles (all)LIMSwiki journal articles on laboratory automationLIMSwiki journal articles on laboratory informaticsLIMSwiki journal articles on materials informaticsNavigation menuPage actionsJournalDiscussionView sourceHistoryPage actionsJournalDiscussionMoreToolsIn other languagesPersonal toolsLog inNavigationMain pageEncyclopedic articlesRecent changesRandom pageHelp about MediaWikiSearch\u00a0 ToolsWhat links hereRelated changesSpecial pagesPermanent linkPage informationPopular publications\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\nPrint\/exportCreate a bookDownload as PDFDownload as PDFDownload as Plain textPrintable version This page was last edited on 20 November 2023, at 21:33.Content is available under a Creative Commons Attribution-ShareAlike 4.0 International License unless otherwise noted.This page has been accessed 527 times.Privacy policyAbout LIMSWikiDisclaimers\n\n\n\n","9e1d6433c962801d5a75756c1046599c_html":"<body class=\"mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-206 ns-subject page-Journal_NIMS-OS_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science rootpage-Journal_NIMS-OS_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science skin-monobook action-view skin--responsive\"><div id=\"rdp-ebb-globalWrapper\"><div id=\"rdp-ebb-column-content\"><div id=\"rdp-ebb-content\" class=\"mw-body\" role=\"main\"><a id=\"rdp-ebb-top\"><\/a>\n<h1 id=\"rdp-ebb-firstHeading\" class=\"firstHeading\" lang=\"en\">Journal:NIMS-OS: An automation software to implement a closed loop between artificial intelligence and robotic experiments in materials science<\/h1><div id=\"rdp-ebb-bodyContent\" class=\"mw-body-content\"><!-- start content --><div id=\"rdp-ebb-mw-content-text\" lang=\"en\" dir=\"ltr\" class=\"mw-content-ltr\"><div class=\"mw-parser-output\">\n\n\n\n<h2><span class=\"mw-headline\" id=\"Abstract\">Abstract<\/span><\/h2>\n<p>NIMS-OS (NIMS Orchestration System) is a <a href=\"https:\/\/www.limswiki.org\/index.php\/Python_(programming_language)\" title=\"Python (programming language)\" class=\"wiki-link\" data-key=\"ef6905a29cbb75d3c71e6bdf6e2915dd\">Python<\/a> library created to realize a closed loop of <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory_automation\" title=\"Laboratory automation\" class=\"wiki-link\" data-key=\"0061880849aeaca05f8aa27ae171f331\">robotic<\/a> experiments and <a href=\"https:\/\/www.limswiki.org\/index.php\/Artificial_intelligence\" title=\"Artificial intelligence\" class=\"wiki-link\" data-key=\"0c45a597361ca47e1cd8112af676276e\">artificial intelligence<\/a> (AI) without human intervention for automated <a href=\"https:\/\/www.limswiki.org\/index.php\/Materials_science\" title=\"Materials science\" class=\"wiki-link\" data-key=\"89f5ce5de41da20cf3a2144a5731d5e6\">materials exploration<\/a>. It uses various combinations of modules to operate autonomously. Each module acts as an AI for materials exploration or a controller for a robotic experiments. As AI techniques, Optimization Tools for PHYSics Based on Bayesian Optimization (PHYSBO), BoundLess Objective-free eXploration (BLOX), phase diagram construction (PDC), and random exploration (RE) methods can be used. Moreover, a system called NIMS Automated Robotic Electrochemical Experiments (NAREE) is available as a set of robotic experimental equipment. <a href=\"https:\/\/www.limswiki.org\/index.php\/Data_visualization\" title=\"Data visualization\" class=\"wiki-link\" data-key=\"4a3b86cba74bc7bb7471aa3fc2fcccc3\">Visualization tools<\/a> for the results are also included, which allows users to check the optimization results in real time. Newly created modules for AI and robotic experiments can be added easily to extend the functionality of the system. In addition, we developed a graphical user interface (GUI)-driven application to control NIMS-OS. To demonstrate the operation of NIMS-OS, we consider an automated exploration for new electrolytes. NIMS-OS is available at <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/github.com\/nimsos-dev\/nimsos\" target=\"_blank\">https:\/\/github.com\/nimsos-dev\/nimsos<\/a>.\n<\/p><p><b>Keywords<\/b>: NIMS-OS, robotic experiments, artificial intelligence, electrochemistry, materials informatics\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Introduction\">Introduction<\/span><\/h2>\n<p>The integration of robotic experiments and <a href=\"https:\/\/www.limswiki.org\/index.php\/Artificial_intelligence\" title=\"Artificial intelligence\" class=\"wiki-link\" data-key=\"0c45a597361ca47e1cd8112af676276e\">artificial intelligence<\/a> (AI) is essential to realize automated materials exploration. If an AI system can take on some <a href=\"https:\/\/www.limswiki.org\/index.php\/Information\" title=\"Information\" class=\"wiki-link\" data-key=\"6300a14d9c2776dcca0999b5ed940e7d\">information<\/a> tasks conventionally performed by human researchers, robotic systems can then execute the required physical tasks and experiments for materials exploration can proceed automatically. Such a platform may be expected to discover many novel materials and lead to substantial innovation in <a href=\"https:\/\/www.limswiki.org\/index.php\/Materials_science\" title=\"Materials science\" class=\"wiki-link\" data-key=\"89f5ce5de41da20cf3a2144a5731d5e6\">materials science<\/a>. In recent years, significant progress has been made in the development of AI techniques and robotic devices suitable for materials exploration.\n<\/p><p>Since the launch of the Materials Genome Initiative<sup id=\"rdp-ebb-cite_ref-1\" class=\"reference\"><a href=\"#cite_note-1\">[1]<\/a><\/sup>, AI techniques have been actively used for materials exploration.<sup id=\"rdp-ebb-cite_ref-2\" class=\"reference\"><a href=\"#cite_note-2\">[2]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-3\" class=\"reference\"><a href=\"#cite_note-3\">[3]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-4\" class=\"reference\"><a href=\"#cite_note-4\">[4]<\/a><\/sup> In general, materials exploration can be regarded as the problem of finding optimal materials from among a materials search space. The elements to be used in the search space must be configured, along with its composition range, process parameter range, and so forth. To solve this problem, black-box optimization methods are useful<sup id=\"rdp-ebb-cite_ref-5\" class=\"reference\"><a href=\"#cite_note-5\">[5]<\/a><\/sup>, and various methods have been developed and applied to fit various needs. Bayesian optimization (BO) is among the most frequently used methods in materials science.<sup id=\"rdp-ebb-cite_ref-6\" class=\"reference\"><a href=\"#cite_note-6\">[6]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-7\" class=\"reference\"><a href=\"#cite_note-7\">[7]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-8\" class=\"reference\"><a href=\"#cite_note-8\">[8]<\/a><\/sup> In this method, promising materials can be selected in the materials search space using the predictions of their properties and the uncertainty of these predictions evaluated by Gaussian process regression. Using BO, various real materials, such as Li-ion conductive materials<sup id=\"rdp-ebb-cite_ref-9\" class=\"reference\"><a href=\"#cite_note-9\">[9]<\/a><\/sup>, multilayered metamaterials<sup id=\"rdp-ebb-cite_ref-10\" class=\"reference\"><a href=\"#cite_note-10\">[10]<\/a><\/sup>, halide perovskite<sup id=\"rdp-ebb-cite_ref-11\" class=\"reference\"><a href=\"#cite_note-11\">[11]<\/a><\/sup>, superalloys<sup id=\"rdp-ebb-cite_ref-12\" class=\"reference\"><a href=\"#cite_note-12\">[12]<\/a><\/sup>, and electrolytes<sup id=\"rdp-ebb-cite_ref-:0_13-0\" class=\"reference\"><a href=\"#cite_note-:0-13\">[13]<\/a><\/sup> have been explored. BO is also used for the automated analysis of materials.<sup id=\"rdp-ebb-cite_ref-14\" class=\"reference\"><a href=\"#cite_note-14\">[14]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:1_15-0\" class=\"reference\"><a href=\"#cite_note-:1-15\">[15]<\/a><\/sup> In addition, many methods have been proposed for black-box optimization in materials exploration, such as genetic algorithms<sup id=\"rdp-ebb-cite_ref-16\" class=\"reference\"><a href=\"#cite_note-16\">[16]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-17\" class=\"reference\"><a href=\"#cite_note-17\">[17]<\/a><\/sup>, Monte Carlo tree search<sup id=\"rdp-ebb-cite_ref-18\" class=\"reference\"><a href=\"#cite_note-18\">[18]<\/a><\/sup>, rare event sampling<sup id=\"rdp-ebb-cite_ref-19\" class=\"reference\"><a href=\"#cite_note-19\">[19]<\/a><\/sup>, and algorithms using an Ising machine.<sup id=\"rdp-ebb-cite_ref-20\" class=\"reference\"><a href=\"#cite_note-20\">[20]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-21\" class=\"reference\"><a href=\"#cite_note-21\">[21]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-22\" class=\"reference\"><a href=\"#cite_note-22\">[22]<\/a><\/sup> In the future, many more innovative methods are expected to be developed.\n<\/p><p>Robotic experiments have progressed to realize <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory_automation\" title=\"Laboratory automation\" class=\"wiki-link\" data-key=\"0061880849aeaca05f8aa27ae171f331\">laboratory automation<\/a> of chemical analysis and high-throughput screening in the field of biology.<sup id=\"rdp-ebb-cite_ref-23\" class=\"reference\"><a href=\"#cite_note-23\">[23]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-24\" class=\"reference\"><a href=\"#cite_note-24\">[24]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-25\" class=\"reference\"><a href=\"#cite_note-25\">[25]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-26\" class=\"reference\"><a href=\"#cite_note-26\">[26]<\/a><\/sup> Various types of automated analyzers and pipetting devices have been developed, and robotic arms have been used as a transport system to connect these systems. Moreover, robotic technology has been used to explore novel materials, such as thin-film materials<sup id=\"rdp-ebb-cite_ref-27\" class=\"reference\"><a href=\"#cite_note-27\">[27]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-28\" class=\"reference\"><a href=\"#cite_note-28\">[28]<\/a><\/sup>, battery electrolytes<sup id=\"rdp-ebb-cite_ref-:0_13-1\" class=\"reference\"><a href=\"#cite_note-:0-13\">[13]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-29\" class=\"reference\"><a href=\"#cite_note-29\">[29]<\/a><\/sup>, and photocatalysts.<sup id=\"rdp-ebb-cite_ref-30\" class=\"reference\"><a href=\"#cite_note-30\">[30]<\/a><\/sup> These studies used BO to automate the proposal of promising experimental conditions. This enables a closed loop of robotic experiments and AI that can perform automated materials exploration without human intervention. This approach involves some key advantages, such as the ability to generate materials data of uniform <a href=\"https:\/\/www.limswiki.org\/index.php\/Quality_(business)\" title=\"Quality (business)\" class=\"wiki-link\" data-key=\"c4ac43430d1c3a3a15d1255257aaea37\">quality<\/a> and the absence of human error. In contrast, at present, robotics systems are limited in their ability to perform complex material synthesis tasks that require the skills of experts. Thus, further innovation in robotic devices will be important.\n<\/p><p>In addition to AI and robotic technologies, the control systems and software used to interlink them are also an important element to realize a closed loop without human intervention. Generally, different AI algorithms should be used depending on the motivation of a materials exploration task. Furthermore, the procedure to control the devices should depends on the nature and characteristics of the robotic systems used. Therefore, control software has thus far been developed on a case-by-case basis for different AI algorithms and robotic systems.\n<\/p><p>In this study, we developed NIMS-OS (NIMS Orchestration System) to realize a closed loop between AI models and robotic experiments, with the aim of establishing a generic control software system. Although this software was written in the <a href=\"https:\/\/www.limswiki.org\/index.php\/Python_(programming_language)\" title=\"Python (programming language)\" class=\"wiki-link\" data-key=\"ef6905a29cbb75d3c71e6bdf6e2915dd\">Python programming language<\/a>, we also developed a graphical user interface (GUI)-driven version to improve operability after installation. NIMS-OS treats each AI algorithm and each robotic system as separate modules (see Figure 1). This enables the implementation of a closed loop with any combination of these modules. If modules for new AI algorithms or robotic systems are prepared, new closed-loop systems can be easily controlled via NIMS-OS. One of the advantages of developing such generic control software is the establishment of technical standards for automated materials exploration. For AI algorithms, we determined standard formats for the input and output. Algorithms created according to the standard format can be immediately tested using any currently available robotic system. Specifically, we developed a standard format in which all the experimental conditions to be explored are listed in advance, and the appropriate experimental conditions that have not yet been tested are selected from the list by AI algorithms. The advantage of this approach is that it enables automated materials exploration utilizing materials databases. When utilizing materials databases, the compositional and structural information needs to be converted into materials descriptors, which serve as the materials search space. However, this search space generated from the materials databases cannot be solely defined by a continuous parameter space, and it requires a selection from the pre-listed descriptors. Of course, optimization of continuous parameters can still be handled approximately by preparing a list of grid points that discretize the continuous parameters. Furthermore, we expect this work to contribute to the development of new AI algorithms for automated materials exploration. For robotic systems, we expect modules developed based on NIMS-OS to increase the commonality of operational procedures, leading to cost reductions as new robotic experimental devices are introduced. Note that ChemOS<sup id=\"rdp-ebb-cite_ref-31\" class=\"reference\"><a href=\"#cite_note-31\">[31]<\/a><\/sup> is similar to NIMS-OS; it was developed as an automation system in the field of chemistry. ChemOS specializes in BO within a defined continuous or discretized parameter space and includes several default modules for various BO methods. On the other hand, NIMS-OS offers the capability to perform automated materials explorations not only within a defined parameter space but also utilizing materials databases. In addition to BO, NIMS-OS also incorporates several default implementations of black-box optimization methods to deal with different motivations in materials explorations.\n<\/p><p><a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig1_Tamura_SciTechAdvMatMeth2023_3-1.jpeg\" class=\"image wiki-link\" data-key=\"069de801d1efe85d16dd45285e8c9419\"><img alt=\"Fig1 Tamura SciTechAdvMatMeth2023 3-1.jpeg\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/c\/c0\/Fig1_Tamura_SciTechAdvMatMeth2023_3-1.jpeg\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 1.<\/b> Image of the combinations of AI algorithms and robotic systems via NIMS-OS.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>Let us briefly introduce the specifications of NIMS-OS. First, a candidates file listing experimental conditions as a materials search space should be prepared in advance. A closed loop is formed according to the following three steps (see Figure 2):\n<\/p>\n<ul><li>Step 1: Select promising experimental conditions from the candidates file using an AI model.<\/li>\n<li>Step 2: Create an input file for the robotic experiments and execute the experiments.<\/li>\n<li>Step 3: Analyze the output from the experiments and update the candidates file based on the experimental results.<\/li><\/ul>\n<p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig2_Tamura_SciTechAdvMatMeth2023_3-1.jpeg\" class=\"image wiki-link\" data-key=\"d41763552a0817e47a76dae27677278a\"><img alt=\"Fig2 Tamura SciTechAdvMatMeth2023 3-1.jpeg\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/f\/f9\/Fig2_Tamura_SciTechAdvMatMeth2023_3-1.jpeg\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 2.<\/b> Procedures in NIMS-OS and roles of each Python scripts.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>Currently, the following AI algorithms are used as modules, which are available for Step 1: (i) Optimization Tools for PHYSics Based on Bayesian Optimization (PHYSBO)<sup id=\"rdp-ebb-cite_ref-:2_32-0\" class=\"reference\"><a href=\"#cite_note-:2-32\">[32]<\/a><\/sup>, (ii) BoundLess Objective-free eXploration (BLOX)<sup id=\"rdp-ebb-cite_ref-:3_33-0\" class=\"reference\"><a href=\"#cite_note-:3-33\">[33]<\/a><\/sup>, and (iii) phase diagram construction (PDC)<sup id=\"rdp-ebb-cite_ref-34\" class=\"reference\"><a href=\"#cite_note-34\">[34]<\/a><\/sup> methods, with the random exploration (RE) approach able to be selected according to the purpose of materials exploration effort. For Steps 2 and 3, a STANdard module (STAN) is provided for robotic experiments, which enables operation checks even without devices, along with a module for NIMS Automated Robotic Electrochemical Experiments (NAREE).<sup id=\"rdp-ebb-cite_ref-:0_13-2\" class=\"reference\"><a href=\"#cite_note-:0-13\">[13]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:4_35-0\" class=\"reference\"><a href=\"#cite_note-:4-35\">[35]<\/a><\/sup> We plan to continue developing additional modules for this system.\n<\/p><p>The reminder of this study is organized as follows. The next section describes the preparation of a candidates file storing experimental conditions, followed by an introduction to the available modules for the AI and robotic experiments in NIMS-OS. Then the use of the Python code, and the usage of the GUI version is explained. As a demonstration, the results of an autonomous electrolyte exploration via a closed-loop approach using PHYSBO and NAREE in NIMS-OS are described. Finally, this work concludes with some discussion and suggests some important avenues for further research.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Preparation_of_candidates_for_experimental_conditions\">Preparation of candidates for experimental conditions<\/span><\/h2>\n<p>A major feature of NIMS-OS is that a data file listing candidate experimental conditions is prepared in advance (we refer to this data file a candidates file). In general, because there are many candidates, conducting experiments in all possible conditions is impractical. Thus, automated materials exploration proceeds by selecting promising experimental conditions from these listed candidates. This makes the closed-loop strategy more generalizable. That is, a variety of exploration motivations and robotic systems can be handled by NIMS-OS.\n<\/p><p>The experimental condition is expressed as a real-valued vector <span class=\"mwe-math-element\"><span class=\"mwe-math-mathml-inline mwe-math-mathml-a11y\" style=\"display: none;\"><\/span><img src=\"https:\/\/en.wikipedia.org\/api\/rest_v1\/media\/math\/render\/svg\/e9a583436b0fc4bb022070c172dd90d71a0ae23a\" class=\"mwe-math-fallback-image-inline\" aria-hidden=\"true\" style=\"vertical-align: -0.671ex; width:7.821ex; height:3.009ex;\" alt=\"{\\displaystyle \\mathbf {x} _{i}\\in \\mathbb {R} ^{d}}\"\/><\/span>. This condition is prepared with information such as the compositions and structures of materials and the processes required to synthesize them. If the number of candidates for the experimental conditions is <i>N<\/i>, the dataset for candidates is defined as <span class=\"mwe-math-element\"><span class=\"mwe-math-mathml-inline mwe-math-mathml-a11y\" style=\"display: none;\"><\/span><img src=\"https:\/\/en.wikipedia.org\/api\/rest_v1\/media\/math\/render\/svg\/e576574569f2f964fb8240d2c67d155b84ee9feb\" class=\"mwe-math-fallback-image-inline\" aria-hidden=\"true\" style=\"vertical-align: -1.005ex; width:16.758ex; height:3.009ex;\" alt=\"{\\displaystyle D=\\{\\mathbf {x} _{i}\\}_{i=1,\\ldots ,N}}\"\/><\/span>. The initial candidates file is created by this dataset <i>D<\/i>. An example of a candidates file with <i>l<\/i> objective functions is presented in Figure 3. All the candidates of <i>D<\/i> are written in the first <i>d<\/i> columns. In this part, there should be no empty spaces. The next <i>l<\/i> columns are used for the objective function values. In this part, at the initial stage, all cells are empty because experiments have not been performed for all the experimental conditions.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig3_Tamura_SciTechAdvMatMeth2023_3-1.jpeg\" class=\"image wiki-link\" data-key=\"14bca0c7f0185297b1ec4a566d19c7d3\"><img alt=\"Fig3 Tamura SciTechAdvMatMeth2023 3-1.jpeg\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/b\/b4\/Fig3_Tamura_SciTechAdvMatMeth2023_3-1.jpeg\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 3.<\/b> (Top panels) Examples of the candidates files of the initial stage and that after some experiments. Here, an example for the case that <i>N<\/i>=9 is shown. (Bottom panels) Examples for the list of descriptors depending on the types of search space. If the continuous parameter space is considered, <span class=\"mwe-math-element\"><span class=\"mwe-math-mathml-inline mwe-math-mathml-a11y\" style=\"display: none;\"><\/span><img src=\"https:\/\/en.wikipedia.org\/api\/rest_v1\/media\/math\/render\/svg\/e576574569f2f964fb8240d2c67d155b84ee9feb\" class=\"mwe-math-fallback-image-inline\" aria-hidden=\"true\" style=\"vertical-align: -1.005ex; width:16.758ex; height:3.009ex;\" alt=\"{\\displaystyle D=\\{\\mathbf {x} _{i}\\}_{i=1,\\ldots ,N}}\"\/><\/span> is the discretized parameters. When the combination of materials is the search space, the bit strings where the material used is represented by 1 and the material not used is represented by 0 are prepared in <span class=\"mwe-math-element\"><span class=\"mwe-math-mathml-inline mwe-math-mathml-a11y\" style=\"display: none;\"><\/span><img src=\"https:\/\/en.wikipedia.org\/api\/rest_v1\/media\/math\/render\/svg\/e576574569f2f964fb8240d2c67d155b84ee9feb\" class=\"mwe-math-fallback-image-inline\" aria-hidden=\"true\" style=\"vertical-align: -1.005ex; width:16.758ex; height:3.009ex;\" alt=\"{\\displaystyle D=\\{\\mathbf {x} _{i}\\}_{i=1,\\ldots ,N}}\"\/><\/span>. Furthermore, materials descriptors from compositions obtained by such as magpie<sup id=\"rdp-ebb-cite_ref-36\" class=\"reference\"><a href=\"#cite_note-36\">[36]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-37\" class=\"reference\"><a href=\"#cite_note-37\">[37]<\/a><\/sup> and fingerprint of molecules obtained by such as RDKit<sup id=\"rdp-ebb-cite_ref-38\" class=\"reference\"><a href=\"#cite_note-38\">[38]<\/a><\/sup> would be used as <span class=\"mwe-math-element\"><span class=\"mwe-math-mathml-inline mwe-math-mathml-a11y\" style=\"display: none;\"><\/span><img src=\"https:\/\/en.wikipedia.org\/api\/rest_v1\/media\/math\/render\/svg\/e576574569f2f964fb8240d2c67d155b84ee9feb\" class=\"mwe-math-fallback-image-inline\" aria-hidden=\"true\" style=\"vertical-align: -1.005ex; width:16.758ex; height:3.009ex;\" alt=\"{\\displaystyle D=\\{\\mathbf {x} _{i}\\}_{i=1,\\ldots ,N}}\"\/><\/span>.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>In NIMS-OS, some promising conditions are selected from among those listed in the candidates file using AI models (available algorithms are described in the next section). When the values of objective functions are obtained by performing experiments, the objective functions in the candidates file are updated accordingly. That is, when the experiments are completed for <i>M<\/i> experimental conditions, only results for <i>M<\/i> conditions are entered at the <i>l<\/i> columns for the objective functions. Thus, at the next step, the experimental conditions are selected from among <span class=\"mwe-math-element\"><span class=\"mwe-math-mathml-inline mwe-math-mathml-a11y\" style=\"display: none;\"><\/span><img src=\"https:\/\/en.wikipedia.org\/api\/rest_v1\/media\/math\/render\/svg\/6b6ca84e6fdfb84c4f3eaf35ce7e4ceae5fce97b\" class=\"mwe-math-fallback-image-inline\" aria-hidden=\"true\" style=\"vertical-align: -0.505ex; width:7.346ex; height:2.343ex;\" alt=\"{\\displaystyle N-M}\"\/><\/span> candidates.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Modules_in_NIMS-OS\">Modules in NIMS-OS<\/span><\/h2>\n<p>In this section, we introduce the modules included in NIMS-OS for AI algorithms and robotic systems. In the present work, we prepared four and two types of modules as AI algorithms and robotic systems, respectively.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"AI_algorithms\">AI algorithms<\/span><\/h3>\n<p>To select promising experimental conditions, three types of AI algorithms are implemented as standard in NIMS-OS. In addition, random exploration can be selected. Each algorithm is briefly explained in this subsection. In the future, more algorithms will be made available.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Bayesian_optimization:_PHYSBO\">Bayesian optimization: PHYSBO<\/span><\/h3>\n<p>BO is an optimization technique using <a href=\"https:\/\/www.limswiki.org\/index.php\/Machine_learning\" title=\"Machine learning\" class=\"wiki-link\" data-key=\"79aab39cfa124c958cd1dbcab3dde122\">machine learning<\/a> (ML) prediction. In this method, by using Gaussian process regression, the value of an objective function is predicted when the experimental conditions are input. The next promising experimental conditions are then selected based on the prediction values. Here, because the Gaussian process can evaluate not only the mean value of the prediction but also its variance, an acquisition function defined by mean and variance can be used to make the selection. In NIMS-OS, BO can be performed using the Python package PHYSBO.<sup id=\"rdp-ebb-cite_ref-:2_32-1\" class=\"reference\"><a href=\"#cite_note-:2-32\">[32]<\/a><\/sup> PHYSBO supports single- and multi-objective optimizations, and multiple proposals are calculated. Note that the number of objective functions is recommended to be no more than three due to excessively large computational time with higher values. In NIMS-OS, Thompson sampling is used to define the acquisition function for rapid calculation. The key point in using PHYSBO is that the exploration is performed to maximize the objective functions. Thus, if a material with the smaller properties is explored, we need to add a negative value to the objective functions.\n<\/p>\n<h4><span class=\"mw-headline\" id=\"Boundless_objective-free_exploration:_BLOX\">Boundless objective-free exploration: BLOX<\/span><\/h4>\n<p>BLOX is a Python package that performs boundless objective-free exploration. It is based on an algorithm designed to select the next experimental conditions, to perform uniform sampling in the space of the objective functions. For materials science, curious materials can be found using BLOX. Specifically, BLOX trains ML models to predict objective functions from experimental conditions. Experimental conditions that realize uniform sampling in the space of objective functions are found based on the Stein discrepancy evaluated using the prediction results. In NIMS-OS, a modified version of the BLOX algorithm that can propose multiple candidates is implemented. To select multiple candidates, after the experimental condition with the largest Stein discrepancy is selected, another condition is selected when the predicted values of the selected condition are regarded as a correct value. This procedure is iterated, and we obtain multiple proposals. In NIMS-OS, random forest regression is used as a prediction model. Although BLOX can handle any number of objective functions, it is recommended that the number of the objective functions be limited to three or four, because exploration in more dimensions requires more time. BLOX has been used to search chemical spaces<sup id=\"rdp-ebb-cite_ref-:3_33-1\" class=\"reference\"><a href=\"#cite_note-:3-33\">[33]<\/a><\/sup> and to explore superhard materials.<sup id=\"rdp-ebb-cite_ref-39\" class=\"reference\"><a href=\"#cite_note-39\">[39]<\/a><\/sup>\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Phase_diagram_construction:_PDC\">Phase diagram construction: PDC<\/span><\/h3>\n<p>PDC is a Python package that can create a detailed phase diagram with a small number of experiments. To investigate a phase diagram, PDC proposes promising experimental conditions for the next experiment by using active learning. Specifically, uncertainty sampling based on the label propagation method finds uncertain points in the phase diagram, and these uncertain points are proposed for the next experiments. PDC was developed to propose multiple experimental conditions for batch experiments.<sup id=\"rdp-ebb-cite_ref-40\" class=\"reference\"><a href=\"#cite_note-40\">[40]<\/a><\/sup> In NIMS-OS, the least confident score is used as an uncertainty score to evaluate uncertain points. Note that, for PDC, the objective function is the phase name or an index of phases, and thus only a one-dimensional objective function can be specified in the candidates file. PDC has been used to create new phase diagrams for the growth conditions of thin film<sup id=\"rdp-ebb-cite_ref-41\" class=\"reference\"><a href=\"#cite_note-41\">[41]<\/a><\/sup> and to determine large and small areas of creep phenomena in polymer materials.<sup id=\"rdp-ebb-cite_ref-42\" class=\"reference\"><a href=\"#cite_note-42\">[42]<\/a><\/sup>\n<\/p>\n<h4><span class=\"mw-headline\" id=\"Random_exploration:_RE\">Random exploration: RE<\/span><\/h4>\n<p>In RE, the next candidate experimental condition is selected randomly. This approach can be used to generate initial data before executing AI algorithms when no experimental data have yet been recorded. Furthermore, it can also be used to generate data for comparison as new AI algorithms can be developed.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Robotic_experiments\">Robotic experiments<\/span><\/h3>\n<p>The module for robotic experiments comprises two Python scripts. The first script creates input files for robotic experiments according to the experimental conditions selected by the AI and commands a robot to begin the experiment. The second script analyzes the experimental results when the experiments are finished and updates the candidates file. At present, two types of modules are implemented in NIMS-OS: STAN and NAREE.\n<\/p>\n<h4><span class=\"mw-headline\" id=\"Standard_module_for_robotic_experiments:_STAN\">Standard module for robotic experiments: STAN<\/span><\/h4>\n<p>STAN is a virtual implementation of the procedure for conducting robotic experiments. Thus, NIMS-OS can be run virtually using this module, even without a robotic device. In this module, the following steps are executed:\n<\/p>\n<ol><li>Create the input files for the robotic experiments in an appropriate folder according to the experimental conditions selected by the AI. In this standard module, we simply create a text file with a date as its name.<\/li>\n<li>Send a signal to the robotic system to begin the experiments. Depending on the machine, various cases can be considered, such as sending a start signal via serial communication. In this standard module, we assume that the experiments are begun by storing the inputend.txt file in the specified folder.<\/li>\n<li>Wait until the robotic experiments are completed. This step includes various operations, such as receiving signals from the robot when the experiment is finished. This standard module assumes that the robot outputs outputend.txt file to indicate that the experiment is finished, and NIMS-OS continues waiting until this file appears.<\/li>\n<li>Read the files of experimental results and extract the values of objective functions. Here, the case of simply reading results.csv, which contains the objective function values, is implemented.<\/li>\n<li>Update the candidates file according to the values extracted in (4).<\/li><\/ol>\n<p>Steps (1) and (2) are performed by preparation_input.py, and analysis_output.py conducts steps (3)-(5). In practice, for use with actual robotic systems, new modules can be created according to this standard module.\n<\/p><p>Additionally, this module can also facilitate closed-loop materials exploration between AI and experiments for processes that are time-consuming and cannot be partially automated. The procedure is as follows: When the proposals.csv file is generated, NIMS-OS automatically enters a sleep mode until experimental results are obtained. Based on the information in proposals.csv, the corresponding manual experiments are conducted. Once the objective function values are obtained through the experiments, a results.csv file is created, containing the objective function values corresponding to each line in proposals.csv. The results.csv file, along with an empty file named outputend.txt, is stored in the specified folder where the experimental results are output. Subsequently, NIMS-OS restarts and generates a new proposals.csv file.\n<\/p>\n<h4><span id=\"rdp-ebb-NIMS_automated_robotic_electrochemical_experiments_(NAREE)_system:_NAREE\"><\/span><span class=\"mw-headline\" id=\"NIMS_automated_robotic_electrochemical_experiments_.28NAREE.29_system:_NAREE\">NIMS automated robotic electrochemical experiments (NAREE) system: NAREE<\/span><\/h4>\n<p>As a robotic system for materials science, the NIMS Automated Robotic Electrochemical Experiments (NAREE) system<sup id=\"rdp-ebb-cite_ref-:0_13-3\" class=\"reference\"><a href=\"#cite_note-:0-13\">[13]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:4_35-1\" class=\"reference\"><a href=\"#cite_note-:4-35\">[35]<\/a><\/sup> can be used in NIMS-OS. NAREE comprises a liquid-handling dispenser, an electrochemical measurement unit, and a robotic arm. By using a microplate-based electrochemical cell equipped with electrodes, the performance of electrolytes prepared by mixing solution by a liquid handling dispenser is electrochemically evaluated in a high-throughput manner. This module was developed according to the procedures of the previously described STAN.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Usage_of_the_NIMS-OS_Python_version\">Usage of the NIMS-OS Python version<\/span><\/h2>\n<h3><span class=\"mw-headline\" id=\"Install\">Install<\/span><\/h3>\n<p>NIMS-OS is written in Python3 programming language (version 3.6 or higher is required), and it can be installed via PyPI as follows:\n<\/p>\n<dl><dd><tt>$ python3 -m pip install nimsos<\/tt><\/dd><\/dl>\n<p>If this installation is successful, the following packages are also installed or updated automatically:\n<\/p>\n<ul><li>Cython<\/li>\n<li>matplotlib<\/li>\n<li>numpy<\/li>\n<li>physbo<\/li>\n<li>scikit-learn<\/li>\n<li>scipy<\/li><\/ul>\n<h3><span class=\"mw-headline\" id=\"Basic_usage\">Basic usage<\/span><\/h3>\n<p>We show a small example program (Program 1) in which PHYSBO is performed. In this program, assuming no experimental results in the candidates file, random exploration is performed in the first cycle.\n<\/p>\n<h4><span class=\"mw-headline\" id=\"Assignment_of_parameters_and_candidates_file\">Assignment of parameters and candidates file<\/span><\/h4>\n<p>First, the parameters for closed-loop experiments are defined. For example, when the number of objective functions is two, the number of proposals for each cycle is two, and the number of cycles is three. We define this in the code as follows:\n<\/p>\n<dl><dd><tt>ObjectivesNum\u2009=\u20092ProposalsNum\u2009=\u20092CyclesNum\u2009=\u20093<\/tt><\/dd><\/dl>\n<p>Next, we specify a .csv file containing the candidates of experimental conditions, which is prepared as described under \"Preparation of candidates for experimental conditions.:\n<\/p>\n<dl><dd><tt>candidates_file\u2009=\u2009\u201c.\/candidates.csv\u201d<\/tt><\/dd><\/dl>\n<p>The name of the file that will contain the experimental conditions selected by the AI is as follows:\n<\/p>\n<dl><dd><tt>proposals_file\u2009=\u2009\u201c.\/proposals.csv\u201d<\/tt><\/dd><\/dl>\n<p>We specify the folder name where the input files for the robotic experiments are stored and the folder name where the results from the experiments are output, respectively, as follows:\n<\/p>\n<dl><dd><tt>input_folder\u2009=\u2009\u201c.\/EXPInput\u201d<\/tt><br \/><\/dd>\n<dd><tt>output_folder\u2009=\u2009\u201c.\/EXPOutput\u201d<\/tt><\/dd><\/dl>\n<h4><span class=\"mw-headline\" id=\"Execution_of_AI\">Execution of AI<\/span><\/h4>\n<p><tt>nimsos.selection<\/tt> is a class to select the next experimental conditions with the help of the AI. For example, <tt>nimsos.selection<\/tt> is used as follows:\n<\/p>\n<dl><dd><tt>nimsos.selection(method\u2009=\u2009\u201cPHYSBO\u201d,<\/tt><\/dd><tt>\n<dd>input_file\u2009=\u2009candidates_file,<\/dd><\/tt><\/dl><tt><dl><dd>output_file\u2009=\u2009proposals_file,<\/dd><\/dl><\/tt><tt><\/tt><dl><tt><dd>\u2003num_objectives\u2009=\u2009ObjectivesNum,<\/dd>\n<\/tt><dd><tt>num_proposals\u2009=\u2009ProposalsNum)<\/tt><\/dd><\/dl>\n<p>The parameters of the method in this class (Program 1) indicate the module for AI algorithms. For the method, \"PHYSBO\" (Bayesian optimization), \"BLOX\" (objective free search), \"PDC\" (phase diagram construction), and \"RE\" (random exploration) are specified. The experimental conditions are selected from the data without the values of objective functions among <tt>input_file<\/tt>. In addition, selected conditions are outputted to <tt>output_file<\/tt>. For <tt>num_objectives<\/tt>, the number of objectives is input, and the number of proposals is specified as <tt>num_proposals<\/tt>. In general, although many hyperparameters should be considered to use the AI, they are determined automatically in NIMS-OS. Note that if there are no experimental results in the candidates file, only \"RE\" is used. For \"PHYSBO,\" \"BLOX,\" and \"PDC,\" some values of objective functions must be stored in the candidates file.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Prog1_Tamura_SciTechAdvMatMeth2023_3-1.jpeg\" class=\"image wiki-link\" data-key=\"b8c39d672883216e2a2f1d22005c6c3a\"><img alt=\"Prog1 Tamura SciTechAdvMatMeth2023 3-1.jpeg\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/1\/1e\/Prog1_Tamura_SciTechAdvMatMeth2023_3-1.jpeg\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Program 1.<\/b> Small example of NIMS-OS for Bayesian optimization.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h4><span class=\"mw-headline\" id=\"Preparation_of_input_files_for_robotic_experiments_and_execution_of_experiments\">Preparation of input files for robotic experiments and execution of experiments<\/span><\/h4>\n<p><tt>nimsos.preparation_input<\/tt> is a class to prepare the input files for robotic experiments and send the start message to the robot. For example, <tt>nimsos.preparation_input<\/tt> is used as follows.\n<\/p>\n<dl><dd><tt>nimsos.preparation_input(machine\u2009=\u2009\u201cSTAN\u201d,<\/tt><\/dd><tt>\n<dd>input_file\u2009=\u2009proposals_file,<\/dd>\n<\/tt><dd><tt>input_folder\u2009=\u2009input_folder)<\/tt><\/dd><\/dl>\n<p>The parameter of <tt>machine<\/tt> selects the module of robotic experiments. For <tt>machine<\/tt>, \"STAN,\" which is the standard module for this procedure, and \"NAREE\" (NIMS automated robotic electrochemical experiments) are used. For <tt>input_file<\/tt>, the experimental conditions selected by the AI are specified. In addition, the folder in the computer where the input files for robotic experiments are stored is referred to as the <tt>input_folder<\/tt>. In the <tt>nimsos.preparation_input module<\/tt>, the two functions <tt>make_machine_file()<\/tt> and <tt>send_message_machine()<\/tt> should be modified depending on the robotic systems used. The former creates the input files for robotic experiments from selected experimental conditions, whereas the latter sends the message to begin the robotic experiments.\n<\/p>\n<h4><span class=\"mw-headline\" id=\"Analysis_of_output_files_from_experiments_and_update_of_candidates_file\">Analysis of output files from experiments and update of candidates file<\/span><\/h4>\n<p><tt>nimsos.analysis_output<\/tt> is a class used to analyze the experimental results and update the candidates file. For example, <tt>nimsos.analysis_output<\/tt> is used as follows:\n<\/p>\n<dl><dd><tt>nimsos.analysis_output(machine\u2009=\u2009\u201cSTAN\u201d,<\/tt><\/dd><tt>\n<dd>input_file\u2009=\u2009proposals_file,<\/dd><\/tt><\/dl><tt><dl><dd>\u2003\u2009output_file\u2009=\u2009candidates_file,<\/dd><\/dl><\/tt><tt><\/tt><dl><tt><dd>\u2003\u2003num_objectives\u2009=\u2009ObjectivesNum,<\/dd>\n<\/tt><dd><tt>\u2003\u2009output_folder\u2009=\u2009output_folder)<\/tt><\/dd><\/dl>\n<p>The parameter of <tt>machine<\/tt> is the same as that found in the <tt>nimsos.preparation_input<\/tt> module, which selects the module for robotic experiments. Here, \"STAN\" and \"NAREE\" can be selected. For <tt>input_file<\/tt>, the experimental conditions selected by the AI are specified, and output_file is the name of the candidates file. The file specified by <tt>output_file<\/tt> is updated by this module. In addition, for <tt>num_objectives<\/tt>, the number of objectives is input. For <tt>output_folder<\/tt>, the folder in the computer where the results from robotic experiments are output is specified. In the <tt>nimsos.analysis_output<\/tt> module, two functions <tt>extract_objectives()<\/tt> and <tt>recieve_exit_message()<\/tt> should be modified depending on the robot systems. The former extracts the values of objective functions from the output files of robotic experiments, and the latter receives the message when the robotic experiments are finished. If \"NAREE\" is selected, <tt>objectives_info<\/tt> should be specified as a dictionary indicating which objective function is extracted from the experimental results.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Visualization_of_the_results\">Visualization of the results<\/span><\/h3>\n<p>By using <tt>nimsos.visualization<\/tt>, the figures of the results are obtained. When this module is used, the new folder named \"fig\" is prepared in advance in the same folder where the main script is stored. The figures are output to this folder. <tt>nimsos.visualization.plot_history<\/tt> and <tt>nimsos.visualization.plot_distribution.plot<\/tt> create figures for the history and distributions of objective functions, respectively. These modules are useful when using AI algorithms other than PDC. In contrast, <tt>nimsos.visualization.plot_phase_diagram.plot<\/tt> creates the predicted phase diagram when PDC is used as an AI algorithm.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Usage_of_the_NIMS-OS_GUI_version\">Usage of the NIMS-OS GUI version<\/span><\/h2>\n<p>A GUI version of NIMS-OS has been developed for easy execution, which is available at <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/github.com\/nimsos-dev\/nimsos-gui\" target=\"_blank\">https:\/\/github.com\/nimsos-dev\/nimsos-gui<\/a>. This can be used after installing the required Python version, as described in the prior section, and performing the installation as described in the manual (<a rel=\"external_link\" class=\"external free\" href=\"https:\/\/nimsos-dev.github.io\/nimsos\/docs\/en\/index.html\" target=\"_blank\">https:\/\/nimsos-dev.github.io\/nimsos\/docs\/en\/index.html<\/a>). Figure 4 shows the operation screen of the NIMS-OS GUI version. In this GUI version, the name of the candidates file is fixed to candidates.csv, and the name of the proposals file is fixed to proposals.csv. \n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig4_Tamura_SciTechAdvMatMeth2023_3-1.jpeg\" class=\"image wiki-link\" data-key=\"39d0392c6fef44ada5602074387c97e5\"><img alt=\"Fig4 Tamura SciTechAdvMatMeth2023 3-1.jpeg\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/d\/d1\/Fig4_Tamura_SciTechAdvMatMeth2023_3-1.jpeg\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 4.<\/b> Operation screen of the NIMS-OS GUI version.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>The execution procedure is as follows:\n<\/p>\n<ol><li>Specify the number of objectives, proposals, and cycles in the \"Parameters\" section.<\/li>\n<li>Select the method to be used in the AI algorithm section. If we use a newly created module for AI method named \"ai_tool_original.py,\" click on Original.<\/li>\n<li>Select the robotic system in the Robotic system section. If we use a newly created module for robotic systems named \"preparation_input_original.py\" and \"analysis_output_original.py,\" click on Original.<\/li>\n<li>Press the \"Run\" button on the \"Controller\" section to begin automated materials exploration.<\/li><\/ol>\n<p>When NIMS-OS is started, the Cycle counter begins to operate. Furthermore, in the \"Time\" section, the amount of time required to execute the AI algorithm and a single cycle are measured, and the remaining time is also output. The standard output of the Python version is displayed in real time in the \"Results\" section, and these output results can be saved as a file by pressing the Output button. In addition, to pause the automated exploration, the user can press the \"Stop\" button of the \"Controller\" section. Note that pressing this button does not stop the process immediately, but when the candidates file is updated, NIMS-OS is stopped. To reset the settings, press the \"Reset\" button on the Controller. The operation with NAREE is shown as a video (Supplemental Movie 1). Note that even for processes that cannot be partially automated, the closed loop between AI and manual experiments can still be achieved by selecting STAN in the \"Robotic system\" section, as explained prior.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Application\">Application<\/span><\/h2>\n<p>To demonstrate the effectiveness of NIMS-OS for the application of automated robotic experiments, we applied the NIMS-OS for NAREE system and performed an exploration for multi-component electrolytes that maximize the performance of lithium metal electrode. The anode-free type microplate based electrochemical cells were fabricated using LiFePO<sub>4<\/sub> as positive electrode and Cu foil as negative electrode. The cells were subjected to a charging process with capacity limitation of 0.05 mAh. After that, the cells were subjected to a discharge process. Here, we defined the discharge time as a one-dimensional objective function. In this case, the longer discharge time represents the better battery performance (higher capacity). Using such an experimental setup, a combination of electrolyte additives was optimized to maximize the discharge time. Five different additives were selected from a list of 16 compounds (Table 1) and injected into an electrochemical cell containing 1\u2009M LiTFSI in TEGDME. In this case, the number of candidates for combination of electrolyte additives is <span class=\"mwe-math-element\"><span class=\"mwe-math-mathml-inline mwe-math-mathml-a11y\" style=\"display: none;\"><\/span><img src=\"https:\/\/en.wikipedia.org\/api\/rest_v1\/media\/math\/render\/svg\/d5fdb6529681965a5f1bf3b8cd8ed990851c22c6\" class=\"mwe-math-fallback-image-inline\" aria-hidden=\"true\" style=\"vertical-align: -0.671ex; width:13.568ex; height:2.509ex;\" alt=\"{\\displaystyle ~_{16}C_{5}=4,\\!368}\"\/><\/span>. The candidate files for this experiment were prepared in a similar manner as shown in Figure 3 (combination of materials). In our experiment, 32 electrochemical cells were prepared in one microplate and 32 experiments were performed in parallel for two\u2009hours.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"3\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 1.<\/b> List of 16 types of additives used in an automated exploration for new electrolytes using the NAREE system. For all additives, the solvent is fixed as TEGDME.\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">ID\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Additive\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Concentration\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">lithium bis(pentafluoroethanesulfonyl)imide (LiBETI)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">LiPF<sub>6<\/sub>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">LiBF<sub>4<\/sub>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">4\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">lithium bis(trifluoro methanesulfonyl)imide (LiTFSI)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">5\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">LiTfO\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">6\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">LiClO<sub>4<\/sub>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">7\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">lithium bis(oxalate)borate (LiBOB)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">10\u2009mM\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">8\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">LiAsF<sub>6<\/sub>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">10\u2009mM\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">9\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">LiF\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">10\u2009mM\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">10\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">N-methyl-2-pyrrodione (NMP)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.%\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">11\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">sulfolane\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.%\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">12\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">dimethyl sulfoxide (DMSO)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.%\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">13\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">propylene carbonate (PC)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.%\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">14\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">ethylene carbonate (EC)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.%\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">15\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">fluoroethylene carbonate (FEC)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.%\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">16\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">vinylene carbonate (VC)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.%\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>For the autonomous experiments for searching multi-component electrolytes using the NAREE system operated by NIMS-OS, at first, 32 parallel experiments (one microplate) were performed by random exploration using RE because we do not have initial data at this stage. After obtaining the initial data by RE, the next five cycles of experiments (five microplates) were performed by BO using PHYSBO. Notably, a fully automated experiment was continuously conducted without any human intervention for 10\u2009hours. After that, addition six cycles of experiments (six microplates) were also performed by BO. In total, 384 experiments were performed. The obtained results can be visualized by using <tt>nimsos.visualization<\/tt> in the Python version of NIMS-OS, and the time course of the objective function and the histogram distribution of the results in the total 384 experiments were summarized in Figure 5. The results clearly revealed that the best electrolyte composition was discovered at the seventh experimental cycle. In Table 2, the details of electrolyte composition for the top 10 samples that enhanced the discharge time were summarized. The electrolyte\u2014containing 100\u2009mM LiPF<sub>6<\/sub>, 100\u2009mM LiTFSI, 2 vol.% PC, 2 vol.% FEC, and 2 vol.% VC\u2014exhibits the highest discharge time of 1,439.09 seconds. It should be noted that the possible maximum discharge time is 1,800 seconds since the current density during discharge was set to 0.1\u2009mA. Thus, there is still much room for improvement of battery performance. In addition, there can be seen that most of the top 10 samples contain VC and\/or FEC. These results are essentially consistent with the knowledge in this field that VC and FEC have a positive effect for improving the performance of the lithium metal electrode.<sup id=\"rdp-ebb-cite_ref-43\" class=\"reference\"><a href=\"#cite_note-43\">[43]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-44\" class=\"reference\"><a href=\"#cite_note-44\">[44]<\/a><\/sup>\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig5_Tamura_SciTechAdvMatMeth2023_3-1.jpeg\" class=\"image wiki-link\" data-key=\"d9e62762527f638fb8c94f0fdef1d600\"><img alt=\"Fig5 Tamura SciTechAdvMatMeth2023 3-1.jpeg\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/3\/34\/Fig5_Tamura_SciTechAdvMatMeth2023_3-1.jpeg\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 5.<\/b> Output results from NIMS-OS for automated exploration for electrolytes using the NAREE system: (a) history_step.png and (b) history_best.png by <tt>nimsos.Visualization.plot_history<\/tt> and (c) distribution.png by <tt>nimsos.Visualization.plot_distribution.plot<\/tt>. The target property is the discharge time and its unit is seconds. In the first cycle, RE is used to generate initial states. After the second cycle, PHYSBO is used.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"8\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 2.<\/b> Top 10 compositions that enhanced the discharge time. The found cycle number is also shown.\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Ranking\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Additive 1\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Additive 2\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Additive 3\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Additive 4\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Additive 5\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Discharge time<br \/>(seconds)\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Found cycle\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM LiPF<sub>6<\/sub>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM LiTFSI\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.% PC\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.% FEC\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.% VC\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1,439.09\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Seventh\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM LiBETI\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM LiTfO\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">10\u2009mM LiBOB\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.% EC\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.% FEC\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1,401.97\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Twelfth\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.% NMP\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.% sulfolane\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.% DMSO\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.% PC\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.% FEC\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1,374.86\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Ninth\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">4\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM LiBF<sub>4<\/sub>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM LiTFSI\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM LiTfO\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">10\u2009mM LiF\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.% FEC\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1,365.57\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Ninth\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">5\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM LiBETI\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM LiBF4<sub>4<\/sub>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">10\u2009mM LiBOB\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.% PC\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.% FEC\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1,364.32\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Twelfth\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">6\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM LiTFSI\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM LiTfO\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">10\u2009mM LiAsF<sub>6<\/sub>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.% FEC\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.% VC\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1,358.99\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Tenth\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">7\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM LiBETI\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">10\u2009mM LiBOB\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">10\u2009mM LiF\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.% EC\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.% VC\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1,357.43\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Tenth\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">8\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM LiBETI\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM LiTFSI\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">10\u2009mM LiBOB\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">10\u2009mM LiF\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.% FEC\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1,356.39\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Ninth\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">9\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM LiBF<sub>4<\/sub>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM LiTFSI\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM LiClO<sub>4<\/sub>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.% sulfolane\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.% VC\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1,347.23\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Eleventh\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">10\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM LiBF<sub>6<\/sub>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\u2009mM LiTfO\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">10\u2009mM LiAsF<sub>6<\/sub>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">10\u2009mM LiF\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2 vol.% EC\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1,346.72\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Seventh\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h2><span class=\"mw-headline\" id=\"Conclusion\">Conclusion<\/span><\/h2>\n<p>In this study, we developed NIMS-OS to implement a closed loop of AI and robotic experiments for automated materials exploration. We anticipate that this software can serve as a generic control system. To use NIMS-OS, a candidates file listing experimental conditions as a materials search space should be prepared in advance. This allows various problems for automated materials exploration to be commonly performed in NIMS-OS. Establishing standards for automated materials exploration is a key advantages of such generic control software. Using NIMS-OS and the NAREE system, we also demonstrated an example of automatic exploration for electrolytes.\n<\/p><p>The compatibility with original robotic systems other than NAREE is discussed. We believe that the most crucial aspects of integrating other robotic systems lie in providing instructions to initiate the robot and determining the completion of the robotic experiment. Regarding the former, the current NAREE system is designed to automatically start an experiment when an input file is stored in a specified folder. Therefore, by making slight modifications to the existing Python script, original robotic systems with this functionality can be easily integrated into NIMS-OS. Even if the PC controlling the experimental system and the PC running the NIMS-OS are different, the robot can be started by sharing the specified folder using a file-sharing service or network-attached storage (NAS). However, if the experimental systems require voltage signal control or <a href=\"https:\/\/www.limswiki.org\/index.php\/Application_programming_interface\" title=\"Application programming interface\" class=\"wiki-link\" data-key=\"36fc319869eba4613cb0854b421b0934\">application programming interface<\/a> (API) control, specific Python code needs to be developed. The development of Python code for voltage signal control or API control is considered a future prospect. Regarding the latter, the experimental results will always be output in the specified folder. Therefore, it is sufficient to determine whether the result files have been generated or not, even if the robotic systems are changed. Furthermore, there may be cases where the experimental system can only be fully controlled by the GUI-driven software. In such cases, it is necessary to manually press a button on the GUI control screen to initiate the robot. However, an automated closed loop can be achieved by installing Robotic Process Automation (RPA) on the PC that controls the experimental system.<sup id=\"rdp-ebb-cite_ref-:1_15-1\" class=\"reference\"><a href=\"#cite_note-:1-15\">[15]<\/a><\/sup> For example, with RPA, the following operations can be performed: (i) identifying the presence of an input file in the specified folder, (ii) providing instructions to initiate the experiments on the GUI operation screen that controls the experimental system, and (iii) deleting the input file once it is confirmed that the experimental results have been generated. Thus, we believe that the current NIMS-OS is designed to be easily adaptable to a variety of original robotic systems in materials science.\n<\/p><p>At present, NIMS-OS does not include a sufficient set of available AI algorithms and robotic experimental systems. For the further growth of this OS, developing and releasing more modules for various AI algorithms and robotic systems will be essential. The NAREE system used in this study can perform sequential operations since all evaluations of proposed experimental conditions by robotic experiments are completed within the same timeframe. However, in realistic experiments, the costs associated with synthesis, device fabrication, and evaluation strongly depend on the specific experimental conditions. In such cases, waiting for all experiments to be completed would be inefficient. To address this issue, BO introduces the concept of asynchronous parallel global optimization.<sup id=\"rdp-ebb-cite_ref-45\" class=\"reference\"><a href=\"#cite_note-45\">[45]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-46\" class=\"reference\"><a href=\"#cite_note-46\">[46]<\/a><\/sup> Therefore, it is necessary to develop a module within NIMS-OS that can facilitate asynchronous parallel optimization. Furthermore, in automated materials exploration, a greater amount of experimental data is generated compared to human experiments. Thus, the ability to store, share, and utilize experimental data for secondary purposes should be implemented as extensions in NIMS-OS. Specifically, a module that facilitates the automatic transfer of data to external storage or data repositories will be essential in enhancing data sharing and utilization. We will continue to enhance the extensions available for NIMS-OS to develop it as a game changer for digital transformation (DX) in materials science.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Supplemental_material\">Supplemental material<\/span><\/h2>\n<ul><li><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.tandfonline.com\/doi\/suppl\/10.1080\/27660400.2023.2232297\/suppl_file\/tstm_a_2232297_sm2310.mp4\" target=\"_blank\">Supplemental Movie 1<\/a> (.mp4; 12,302 KB)<\/li><\/ul>\n<h2><span id=\"rdp-ebb-Abbreviations,_acronyms,_and_initialisms\"><\/span><span class=\"mw-headline\" id=\"Abbreviations.2C_acronyms.2C_and_initialisms\">Abbreviations, acronyms, and initialisms<\/span><\/h2>\n<ul><li><b>AI<\/b>: artificial intelligence<\/li>\n<li><b>BO<\/b>: Bayesian optimization<\/li>\n<li><b>BLOX<\/b>: BoundLess Objective-free eXploration<\/li>\n<li><b>DMSO<\/b>: dimethyl sulfoxide<\/li>\n<li><b>EC<\/b>: ethylene carbonate<\/li>\n<li><b>FEC<\/b>: fluoroethylene carbonate<\/li>\n<li><b>GUI<\/b>: graphical user interface<\/li>\n<li><b>LiBETI<\/b>: lithium bis(pentafluoroethanesulfonyl)imide<\/li>\n<li><b>LiBOB<\/b>: lithium bis(oxalate)borate<\/li>\n<li><b>LiTFSI<\/b>: lithium bis(trifluoro methanesulfonyl)imide<\/li>\n<li><b>ML<\/b>: machine learning<\/li>\n<li><b>NAREE<\/b>: NIMS Automated Robotic Electrochemical Experiments<\/li>\n<li><b>NIMS-OS<\/b>: NIMS Orchestration System<\/li>\n<li><b>NMP<\/b>: N-methyl-2-pyrrodione<\/li>\n<li><b>PC<\/b>: propylene carbonate<\/li>\n<li><b>PDC<\/b>: phase diagram construction<\/li>\n<li><b>PHYSBO<\/b>: Optimization Tools for PHYSics Based on Bayesian Optimization<\/li>\n<li><b>RE<\/b>: random exploration<\/li>\n<li><b>STAN<\/b>: STANdard module<\/li>\n<li><b>VC<\/b>: vinylene carbonate<\/li><\/ul>\n<h2><span class=\"mw-headline\" id=\"Acknowledgements\">Acknowledgements<\/span><\/h2>\n<p>The authors thank Masahiko Demura, Hideki Yoshikawa, and Masanobu Naito for valuable discussions. The authors also thank Kazuha Nakamura for experimental contributions, and thank Satoshi Murata, Daisuke Ryuno, and Hiromichi Taketa for the development of NIMS-OS.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Funding\">Funding<\/span><\/h3>\n<p>The work was supported by the MEXT Program: Data Creation and Utilization-Type Material Research and Development Project [JPMXP1122712807].\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Correction_statement\">Correction statement<\/span><\/h3>\n<p>This article has been republished with minor changes. These changes do not impact the academic content of the article.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Supplementary_data\">Supplementary data<\/span><\/h3>\n<p>Supplemental data for this article can be accessed online at <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/doi.org\/10.1080\/27660400.2023.2232297\" target=\"_blank\">https:\/\/doi.org\/10.1080\/27660400.2023.2232297<\/a>.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Conflict_of_interest\">Conflict of interest<\/span><\/h3>\n<p>No potential conflict of interest was reported by the author(s).\n<\/p>\n<h2><span class=\"mw-headline\" id=\"References\">References<\/span><\/h2>\n<div class=\"reflist references-column-width\" style=\"-moz-column-width: 30em; -webkit-column-width: 30em; column-width: 30em; list-style-type: decimal;\">\n<div class=\"mw-references-wrap mw-references-columns\"><ol class=\"references\">\n<li id=\"cite_note-1\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-1\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">White, Ashley (1 August 2012). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1557\/mrs.2012.194\" target=\"_blank\">\"The Materials Genome Initiative: One year on\"<\/a> (in en). <i>MRS Bulletin<\/i> <b>37<\/b> (8): 715\u2013716. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1557%2Fmrs.2012.194\" target=\"_blank\">10.1557\/mrs.2012.194<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0883-7694\" target=\"_blank\">0883-7694<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1557\/mrs.2012.194\" target=\"_blank\">http:\/\/link.springer.com\/10.1557\/mrs.2012.194<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+Materials+Genome+Initiative%3A+One+year+on&rft.jtitle=MRS+Bulletin&rft.aulast=White&rft.aufirst=Ashley&rft.au=White%2C%26%2332%3BAshley&rft.date=1+August+2012&rft.volume=37&rft.issue=8&rft.pages=715%E2%80%93716&rft_id=info:doi\/10.1557%2Fmrs.2012.194&rft.issn=0883-7694&rft_id=http%3A%2F%2Flink.springer.com%2F10.1557%2Fmrs.2012.194&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-2\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-2\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ramprasad, Rampi; Batra, Rohit; Pilania, Ghanshyam; Mannodi-Kanakkithodi, Arun; Kim, Chiho (13 December 2017). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41524-017-0056-5\" target=\"_blank\">\"Machine learning in materials informatics: recent applications and prospects\"<\/a> (in en). <i>npj Computational Materials<\/i> <b>3<\/b> (1): 54. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41524-017-0056-5\" target=\"_blank\">10.1038\/s41524-017-0056-5<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2057-3960\" target=\"_blank\">2057-3960<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41524-017-0056-5\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41524-017-0056-5<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Machine+learning+in+materials+informatics%3A+recent+applications+and+prospects&rft.jtitle=npj+Computational+Materials&rft.aulast=Ramprasad&rft.aufirst=Rampi&rft.au=Ramprasad%2C%26%2332%3BRampi&rft.au=Batra%2C%26%2332%3BRohit&rft.au=Pilania%2C%26%2332%3BGhanshyam&rft.au=Mannodi-Kanakkithodi%2C%26%2332%3BArun&rft.au=Kim%2C%26%2332%3BChiho&rft.date=13+December+2017&rft.volume=3&rft.issue=1&rft.pages=54&rft_id=info:doi\/10.1038%2Fs41524-017-0056-5&rft.issn=2057-3960&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41524-017-0056-5&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-3\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-3\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Schmidt, Jonathan; Marques, M\u00e1rio R. G.; Botti, Silvana; Marques, Miguel A. L. (8 August 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41524-019-0221-0\" target=\"_blank\">\"Recent advances and applications of machine learning in solid-state materials science\"<\/a> (in en). <i>npj Computational Materials<\/i> <b>5<\/b> (1): 83. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41524-019-0221-0\" target=\"_blank\">10.1038\/s41524-019-0221-0<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2057-3960\" target=\"_blank\">2057-3960<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41524-019-0221-0\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41524-019-0221-0<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Recent+advances+and+applications+of+machine+learning+in+solid-state+materials+science&rft.jtitle=npj+Computational+Materials&rft.aulast=Schmidt&rft.aufirst=Jonathan&rft.au=Schmidt%2C%26%2332%3BJonathan&rft.au=Marques%2C%26%2332%3BM%C3%A1rio+R.+G.&rft.au=Botti%2C%26%2332%3BSilvana&rft.au=Marques%2C%26%2332%3BMiguel+A.+L.&rft.date=8+August+2019&rft.volume=5&rft.issue=1&rft.pages=83&rft_id=info:doi\/10.1038%2Fs41524-019-0221-0&rft.issn=2057-3960&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41524-019-0221-0&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-4\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-4\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ling, Chen (18 February 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41524-022-00713-x\" target=\"_blank\">\"A review of the recent progress in battery informatics\"<\/a> (in en). <i>npj Computational Materials<\/i> <b>8<\/b> (1): 33. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41524-022-00713-x\" target=\"_blank\">10.1038\/s41524-022-00713-x<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2057-3960\" target=\"_blank\">2057-3960<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41524-022-00713-x\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41524-022-00713-x<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+review+of+the+recent+progress+in+battery+informatics&rft.jtitle=npj+Computational+Materials&rft.aulast=Ling&rft.aufirst=Chen&rft.au=Ling%2C%26%2332%3BChen&rft.date=18+February+2022&rft.volume=8&rft.issue=1&rft.pages=33&rft_id=info:doi\/10.1038%2Fs41524-022-00713-x&rft.issn=2057-3960&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41524-022-00713-x&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-5\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-5\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Terayama, Kei; Sumita, Masato; Tamura, Ryo; Tsuda, Koji (16 March 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acs.accounts.0c00713\" target=\"_blank\">\"Black-Box Optimization for Automated Discovery\"<\/a> (in en). <i>Accounts of Chemical Research<\/i> <b>54<\/b> (6): 1334\u20131346. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1021%2Facs.accounts.0c00713\" target=\"_blank\">10.1021\/acs.accounts.0c00713<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0001-4842\" target=\"_blank\">0001-4842<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acs.accounts.0c00713\" target=\"_blank\">https:\/\/pubs.acs.org\/doi\/10.1021\/acs.accounts.0c00713<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Black-Box+Optimization+for+Automated+Discovery&rft.jtitle=Accounts+of+Chemical+Research&rft.aulast=Terayama&rft.aufirst=Kei&rft.au=Terayama%2C%26%2332%3BKei&rft.au=Sumita%2C%26%2332%3BMasato&rft.au=Tamura%2C%26%2332%3BRyo&rft.au=Tsuda%2C%26%2332%3BKoji&rft.date=16+March+2021&rft.volume=54&rft.issue=6&rft.pages=1334%E2%80%931346&rft_id=info:doi\/10.1021%2Facs.accounts.0c00713&rft.issn=0001-4842&rft_id=https%3A%2F%2Fpubs.acs.org%2Fdoi%2F10.1021%2Facs.accounts.0c00713&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-6\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-6\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ueno, Tsuyoshi; Rhone, Trevor David; Hou, Zhufeng; Mizoguchi, Teruyasu; Tsuda, Koji (1 June 2016). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2352924516300035\" target=\"_blank\">\"COMBO: An efficient Bayesian optimization library for materials science\"<\/a> (in en). <i>Materials Discovery<\/i> <b>4<\/b>: 18\u201321. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.md.2016.04.001\" target=\"_blank\">10.1016\/j.md.2016.04.001<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2352924516300035\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2352924516300035<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=COMBO%3A+An+efficient+Bayesian+optimization+library+for+materials+science&rft.jtitle=Materials+Discovery&rft.aulast=Ueno&rft.aufirst=Tsuyoshi&rft.au=Ueno%2C%26%2332%3BTsuyoshi&rft.au=Rhone%2C%26%2332%3BTrevor+David&rft.au=Hou%2C%26%2332%3BZhufeng&rft.au=Mizoguchi%2C%26%2332%3BTeruyasu&rft.au=Tsuda%2C%26%2332%3BKoji&rft.date=1+June+2016&rft.volume=4&rft.pages=18%E2%80%9321&rft_id=info:doi\/10.1016%2Fj.md.2016.04.001&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2352924516300035&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-7\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-7\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation book\">Packwood, Daniel (2017). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/978-981-10-6781-5\" target=\"_blank\"><i>Bayesian Optimization for Materials Science<\/i><\/a>. SpringerBriefs in the Mathematics of Materials. <b>3<\/b>. Singapore: Springer Singapore. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2F978-981-10-6781-5\" target=\"_blank\">10.1007\/978-981-10-6781-5<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-981-10-6780-8<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/978-981-10-6781-5\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/978-981-10-6781-5<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=Bayesian+Optimization+for+Materials+Science&rft.aulast=Packwood&rft.aufirst=Daniel&rft.au=Packwood%2C%26%2332%3BDaniel&rft.date=2017&rft.series=SpringerBriefs+in+the+Mathematics+of+Materials&rft.volume=3&rft.place=Singapore&rft.pub=Springer+Singapore&rft_id=info:doi\/10.1007%2F978-981-10-6781-5&rft.isbn=978-981-10-6780-8&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2F978-981-10-6781-5&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-8\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-8\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Jin, Yimeng; Kumar, Priyank V. (2023). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/xlink.rsc.org\/?DOI=D2NR07147A\" target=\"_blank\">\"Bayesian optimisation for efficient material discovery: a mini review\"<\/a> (in en). <i>Nanoscale<\/i> <b>15<\/b> (26): 10975\u201310984. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1039%2FD2NR07147A\" target=\"_blank\">10.1039\/D2NR07147A<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2040-3364\" target=\"_blank\">2040-3364<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/xlink.rsc.org\/?DOI=D2NR07147A\" target=\"_blank\">http:\/\/xlink.rsc.org\/?DOI=D2NR07147A<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Bayesian+optimisation+for+efficient+material+discovery%3A+a+mini+review&rft.jtitle=Nanoscale&rft.aulast=Jin&rft.aufirst=Yimeng&rft.au=Jin%2C%26%2332%3BYimeng&rft.au=Kumar%2C%26%2332%3BPriyank+V.&rft.date=2023&rft.volume=15&rft.issue=26&rft.pages=10975%E2%80%9310984&rft_id=info:doi\/10.1039%2FD2NR07147A&rft.issn=2040-3364&rft_id=http%3A%2F%2Fxlink.rsc.org%2F%3FDOI%3DD2NR07147A&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-9\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-9\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Homma, Kenji; Liu, Yu; Sumita, Masato; Tamura, Ryo; Fushimi, Naoki; Iwata, Junichi; Tsuda, Koji; Kaneta, Chioko (18 June 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acs.jpcc.9b11654\" target=\"_blank\">\"Optimization of a Heterogeneous Ternary Li 3 PO 4 \u2013Li 3 BO 3 \u2013Li 2 SO 4 Mixture for Li-Ion Conductivity by Machine Learning\"<\/a> (in en). <i>The Journal of Physical Chemistry C<\/i> <b>124<\/b> (24): 12865\u201312870. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1021%2Facs.jpcc.9b11654\" target=\"_blank\">10.1021\/acs.jpcc.9b11654<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1932-7447\" target=\"_blank\">1932-7447<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acs.jpcc.9b11654\" target=\"_blank\">https:\/\/pubs.acs.org\/doi\/10.1021\/acs.jpcc.9b11654<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Optimization+of+a+Heterogeneous+Ternary+Li+3+PO+4+%E2%80%93Li+3+BO+3+%E2%80%93Li+2+SO+4+Mixture+for+Li-Ion+Conductivity+by+Machine+Learning&rft.jtitle=The+Journal+of+Physical+Chemistry+C&rft.aulast=Homma&rft.aufirst=Kenji&rft.au=Homma%2C%26%2332%3BKenji&rft.au=Liu%2C%26%2332%3BYu&rft.au=Sumita%2C%26%2332%3BMasato&rft.au=Tamura%2C%26%2332%3BRyo&rft.au=Fushimi%2C%26%2332%3BNaoki&rft.au=Iwata%2C%26%2332%3BJunichi&rft.au=Tsuda%2C%26%2332%3BKoji&rft.au=Kaneta%2C%26%2332%3BChioko&rft.date=18+June+2020&rft.volume=124&rft.issue=24&rft.pages=12865%E2%80%9312870&rft_id=info:doi\/10.1021%2Facs.jpcc.9b11654&rft.issn=1932-7447&rft_id=https%3A%2F%2Fpubs.acs.org%2Fdoi%2F10.1021%2Facs.jpcc.9b11654&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-10\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-10\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Sakurai, Atsushi; Yada, Kyohei; Simomura, Tetsushi; Ju, Shenghong; Kashiwagi, Makoto; Okada, Hideyuki; Nagao, Tadaaki; Tsuda, Koji <i>et al.<\/i> (27 February 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acscentsci.8b00802\" target=\"_blank\">\"Ultranarrow-Band Wavelength-Selective Thermal Emission with Aperiodic Multilayered Metamaterials Designed by Bayesian Optimization\"<\/a> (in en). <i>ACS Central Science<\/i> <b>5<\/b> (2): 319\u2013326. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1021%2Facscentsci.8b00802\" target=\"_blank\">10.1021\/acscentsci.8b00802<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2374-7943\" target=\"_blank\">2374-7943<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC6396383\/\" target=\"_blank\">PMC6396383<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/30834320\" target=\"_blank\">30834320<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acscentsci.8b00802\" target=\"_blank\">https:\/\/pubs.acs.org\/doi\/10.1021\/acscentsci.8b00802<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Ultranarrow-Band+Wavelength-Selective+Thermal+Emission+with+Aperiodic+Multilayered+Metamaterials+Designed+by+Bayesian+Optimization&rft.jtitle=ACS+Central+Science&rft.aulast=Sakurai&rft.aufirst=Atsushi&rft.au=Sakurai%2C%26%2332%3BAtsushi&rft.au=Yada%2C%26%2332%3BKyohei&rft.au=Simomura%2C%26%2332%3BTetsushi&rft.au=Ju%2C%26%2332%3BShenghong&rft.au=Kashiwagi%2C%26%2332%3BMakoto&rft.au=Okada%2C%26%2332%3BHideyuki&rft.au=Nagao%2C%26%2332%3BTadaaki&rft.au=Tsuda%2C%26%2332%3BKoji&rft.au=Shiomi%2C%26%2332%3BJunichiro&rft.date=27+February+2019&rft.volume=5&rft.issue=2&rft.pages=319%E2%80%93326&rft_id=info:doi\/10.1021%2Facscentsci.8b00802&rft.issn=2374-7943&rft_id=info:pmc\/PMC6396383&rft_id=info:pmid\/30834320&rft_id=https%3A%2F%2Fpubs.acs.org%2Fdoi%2F10.1021%2Facscentsci.8b00802&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-11\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-11\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Sun, Shijing; Tiihonen, Armi; Oviedo, Felipe; Liu, Zhe; Thapa, Janak; Zhao, Yicheng; Hartono, Noor Titan P.; Goyal, Anuj <i>et al.<\/i> (1 April 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2590238521000084\" target=\"_blank\">\"A data fusion approach to optimize compositional stability of halide perovskites\"<\/a> (in en). <i>Matter<\/i> <b>4<\/b> (4): 1305\u20131322. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.matt.2021.01.008\" target=\"_blank\">10.1016\/j.matt.2021.01.008<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2590238521000084\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2590238521000084<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+data+fusion+approach+to+optimize+compositional+stability+of+halide+perovskites&rft.jtitle=Matter&rft.aulast=Sun&rft.aufirst=Shijing&rft.au=Sun%2C%26%2332%3BShijing&rft.au=Tiihonen%2C%26%2332%3BArmi&rft.au=Oviedo%2C%26%2332%3BFelipe&rft.au=Liu%2C%26%2332%3BZhe&rft.au=Thapa%2C%26%2332%3BJanak&rft.au=Zhao%2C%26%2332%3BYicheng&rft.au=Hartono%2C%26%2332%3BNoor+Titan+P.&rft.au=Goyal%2C%26%2332%3BAnuj&rft.au=Heumueller%2C%26%2332%3BThomas&rft.date=1+April+2021&rft.volume=4&rft.issue=4&rft.pages=1305%E2%80%931322&rft_id=info:doi\/10.1016%2Fj.matt.2021.01.008&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2590238521000084&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-12\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-12\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Tamura, Ryo; Osada, Toshio; Minagawa, Kazumi; Kohata, Takuma; Hirosawa, Masashi; Tsuda, Koji; Kawagishi, Kyoko (1 January 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0264127520308261\" target=\"_blank\">\"Machine learning-driven optimization in powder manufacturing of Ni-Co based superalloy\"<\/a> (in en). <i>Materials & Design<\/i> <b>198<\/b>: 109290. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.matdes.2020.109290\" target=\"_blank\">10.1016\/j.matdes.2020.109290<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0264127520308261\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0264127520308261<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Machine+learning-driven+optimization+in+powder+manufacturing+of+Ni-Co+based+superalloy&rft.jtitle=Materials+%26+Design&rft.aulast=Tamura&rft.aufirst=Ryo&rft.au=Tamura%2C%26%2332%3BRyo&rft.au=Osada%2C%26%2332%3BToshio&rft.au=Minagawa%2C%26%2332%3BKazumi&rft.au=Kohata%2C%26%2332%3BTakuma&rft.au=Hirosawa%2C%26%2332%3BMasashi&rft.au=Tsuda%2C%26%2332%3BKoji&rft.au=Kawagishi%2C%26%2332%3BKyoko&rft.date=1+January+2021&rft.volume=198&rft.pages=109290&rft_id=info:doi\/10.1016%2Fj.matdes.2020.109290&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0264127520308261&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:0-13\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:0_13-0\">13.0<\/a><\/sup> <sup><a href=\"#cite_ref-:0_13-1\">13.1<\/a><\/sup> <sup><a href=\"#cite_ref-:0_13-2\">13.2<\/a><\/sup> <sup><a href=\"#cite_ref-:0_13-3\">13.3<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Matsuda, Shoichi; Lambard, Guillaume; Sodeyama, Keitaro (1 April 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2666386422001023\" target=\"_blank\">\"Data-driven automated robotic experiments accelerate discovery of multi-component electrolyte for rechargeable Li\u2013O2 batteries\"<\/a> (in en). <i>Cell Reports Physical Science<\/i> <b>3<\/b> (4): 100832. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.xcrp.2022.100832\" target=\"_blank\">10.1016\/j.xcrp.2022.100832<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2666386422001023\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2666386422001023<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Data-driven+automated+robotic+experiments+accelerate+discovery+of+multi-component+electrolyte+for+rechargeable+Li%E2%80%93O2+batteries&rft.jtitle=Cell+Reports+Physical+Science&rft.aulast=Matsuda&rft.aufirst=Shoichi&rft.au=Matsuda%2C%26%2332%3BShoichi&rft.au=Lambard%2C%26%2332%3BGuillaume&rft.au=Sodeyama%2C%26%2332%3BKeitaro&rft.date=1+April+2022&rft.volume=3&rft.issue=4&rft.pages=100832&rft_id=info:doi\/10.1016%2Fj.xcrp.2022.100832&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2666386422001023&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-14\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-14\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ozaki, Yoshihiko; Suzuki, Yuta; Hawai, Takafumi; Saito, Kotaro; Onishi, Masaki; Ono, Kanta (5 June 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41524-020-0330-9\" target=\"_blank\">\"Automated crystal structure analysis based on blackbox optimisation\"<\/a> (in en). <i>npj Computational Materials<\/i> <b>6<\/b> (1): 75. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41524-020-0330-9\" target=\"_blank\">10.1038\/s41524-020-0330-9<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2057-3960\" target=\"_blank\">2057-3960<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41524-020-0330-9\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41524-020-0330-9<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Automated+crystal+structure+analysis+based+on+blackbox+optimisation&rft.jtitle=npj+Computational+Materials&rft.aulast=Ozaki&rft.aufirst=Yoshihiko&rft.au=Ozaki%2C%26%2332%3BYoshihiko&rft.au=Suzuki%2C%26%2332%3BYuta&rft.au=Hawai%2C%26%2332%3BTakafumi&rft.au=Saito%2C%26%2332%3BKotaro&rft.au=Onishi%2C%26%2332%3BMasaki&rft.au=Ono%2C%26%2332%3BKanta&rft.date=5+June+2020&rft.volume=6&rft.issue=1&rft.pages=75&rft_id=info:doi\/10.1038%2Fs41524-020-0330-9&rft.issn=2057-3960&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41524-020-0330-9&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:1-15\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:1_15-0\">15.0<\/a><\/sup> <sup><a href=\"#cite_ref-:1_15-1\">15.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Tamura, Ryo; Sumita, Masato; Terayama, Kei; Tsuda, Koji; Izumi, Fujio; Matsushita, Yoshitaka (31 December 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/27660400.2022.2146470\" target=\"_blank\">\"Automatic Rietveld refinement by robotic process automation with RIETAN-FP\"<\/a> (in en). <i>Science and Technology of Advanced Materials: Methods<\/i> <b>2<\/b> (1): 435\u2013444. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1080%2F27660400.2022.2146470\" target=\"_blank\">10.1080\/27660400.2022.2146470<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2766-0400\" target=\"_blank\">2766-0400<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/27660400.2022.2146470\" target=\"_blank\">https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/27660400.2022.2146470<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Automatic+Rietveld+refinement+by+robotic+process+automation+with+RIETAN-FP&rft.jtitle=Science+and+Technology+of+Advanced+Materials%3A+Methods&rft.aulast=Tamura&rft.aufirst=Ryo&rft.au=Tamura%2C%26%2332%3BRyo&rft.au=Sumita%2C%26%2332%3BMasato&rft.au=Terayama%2C%26%2332%3BKei&rft.au=Tsuda%2C%26%2332%3BKoji&rft.au=Izumi%2C%26%2332%3BFujio&rft.au=Matsushita%2C%26%2332%3BYoshitaka&rft.date=31+December+2022&rft.volume=2&rft.issue=1&rft.pages=435%E2%80%93444&rft_id=info:doi\/10.1080%2F27660400.2022.2146470&rft.issn=2766-0400&rft_id=https%3A%2F%2Fwww.tandfonline.com%2Fdoi%2Ffull%2F10.1080%2F27660400.2022.2146470&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-16\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-16\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Chakraborti, N. (1 June 2004). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.tandfonline.com\/doi\/full\/10.1179\/095066004225021909\" target=\"_blank\">\"Genetic algorithms in materials design and processing\"<\/a> (in en). <i>International Materials Reviews<\/i> <b>49<\/b> (3-4): 246\u2013260. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1179%2F095066004225021909\" target=\"_blank\">10.1179\/095066004225021909<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0950-6608\" target=\"_blank\">0950-6608<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.tandfonline.com\/doi\/full\/10.1179\/095066004225021909\" target=\"_blank\">http:\/\/www.tandfonline.com\/doi\/full\/10.1179\/095066004225021909<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Genetic+algorithms+in+materials+design+and+processing&rft.jtitle=International+Materials+Reviews&rft.aulast=Chakraborti&rft.aufirst=N.&rft.au=Chakraborti%2C%26%2332%3BN.&rft.date=1+June+2004&rft.volume=49&rft.issue=3-4&rft.pages=246%E2%80%93260&rft_id=info:doi\/10.1179%2F095066004225021909&rft.issn=0950-6608&rft_id=http%3A%2F%2Fwww.tandfonline.com%2Fdoi%2Ffull%2F10.1179%2F095066004225021909&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-17\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-17\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Patra, Tarak K.; Meenakshisundaram, Venkatesh; Hung, Jui-Hsiang; Simmons, David S. (13 February 2017). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acscombsci.6b00136\" target=\"_blank\">\"Neural-Network-Biased Genetic Algorithms for Materials Design: Evolutionary Algorithms That Learn\"<\/a> (in en). <i>ACS Combinatorial Science<\/i> <b>19<\/b> (2): 96\u2013107. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1021%2Facscombsci.6b00136\" target=\"_blank\">10.1021\/acscombsci.6b00136<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2156-8952\" target=\"_blank\">2156-8952<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acscombsci.6b00136\" target=\"_blank\">https:\/\/pubs.acs.org\/doi\/10.1021\/acscombsci.6b00136<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Neural-Network-Biased+Genetic+Algorithms+for+Materials+Design%3A+Evolutionary+Algorithms+That+Learn&rft.jtitle=ACS+Combinatorial+Science&rft.aulast=Patra&rft.aufirst=Tarak+K.&rft.au=Patra%2C%26%2332%3BTarak+K.&rft.au=Meenakshisundaram%2C%26%2332%3BVenkatesh&rft.au=Hung%2C%26%2332%3BJui-Hsiang&rft.au=Simmons%2C%26%2332%3BDavid+S.&rft.date=13+February+2017&rft.volume=19&rft.issue=2&rft.pages=96%E2%80%93107&rft_id=info:doi\/10.1021%2Facscombsci.6b00136&rft.issn=2156-8952&rft_id=https%3A%2F%2Fpubs.acs.org%2Fdoi%2F10.1021%2Facscombsci.6b00136&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-18\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-18\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">M. Dieb, Thaer; Ju, Shenghong; Yoshizoe, Kazuki; Hou, Zhufeng; Shiomi, Junichiro; Tsuda, Koji (31 December 2017). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/14686996.2017.1344083\" target=\"_blank\">\"MDTS: automatic complex materials design using Monte Carlo tree search\"<\/a> (in en). <i>Science and Technology of Advanced Materials<\/i> <b>18<\/b> (1): 498\u2013503. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1080%2F14686996.2017.1344083\" target=\"_blank\">10.1080\/14686996.2017.1344083<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1468-6996\" target=\"_blank\">1468-6996<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC5532970\/\" target=\"_blank\">PMC5532970<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/28804525\" target=\"_blank\">28804525<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/14686996.2017.1344083\" target=\"_blank\">https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/14686996.2017.1344083<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=MDTS%3A+automatic+complex+materials+design+using+Monte+Carlo+tree+search&rft.jtitle=Science+and+Technology+of+Advanced+Materials&rft.aulast=M.+Dieb&rft.aufirst=Thaer&rft.au=M.+Dieb%2C%26%2332%3BThaer&rft.au=Ju%2C%26%2332%3BShenghong&rft.au=Yoshizoe%2C%26%2332%3BKazuki&rft.au=Hou%2C%26%2332%3BZhufeng&rft.au=Shiomi%2C%26%2332%3BJunichiro&rft.au=Tsuda%2C%26%2332%3BKoji&rft.date=31+December+2017&rft.volume=18&rft.issue=1&rft.pages=498%E2%80%93503&rft_id=info:doi\/10.1080%2F14686996.2017.1344083&rft.issn=1468-6996&rft_id=info:pmc\/PMC5532970&rft_id=info:pmid\/28804525&rft_id=https%3A%2F%2Fwww.tandfonline.com%2Fdoi%2Ffull%2F10.1080%2F14686996.2017.1344083&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-19\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-19\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Li, Jiawen; Zhang, Jinzhe; Tamura, Ryo; Tsuda, Koji (2022). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/xlink.rsc.org\/?DOI=D1DD00043H\" target=\"_blank\">\"Self-learning entropic population annealing for interpretable materials design\"<\/a> (in en). <i>Digital Discovery<\/i> <b>1<\/b> (3): 295\u2013302. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1039%2FD1DD00043H\" target=\"_blank\">10.1039\/D1DD00043H<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2635-098X\" target=\"_blank\">2635-098X<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/xlink.rsc.org\/?DOI=D1DD00043H\" target=\"_blank\">http:\/\/xlink.rsc.org\/?DOI=D1DD00043H<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Self-learning+entropic+population+annealing+for+interpretable+materials+design&rft.jtitle=Digital+Discovery&rft.aulast=Li&rft.aufirst=Jiawen&rft.au=Li%2C%26%2332%3BJiawen&rft.au=Zhang%2C%26%2332%3BJinzhe&rft.au=Tamura%2C%26%2332%3BRyo&rft.au=Tsuda%2C%26%2332%3BKoji&rft.date=2022&rft.volume=1&rft.issue=3&rft.pages=295%E2%80%93302&rft_id=info:doi\/10.1039%2FD1DD00043H&rft.issn=2635-098X&rft_id=http%3A%2F%2Fxlink.rsc.org%2F%3FDOI%3DD1DD00043H&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-20\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-20\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Kitai, Koki; Guo, Jiang; Ju, Shenghong; Tanaka, Shu; Tsuda, Koji; Shiomi, Junichiro; Tamura, Ryo (16 March 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/link.aps.org\/doi\/10.1103\/PhysRevResearch.2.013319\" target=\"_blank\">\"Designing metamaterials with quantum annealing and factorization machines\"<\/a> (in en). <i>Physical Review Research<\/i> <b>2<\/b> (1): 013319. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1103%2FPhysRevResearch.2.013319\" target=\"_blank\">10.1103\/PhysRevResearch.2.013319<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2643-1564\" target=\"_blank\">2643-1564<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/link.aps.org\/doi\/10.1103\/PhysRevResearch.2.013319\" target=\"_blank\">https:\/\/link.aps.org\/doi\/10.1103\/PhysRevResearch.2.013319<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Designing+metamaterials+with+quantum+annealing+and+factorization+machines&rft.jtitle=Physical+Review+Research&rft.aulast=Kitai&rft.aufirst=Koki&rft.au=Kitai%2C%26%2332%3BKoki&rft.au=Guo%2C%26%2332%3BJiang&rft.au=Ju%2C%26%2332%3BShenghong&rft.au=Tanaka%2C%26%2332%3BShu&rft.au=Tsuda%2C%26%2332%3BKoji&rft.au=Shiomi%2C%26%2332%3BJunichiro&rft.au=Tamura%2C%26%2332%3BRyo&rft.date=16+March+2020&rft.volume=2&rft.issue=1&rft.pages=013319&rft_id=info:doi\/10.1103%2FPhysRevResearch.2.013319&rft.issn=2643-1564&rft_id=https%3A%2F%2Flink.aps.org%2Fdoi%2F10.1103%2FPhysRevResearch.2.013319&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-21\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-21\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Koshikawa, Ami S.; Ohzeki, Masayuki; Kadowaki, Tadashi; Tanaka, Kazuyuki (15 June 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/journals.jps.jp\/doi\/10.7566\/JPSJ.90.064001\" target=\"_blank\">\"Benchmark Test of Black-box Optimization Using D-Wave Quantum Annealer\"<\/a> (in en). <i>Journal of the Physical Society of Japan<\/i> <b>90<\/b> (6): 064001. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.7566%2FJPSJ.90.064001\" target=\"_blank\">10.7566\/JPSJ.90.064001<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0031-9015\" target=\"_blank\">0031-9015<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/journals.jps.jp\/doi\/10.7566\/JPSJ.90.064001\" target=\"_blank\">https:\/\/journals.jps.jp\/doi\/10.7566\/JPSJ.90.064001<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Benchmark+Test+of+Black-box+Optimization+Using+D-Wave+Quantum+Annealer&rft.jtitle=Journal+of+the+Physical+Society+of+Japan&rft.aulast=Koshikawa&rft.aufirst=Ami+S.&rft.au=Koshikawa%2C%26%2332%3BAmi+S.&rft.au=Ohzeki%2C%26%2332%3BMasayuki&rft.au=Kadowaki%2C%26%2332%3BTadashi&rft.au=Tanaka%2C%26%2332%3BKazuyuki&rft.date=15+June+2021&rft.volume=90&rft.issue=6&rft.pages=064001&rft_id=info:doi\/10.7566%2FJPSJ.90.064001&rft.issn=0031-9015&rft_id=https%3A%2F%2Fjournals.jps.jp%2Fdoi%2F10.7566%2FJPSJ.90.064001&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-22\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-22\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Izawa, Syun; Kitai, Koki; Tanaka, Shu; Tamura, Ryo; Tsuda, Koji (21 April 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/link.aps.org\/doi\/10.1103\/PhysRevResearch.4.023062\" target=\"_blank\">\"Continuous black-box optimization with an Ising machine and random subspace coding\"<\/a> (in en). <i>Physical Review Research<\/i> <b>4<\/b> (2): 023062. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1103%2FPhysRevResearch.4.023062\" target=\"_blank\">10.1103\/PhysRevResearch.4.023062<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2643-1564\" target=\"_blank\">2643-1564<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/link.aps.org\/doi\/10.1103\/PhysRevResearch.4.023062\" target=\"_blank\">https:\/\/link.aps.org\/doi\/10.1103\/PhysRevResearch.4.023062<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Continuous+black-box+optimization+with+an+Ising+machine+and+random+subspace+coding&rft.jtitle=Physical+Review+Research&rft.aulast=Izawa&rft.aufirst=Syun&rft.au=Izawa%2C%26%2332%3BSyun&rft.au=Kitai%2C%26%2332%3BKoki&rft.au=Tanaka%2C%26%2332%3BShu&rft.au=Tamura%2C%26%2332%3BRyo&rft.au=Tsuda%2C%26%2332%3BKoji&rft.date=21+April+2022&rft.volume=4&rft.issue=2&rft.pages=023062&rft_id=info:doi\/10.1103%2FPhysRevResearch.4.023062&rft.issn=2643-1564&rft_id=https%3A%2F%2Flink.aps.org%2Fdoi%2F10.1103%2FPhysRevResearch.4.023062&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-23\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-23\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">King, Ross D.; Rowland, Jem; Oliver, Stephen G.; Young, Michael; Aubrey, Wayne; Byrne, Emma; Liakata, Maria; Markham, Magdalena <i>et al.<\/i> (3 April 2009). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.science.org\/doi\/10.1126\/science.1165620\" target=\"_blank\">\"The Automation of Science\"<\/a> (in en). <i>Science<\/i> <b>324<\/b> (5923): 85\u201389. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1126%2Fscience.1165620\" target=\"_blank\">10.1126\/science.1165620<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0036-8075\" target=\"_blank\">0036-8075<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.science.org\/doi\/10.1126\/science.1165620\" target=\"_blank\">https:\/\/www.science.org\/doi\/10.1126\/science.1165620<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+Automation+of+Science&rft.jtitle=Science&rft.aulast=King&rft.aufirst=Ross+D.&rft.au=King%2C%26%2332%3BRoss+D.&rft.au=Rowland%2C%26%2332%3BJem&rft.au=Oliver%2C%26%2332%3BStephen+G.&rft.au=Young%2C%26%2332%3BMichael&rft.au=Aubrey%2C%26%2332%3BWayne&rft.au=Byrne%2C%26%2332%3BEmma&rft.au=Liakata%2C%26%2332%3BMaria&rft.au=Markham%2C%26%2332%3BMagdalena&rft.au=Pir%2C%26%2332%3BPinar&rft.date=3+April+2009&rft.volume=324&rft.issue=5923&rft.pages=85%E2%80%9389&rft_id=info:doi\/10.1126%2Fscience.1165620&rft.issn=0036-8075&rft_id=https%3A%2F%2Fwww.science.org%2Fdoi%2F10.1126%2Fscience.1165620&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-24\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-24\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Boyd, James (18 January 2002). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.science.org\/doi\/10.1126\/science.295.5554.517\" target=\"_blank\">\"Robotic Laboratory Automation\"<\/a> (in en). <i>Science<\/i> <b>295<\/b> (5554): 517\u2013518. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1126%2Fscience.295.5554.517\" target=\"_blank\">10.1126\/science.295.5554.517<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0036-8075\" target=\"_blank\">0036-8075<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.science.org\/doi\/10.1126\/science.295.5554.517\" target=\"_blank\">https:\/\/www.science.org\/doi\/10.1126\/science.295.5554.517<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Robotic+Laboratory+Automation&rft.jtitle=Science&rft.aulast=Boyd&rft.aufirst=James&rft.au=Boyd%2C%26%2332%3BJames&rft.date=18+January+2002&rft.volume=295&rft.issue=5554&rft.pages=517%E2%80%93518&rft_id=info:doi\/10.1126%2Fscience.295.5554.517&rft.issn=0036-8075&rft_id=https%3A%2F%2Fwww.science.org%2Fdoi%2F10.1126%2Fscience.295.5554.517&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-25\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-25\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Olsen, Kevin (1 December 2012). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S247263032201706X\" target=\"_blank\">\"The First 110 Years of Laboratory Automation: Technologies, Applications, and the Creative Scientist\"<\/a> (in en). <i>SLAS Technology<\/i> <b>17<\/b> (6): 469\u2013480. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1177%2F2211068212455631\" target=\"_blank\">10.1177\/2211068212455631<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S247263032201706X\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S247263032201706X<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+First+110+Years+of+Laboratory+Automation%3A+Technologies%2C+Applications%2C+and+the+Creative+Scientist&rft.jtitle=SLAS+Technology&rft.aulast=Olsen&rft.aufirst=Kevin&rft.au=Olsen%2C%26%2332%3BKevin&rft.date=1+December+2012&rft.volume=17&rft.issue=6&rft.pages=469%E2%80%93480&rft_id=info:doi\/10.1177%2F2211068212455631&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS247263032201706X&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-26\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-26\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Macarron, Ricardo; Banks, Martyn N.; Bojanic, Dejan; Burns, David J.; Cirovic, Dragan A.; Garyantes, Tina; Green, Darren V. S.; Hertzberg, Robert P. <i>et al.<\/i> (1 March 2011). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/nrd3368\" target=\"_blank\">\"Impact of high-throughput screening in biomedical research\"<\/a> (in en). <i>Nature Reviews Drug Discovery<\/i> <b>10<\/b> (3): 188\u2013195. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fnrd3368\" target=\"_blank\">10.1038\/nrd3368<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1474-1776\" target=\"_blank\">1474-1776<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/nrd3368\" target=\"_blank\">https:\/\/www.nature.com\/articles\/nrd3368<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Impact+of+high-throughput+screening+in+biomedical+research&rft.jtitle=Nature+Reviews+Drug+Discovery&rft.aulast=Macarron&rft.aufirst=Ricardo&rft.au=Macarron%2C%26%2332%3BRicardo&rft.au=Banks%2C%26%2332%3BMartyn+N.&rft.au=Bojanic%2C%26%2332%3BDejan&rft.au=Burns%2C%26%2332%3BDavid+J.&rft.au=Cirovic%2C%26%2332%3BDragan+A.&rft.au=Garyantes%2C%26%2332%3BTina&rft.au=Green%2C%26%2332%3BDarren+V.+S.&rft.au=Hertzberg%2C%26%2332%3BRobert+P.&rft.au=Janzen%2C%26%2332%3BWilliam+P.&rft.date=1+March+2011&rft.volume=10&rft.issue=3&rft.pages=188%E2%80%93195&rft_id=info:doi\/10.1038%2Fnrd3368&rft.issn=1474-1776&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fnrd3368&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-27\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-27\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">MacLeod, B. P.; Parlane, F. G. L.; Morrissey, T. D.; H\u00e4se, F.; Roch, L. M.; Dettelbach, K. E.; Moreira, R.; Yunker, L. P. E. <i>et al.<\/i> (15 May 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.science.org\/doi\/10.1126\/sciadv.aaz8867\" target=\"_blank\">\"Self-driving laboratory for accelerated discovery of thin-film materials\"<\/a> (in en). <i>Science Advances<\/i> <b>6<\/b> (20): eaaz8867. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1126%2Fsciadv.aaz8867\" target=\"_blank\">10.1126\/sciadv.aaz8867<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2375-2548\" target=\"_blank\">2375-2548<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC7220369\/\" target=\"_blank\">PMC7220369<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/32426501\" target=\"_blank\">32426501<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.science.org\/doi\/10.1126\/sciadv.aaz8867\" target=\"_blank\">https:\/\/www.science.org\/doi\/10.1126\/sciadv.aaz8867<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Self-driving+laboratory+for+accelerated+discovery+of+thin-film+materials&rft.jtitle=Science+Advances&rft.aulast=MacLeod&rft.aufirst=B.+P.&rft.au=MacLeod%2C%26%2332%3BB.+P.&rft.au=Parlane%2C%26%2332%3BF.+G.+L.&rft.au=Morrissey%2C%26%2332%3BT.+D.&rft.au=H%C3%A4se%2C%26%2332%3BF.&rft.au=Roch%2C%26%2332%3BL.+M.&rft.au=Dettelbach%2C%26%2332%3BK.+E.&rft.au=Moreira%2C%26%2332%3BR.&rft.au=Yunker%2C%26%2332%3BL.+P.+E.&rft.au=Rooney%2C%26%2332%3BM.+B.&rft.date=15+May+2020&rft.volume=6&rft.issue=20&rft.pages=eaaz8867&rft_id=info:doi\/10.1126%2Fsciadv.aaz8867&rft.issn=2375-2548&rft_id=info:pmc\/PMC7220369&rft_id=info:pmid\/32426501&rft_id=https%3A%2F%2Fwww.science.org%2Fdoi%2F10.1126%2Fsciadv.aaz8867&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-28\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-28\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Shimizu, Ryota; Kobayashi, Shigeru; Watanabe, Yuki; Ando, Yasunobu; Hitosugi, Taro (1 November 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubs.aip.org\/apm\/article\/8\/11\/111110\/569817\/Autonomous-materials-synthesis-by-machine-learning\" target=\"_blank\">\"Autonomous materials synthesis by machine learning and robotics\"<\/a> (in en). <i>APL Materials<\/i> <b>8<\/b> (11): 111110. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1063%2F5.0020370\" target=\"_blank\">10.1063\/5.0020370<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2166-532X\" target=\"_blank\">2166-532X<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubs.aip.org\/apm\/article\/8\/11\/111110\/569817\/Autonomous-materials-synthesis-by-machine-learning\" target=\"_blank\">https:\/\/pubs.aip.org\/apm\/article\/8\/11\/111110\/569817\/Autonomous-materials-synthesis-by-machine-learning<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Autonomous+materials+synthesis+by+machine+learning+and+robotics&rft.jtitle=APL+Materials&rft.aulast=Shimizu&rft.aufirst=Ryota&rft.au=Shimizu%2C%26%2332%3BRyota&rft.au=Kobayashi%2C%26%2332%3BShigeru&rft.au=Watanabe%2C%26%2332%3BYuki&rft.au=Ando%2C%26%2332%3BYasunobu&rft.au=Hitosugi%2C%26%2332%3BTaro&rft.date=1+November+2020&rft.volume=8&rft.issue=11&rft.pages=111110&rft_id=info:doi\/10.1063%2F5.0020370&rft.issn=2166-532X&rft_id=https%3A%2F%2Fpubs.aip.org%2Fapm%2Farticle%2F8%2F11%2F111110%2F569817%2FAutonomous-materials-synthesis-by-machine-learning&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-29\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-29\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Dave, Adarsh; Mitchell, Jared; Kandasamy, Kirthevasan; Wang, Han; Burke, Sven; Paria, Biswajit; P\u00f3czos, Barnab\u00e1s; Whitacre, Jay <i>et al.<\/i> (1 December 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2666386420302861\" target=\"_blank\">\"Autonomous Discovery of Battery Electrolytes with Robotic Experimentation and Machine Learning\"<\/a> (in en). <i>Cell Reports Physical Science<\/i> <b>1<\/b> (12): 100264. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.xcrp.2020.100264\" target=\"_blank\">10.1016\/j.xcrp.2020.100264<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2666386420302861\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2666386420302861<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Autonomous+Discovery+of+Battery+Electrolytes+with+Robotic+Experimentation+and+Machine+Learning&rft.jtitle=Cell+Reports+Physical+Science&rft.aulast=Dave&rft.aufirst=Adarsh&rft.au=Dave%2C%26%2332%3BAdarsh&rft.au=Mitchell%2C%26%2332%3BJared&rft.au=Kandasamy%2C%26%2332%3BKirthevasan&rft.au=Wang%2C%26%2332%3BHan&rft.au=Burke%2C%26%2332%3BSven&rft.au=Paria%2C%26%2332%3BBiswajit&rft.au=P%C3%B3czos%2C%26%2332%3BBarnab%C3%A1s&rft.au=Whitacre%2C%26%2332%3BJay&rft.au=Viswanathan%2C%26%2332%3BVenkatasubramanian&rft.date=1+December+2020&rft.volume=1&rft.issue=12&rft.pages=100264&rft_id=info:doi\/10.1016%2Fj.xcrp.2020.100264&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2666386420302861&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-30\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-30\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Burger, Benjamin; Maffettone, Phillip M.; Gusev, Vladimir V.; Aitchison, Catherine M.; Bai, Yang; Wang, Xiaoyan; Li, Xiaobo; Alston, Ben M. <i>et al.<\/i> (9 July 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41586-020-2442-2\" target=\"_blank\">\"A mobile robotic chemist\"<\/a> (in en). <i>Nature<\/i> <b>583<\/b> (7815): 237\u2013241. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41586-020-2442-2\" target=\"_blank\">10.1038\/s41586-020-2442-2<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0028-0836\" target=\"_blank\">0028-0836<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41586-020-2442-2\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41586-020-2442-2<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+mobile+robotic+chemist&rft.jtitle=Nature&rft.aulast=Burger&rft.aufirst=Benjamin&rft.au=Burger%2C%26%2332%3BBenjamin&rft.au=Maffettone%2C%26%2332%3BPhillip+M.&rft.au=Gusev%2C%26%2332%3BVladimir+V.&rft.au=Aitchison%2C%26%2332%3BCatherine+M.&rft.au=Bai%2C%26%2332%3BYang&rft.au=Wang%2C%26%2332%3BXiaoyan&rft.au=Li%2C%26%2332%3BXiaobo&rft.au=Alston%2C%26%2332%3BBen+M.&rft.au=Li%2C%26%2332%3BBuyi&rft.date=9+July+2020&rft.volume=583&rft.issue=7815&rft.pages=237%E2%80%93241&rft_id=info:doi\/10.1038%2Fs41586-020-2442-2&rft.issn=0028-0836&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41586-020-2442-2&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-31\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-31\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Roch, Lo\u00efc M.; H\u00e4se, Florian; Kreisbeck, Christoph; Tamayo-Mendoza, Teresa; Yunker, Lars P. E.; Hein, Jason E.; Aspuru-Guzik, Al\u00e1n (16 April 2020). Hu, Jianjun. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/dx.plos.org\/10.1371\/journal.pone.0229862\" target=\"_blank\">\"ChemOS: An orchestration software to democratize autonomous discovery\"<\/a> (in en). <i>PLOS ONE<\/i> <b>15<\/b> (4): e0229862. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1371%2Fjournal.pone.0229862\" target=\"_blank\">10.1371\/journal.pone.0229862<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1932-6203\" target=\"_blank\">1932-6203<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC7161969\/\" target=\"_blank\">PMC7161969<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/32298284\" target=\"_blank\">32298284<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/dx.plos.org\/10.1371\/journal.pone.0229862\" target=\"_blank\">https:\/\/dx.plos.org\/10.1371\/journal.pone.0229862<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=ChemOS%3A+An+orchestration+software+to+democratize+autonomous+discovery&rft.jtitle=PLOS+ONE&rft.aulast=Roch&rft.aufirst=Lo%C3%AFc+M.&rft.au=Roch%2C%26%2332%3BLo%C3%AFc+M.&rft.au=H%C3%A4se%2C%26%2332%3BFlorian&rft.au=Kreisbeck%2C%26%2332%3BChristoph&rft.au=Tamayo-Mendoza%2C%26%2332%3BTeresa&rft.au=Yunker%2C%26%2332%3BLars+P.+E.&rft.au=Hein%2C%26%2332%3BJason+E.&rft.au=Aspuru-Guzik%2C%26%2332%3BAl%C3%A1n&rft.date=16+April+2020&rft.volume=15&rft.issue=4&rft.pages=e0229862&rft_id=info:doi\/10.1371%2Fjournal.pone.0229862&rft.issn=1932-6203&rft_id=info:pmc\/PMC7161969&rft_id=info:pmid\/32298284&rft_id=https%3A%2F%2Fdx.plos.org%2F10.1371%2Fjournal.pone.0229862&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:2-32\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:2_32-0\">32.0<\/a><\/sup> <sup><a href=\"#cite_ref-:2_32-1\">32.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Motoyama, Yuichi; Tamura, Ryo; Yoshimi, Kazuyoshi; Terayama, Kei; Ueno, Tsuyoshi; Tsuda, Koji (1 September 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0010465522001242\" target=\"_blank\">\"Bayesian optimization package: PHYSBO\"<\/a> (in en). <i>Computer Physics Communications<\/i> <b>278<\/b>: 108405. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.cpc.2022.108405\" target=\"_blank\">10.1016\/j.cpc.2022.108405<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0010465522001242\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0010465522001242<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Bayesian+optimization+package%3A+PHYSBO&rft.jtitle=Computer+Physics+Communications&rft.aulast=Motoyama&rft.aufirst=Yuichi&rft.au=Motoyama%2C%26%2332%3BYuichi&rft.au=Tamura%2C%26%2332%3BRyo&rft.au=Yoshimi%2C%26%2332%3BKazuyoshi&rft.au=Terayama%2C%26%2332%3BKei&rft.au=Ueno%2C%26%2332%3BTsuyoshi&rft.au=Tsuda%2C%26%2332%3BKoji&rft.date=1+September+2022&rft.volume=278&rft.pages=108405&rft_id=info:doi\/10.1016%2Fj.cpc.2022.108405&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0010465522001242&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:3-33\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:3_33-0\">33.0<\/a><\/sup> <sup><a href=\"#cite_ref-:3_33-1\">33.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Terayama, Kei; Sumita, Masato; Tamura, Ryo; Payne, Daniel T.; Chahal, Mandeep K.; Ishihara, Shinsuke; Tsuda, Koji (2020). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/xlink.rsc.org\/?DOI=D0SC00982B\" target=\"_blank\">\"Pushing property limits in materials discovery via boundless objective-free exploration\"<\/a> (in en). <i>Chemical Science<\/i> <b>11<\/b> (23): 5959\u20135968. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1039%2FD0SC00982B\" target=\"_blank\">10.1039\/D0SC00982B<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2041-6520\" target=\"_blank\">2041-6520<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC7409358\/\" target=\"_blank\">PMC7409358<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/32832058\" target=\"_blank\">32832058<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/xlink.rsc.org\/?DOI=D0SC00982B\" target=\"_blank\">http:\/\/xlink.rsc.org\/?DOI=D0SC00982B<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Pushing+property+limits+in+materials+discovery+via+boundless+objective-free+exploration&rft.jtitle=Chemical+Science&rft.aulast=Terayama&rft.aufirst=Kei&rft.au=Terayama%2C%26%2332%3BKei&rft.au=Sumita%2C%26%2332%3BMasato&rft.au=Tamura%2C%26%2332%3BRyo&rft.au=Payne%2C%26%2332%3BDaniel+T.&rft.au=Chahal%2C%26%2332%3BMandeep+K.&rft.au=Ishihara%2C%26%2332%3BShinsuke&rft.au=Tsuda%2C%26%2332%3BKoji&rft.date=2020&rft.volume=11&rft.issue=23&rft.pages=5959%E2%80%935968&rft_id=info:doi\/10.1039%2FD0SC00982B&rft.issn=2041-6520&rft_id=info:pmc\/PMC7409358&rft_id=info:pmid\/32832058&rft_id=http%3A%2F%2Fxlink.rsc.org%2F%3FDOI%3DD0SC00982B&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-34\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-34\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Terayama, Kei; Tamura, Ryo; Nose, Yoshitaro; Hiramatsu, Hidenori; Hosono, Hideo; Okuno, Yasushi; Tsuda, Koji (8 March 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/link.aps.org\/doi\/10.1103\/PhysRevMaterials.3.033802\" target=\"_blank\">\"Efficient construction method for phase diagrams using uncertainty sampling\"<\/a> (in en). <i>Physical Review Materials<\/i> <b>3<\/b> (3): 033802. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1103%2FPhysRevMaterials.3.033802\" target=\"_blank\">10.1103\/PhysRevMaterials.3.033802<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2475-9953\" target=\"_blank\">2475-9953<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/link.aps.org\/doi\/10.1103\/PhysRevMaterials.3.033802\" target=\"_blank\">https:\/\/link.aps.org\/doi\/10.1103\/PhysRevMaterials.3.033802<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Efficient+construction+method+for+phase+diagrams+using+uncertainty+sampling&rft.jtitle=Physical+Review+Materials&rft.aulast=Terayama&rft.aufirst=Kei&rft.au=Terayama%2C%26%2332%3BKei&rft.au=Tamura%2C%26%2332%3BRyo&rft.au=Nose%2C%26%2332%3BYoshitaro&rft.au=Hiramatsu%2C%26%2332%3BHidenori&rft.au=Hosono%2C%26%2332%3BHideo&rft.au=Okuno%2C%26%2332%3BYasushi&rft.au=Tsuda%2C%26%2332%3BKoji&rft.date=8+March+2019&rft.volume=3&rft.issue=3&rft.pages=033802&rft_id=info:doi\/10.1103%2FPhysRevMaterials.3.033802&rft.issn=2475-9953&rft_id=https%3A%2F%2Flink.aps.org%2Fdoi%2F10.1103%2FPhysRevMaterials.3.033802&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:4-35\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:4_35-0\">35.0<\/a><\/sup> <sup><a href=\"#cite_ref-:4_35-1\">35.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Matsuda, Shoichi; Nishioka, Kiho; Nakanishi, Shuji (17 April 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41598-019-42766-x\" target=\"_blank\">\"High-throughput combinatorial screening of multi-component electrolyte additives to improve the performance of Li metal secondary batteries\"<\/a> (in en). <i>Scientific Reports<\/i> <b>9<\/b> (1): 6211. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41598-019-42766-x\" target=\"_blank\">10.1038\/s41598-019-42766-x<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2045-2322\" target=\"_blank\">2045-2322<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC6470175\/\" target=\"_blank\">PMC6470175<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/30996343\" target=\"_blank\">30996343<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41598-019-42766-x\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41598-019-42766-x<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=High-throughput+combinatorial+screening+of+multi-component+electrolyte+additives+to+improve+the+performance+of+Li+metal+secondary+batteries&rft.jtitle=Scientific+Reports&rft.aulast=Matsuda&rft.aufirst=Shoichi&rft.au=Matsuda%2C%26%2332%3BShoichi&rft.au=Nishioka%2C%26%2332%3BKiho&rft.au=Nakanishi%2C%26%2332%3BShuji&rft.date=17+April+2019&rft.volume=9&rft.issue=1&rft.pages=6211&rft_id=info:doi\/10.1038%2Fs41598-019-42766-x&rft.issn=2045-2322&rft_id=info:pmc\/PMC6470175&rft_id=info:pmid\/30996343&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41598-019-42766-x&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-36\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-36\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ong, Shyue Ping; Richards, William Davidson; Jain, Anubhav; Hautier, Geoffroy; Kocher, Michael; Cholia, Shreyas; Gunter, Dan; Chevrier, Vincent L. <i>et al.<\/i> (1 February 2013). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0927025612006295\" target=\"_blank\">\"Python Materials Genomics (pymatgen): A robust, open-source python library for materials analysis\"<\/a> (in en). <i>Computational Materials Science<\/i> <b>68<\/b>: 314\u2013319. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.commatsci.2012.10.028\" target=\"_blank\">10.1016\/j.commatsci.2012.10.028<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0927025612006295\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0927025612006295<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Python+Materials+Genomics+%28pymatgen%29%3A+A+robust%2C+open-source+python+library+for+materials+analysis&rft.jtitle=Computational+Materials+Science&rft.aulast=Ong&rft.aufirst=Shyue+Ping&rft.au=Ong%2C%26%2332%3BShyue+Ping&rft.au=Richards%2C%26%2332%3BWilliam+Davidson&rft.au=Jain%2C%26%2332%3BAnubhav&rft.au=Hautier%2C%26%2332%3BGeoffroy&rft.au=Kocher%2C%26%2332%3BMichael&rft.au=Cholia%2C%26%2332%3BShreyas&rft.au=Gunter%2C%26%2332%3BDan&rft.au=Chevrier%2C%26%2332%3BVincent+L.&rft.au=Persson%2C%26%2332%3BKristin+A.&rft.date=1+February+2013&rft.volume=68&rft.pages=314%E2%80%93319&rft_id=info:doi\/10.1016%2Fj.commatsci.2012.10.028&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0927025612006295&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-37\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-37\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ward, Logan; Agrawal, Ankit; Choudhary, Alok; Wolverton, Christopher (26 August 2016). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/npjcompumats201628\" target=\"_blank\">\"A general-purpose machine learning framework for predicting properties of inorganic materials\"<\/a> (in en). <i>npj Computational Materials<\/i> <b>2<\/b> (1): 16028. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fnpjcompumats.2016.28\" target=\"_blank\">10.1038\/npjcompumats.2016.28<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2057-3960\" target=\"_blank\">2057-3960<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/npjcompumats201628\" target=\"_blank\">https:\/\/www.nature.com\/articles\/npjcompumats201628<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+general-purpose+machine+learning+framework+for+predicting+properties+of+inorganic+materials&rft.jtitle=npj+Computational+Materials&rft.aulast=Ward&rft.aufirst=Logan&rft.au=Ward%2C%26%2332%3BLogan&rft.au=Agrawal%2C%26%2332%3BAnkit&rft.au=Choudhary%2C%26%2332%3BAlok&rft.au=Wolverton%2C%26%2332%3BChristopher&rft.date=26+August+2016&rft.volume=2&rft.issue=1&rft.pages=16028&rft_id=info:doi\/10.1038%2Fnpjcompumats.2016.28&rft.issn=2057-3960&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fnpjcompumats201628&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-38\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-38\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.rdkit.org\/\" target=\"_blank\">\"RDKit: Open-Source Cheminformatics Software\"<\/a>. 2023<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.rdkit.org\/\" target=\"_blank\">https:\/\/www.rdkit.org\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=RDKit%3A+Open-Source+Cheminformatics+Software&rft.atitle=&rft.date=2023&rft_id=https%3A%2F%2Fwww.rdkit.org%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-39\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-39\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ojih, Joshua; Al-Fahdi, Mohammed; Rodriguez, Alejandro David; Choudhary, Kamal; Hu, Ming (4 July 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41524-022-00836-1\" target=\"_blank\">\"Efficiently searching extreme mechanical properties via boundless objective-free exploration and minimal first-principles calculations\"<\/a> (in en). <i>npj Computational Materials<\/i> <b>8<\/b> (1): 143. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41524-022-00836-1\" target=\"_blank\">10.1038\/s41524-022-00836-1<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2057-3960\" target=\"_blank\">2057-3960<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41524-022-00836-1\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41524-022-00836-1<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Efficiently+searching+extreme+mechanical+properties+via+boundless+objective-free+exploration+and+minimal+first-principles+calculations&rft.jtitle=npj+Computational+Materials&rft.aulast=Ojih&rft.aufirst=Joshua&rft.au=Ojih%2C%26%2332%3BJoshua&rft.au=Al-Fahdi%2C%26%2332%3BMohammed&rft.au=Rodriguez%2C%26%2332%3BAlejandro+David&rft.au=Choudhary%2C%26%2332%3BKamal&rft.au=Hu%2C%26%2332%3BMing&rft.date=4+July+2022&rft.volume=8&rft.issue=1&rft.pages=143&rft_id=info:doi\/10.1038%2Fs41524-022-00836-1&rft.issn=2057-3960&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41524-022-00836-1&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-40\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-40\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Tamura, Ryo; Deffrennes, Guillaume; Han, Kwangsik; Abe, Taichi; Morito, Haruhiko; Nakamura, Yasuyuki; Naito, Masanobu; Katsube, Ryoji <i>et al.<\/i> (31 December 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/27660400.2022.2076548\" target=\"_blank\">\"Machine-Learning-Based phase diagram construction for high-throughput batch experiments\"<\/a> (in en). <i>Science and Technology of Advanced Materials: Methods<\/i> <b>2<\/b> (1): 153\u2013161. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1080%2F27660400.2022.2076548\" target=\"_blank\">10.1080\/27660400.2022.2076548<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2766-0400\" target=\"_blank\">2766-0400<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/27660400.2022.2076548\" target=\"_blank\">https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/27660400.2022.2076548<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Machine-Learning-Based+phase+diagram+construction+for+high-throughput+batch+experiments&rft.jtitle=Science+and+Technology+of+Advanced+Materials%3A+Methods&rft.aulast=Tamura&rft.aufirst=Ryo&rft.au=Tamura%2C%26%2332%3BRyo&rft.au=Deffrennes%2C%26%2332%3BGuillaume&rft.au=Han%2C%26%2332%3BKwangsik&rft.au=Abe%2C%26%2332%3BTaichi&rft.au=Morito%2C%26%2332%3BHaruhiko&rft.au=Nakamura%2C%26%2332%3BYasuyuki&rft.au=Naito%2C%26%2332%3BMasanobu&rft.au=Katsube%2C%26%2332%3BRyoji&rft.au=Nose%2C%26%2332%3BYoshitaro&rft.date=31+December+2022&rft.volume=2&rft.issue=1&rft.pages=153%E2%80%93161&rft_id=info:doi\/10.1080%2F27660400.2022.2076548&rft.issn=2766-0400&rft_id=https%3A%2F%2Fwww.tandfonline.com%2Fdoi%2Ffull%2F10.1080%2F27660400.2022.2076548&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-41\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-41\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Katsube, Ryoji; Terayama, Kei; Tamura, Ryo; Nose, Yoshitaro (1 June 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acsmaterialslett.0c00104\" target=\"_blank\">\"Experimental Establishment of Phase Diagrams Guided by Uncertainty Sampling: An Application to the Deposition of Zn\u2013Sn\u2013P Films by Molecular Beam Epitaxy\"<\/a> (in en). <i>ACS Materials Letters<\/i> <b>2<\/b> (6): 571\u2013575. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1021%2Facsmaterialslett.0c00104\" target=\"_blank\">10.1021\/acsmaterialslett.0c00104<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2639-4979\" target=\"_blank\">2639-4979<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acsmaterialslett.0c00104\" target=\"_blank\">https:\/\/pubs.acs.org\/doi\/10.1021\/acsmaterialslett.0c00104<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Experimental+Establishment+of+Phase+Diagrams+Guided+by+Uncertainty+Sampling%3A+An+Application+to+the+Deposition+of+Zn%E2%80%93Sn%E2%80%93P+Films+by+Molecular+Beam+Epitaxy&rft.jtitle=ACS+Materials+Letters&rft.aulast=Katsube&rft.aufirst=Ryoji&rft.au=Katsube%2C%26%2332%3BRyoji&rft.au=Terayama%2C%26%2332%3BKei&rft.au=Tamura%2C%26%2332%3BRyo&rft.au=Nose%2C%26%2332%3BYoshitaro&rft.date=1+June+2020&rft.volume=2&rft.issue=6&rft.pages=571%E2%80%93575&rft_id=info:doi\/10.1021%2Facsmaterialslett.0c00104&rft.issn=2639-4979&rft_id=https%3A%2F%2Fpubs.acs.org%2Fdoi%2F10.1021%2Facsmaterialslett.0c00104&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-42\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-42\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Hu, Wei-Hsun; Chen, Ta-Te; Tamura, Ryo; Terayama, Kei; Wang, Siqian; Watanabe, Ikumu; Naito, Masanobu (31 December 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/14686996.2021.2025426\" target=\"_blank\">\"Topological alternation from structurally adaptable to mechanically stable crosslinked polymer\"<\/a> (in en). <i>Science and Technology of Advanced Materials<\/i> <b>23<\/b> (1): 66\u201375. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1080%2F14686996.2021.2025426\" target=\"_blank\">10.1080\/14686996.2021.2025426<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1468-6996\" target=\"_blank\">1468-6996<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC8812728\/\" target=\"_blank\">PMC8812728<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/35125966\" target=\"_blank\">35125966<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/14686996.2021.2025426\" target=\"_blank\">https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/14686996.2021.2025426<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Topological+alternation+from+structurally+adaptable+to+mechanically+stable+crosslinked+polymer&rft.jtitle=Science+and+Technology+of+Advanced+Materials&rft.aulast=Hu&rft.aufirst=Wei-Hsun&rft.au=Hu%2C%26%2332%3BWei-Hsun&rft.au=Chen%2C%26%2332%3BTa-Te&rft.au=Tamura%2C%26%2332%3BRyo&rft.au=Terayama%2C%26%2332%3BKei&rft.au=Wang%2C%26%2332%3BSiqian&rft.au=Watanabe%2C%26%2332%3BIkumu&rft.au=Naito%2C%26%2332%3BMasanobu&rft.date=31+December+2022&rft.volume=23&rft.issue=1&rft.pages=66%E2%80%9375&rft_id=info:doi\/10.1080%2F14686996.2021.2025426&rft.issn=1468-6996&rft_id=info:pmc\/PMC8812728&rft_id=info:pmid\/35125966&rft_id=https%3A%2F%2Fwww.tandfonline.com%2Fdoi%2Ffull%2F10.1080%2F14686996.2021.2025426&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-43\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-43\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ota, Hitoshi; Shima, Kunihisa; Ue, Makoto; Yamaki, Jun-ichi (1 February 2004). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0013468603007266\" target=\"_blank\">\"Effect of vinylene carbonate as additive to electrolyte for lithium metal anode\"<\/a> (in en). <i>Electrochimica Acta<\/i> <b>49<\/b> (4): 565\u2013572. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.electacta.2003.09.010\" target=\"_blank\">10.1016\/j.electacta.2003.09.010<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0013468603007266\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0013468603007266<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Effect+of+vinylene+carbonate+as+additive+to+electrolyte+for+lithium+metal+anode&rft.jtitle=Electrochimica+Acta&rft.aulast=Ota&rft.aufirst=Hitoshi&rft.au=Ota%2C%26%2332%3BHitoshi&rft.au=Shima%2C%26%2332%3BKunihisa&rft.au=Ue%2C%26%2332%3BMakoto&rft.au=Yamaki%2C%26%2332%3BJun-ichi&rft.date=1+February+2004&rft.volume=49&rft.issue=4&rft.pages=565%E2%80%93572&rft_id=info:doi\/10.1016%2Fj.electacta.2003.09.010&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0013468603007266&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-44\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-44\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Zhang, Xue-Qiang; Cheng, Xin-Bing; Chen, Xiang; Yan, Chong; Zhang, Qiang (1 March 2017). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/adfm.201605989\" target=\"_blank\">\"Fluoroethylene Carbonate Additives to Render Uniform Li Deposits in Lithium Metal Batteries\"<\/a> (in en). <i>Advanced Functional Materials<\/i> <b>27<\/b> (10): 1605989. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1002%2Fadfm.201605989\" target=\"_blank\">10.1002\/adfm.201605989<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/adfm.201605989\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/adfm.201605989<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Fluoroethylene+Carbonate+Additives+to+Render+Uniform+Li+Deposits+in+Lithium+Metal+Batteries&rft.jtitle=Advanced+Functional+Materials&rft.aulast=Zhang&rft.aufirst=Xue-Qiang&rft.au=Zhang%2C%26%2332%3BXue-Qiang&rft.au=Cheng%2C%26%2332%3BXin-Bing&rft.au=Chen%2C%26%2332%3BXiang&rft.au=Yan%2C%26%2332%3BChong&rft.au=Zhang%2C%26%2332%3BQiang&rft.date=1+March+2017&rft.volume=27&rft.issue=10&rft.pages=1605989&rft_id=info:doi\/10.1002%2Fadfm.201605989&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1002%2Fadfm.201605989&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-45\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-45\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Ginsbourger, D.; Janusevskis, J.; Le Riche, R. (2011). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/hal.science\/hal-00507632\" target=\"_blank\">\"Dealing with asynchronicity in parallel Gaussian Process based global optimization\"<\/a>. Mines Saint-Etienne<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/hal.science\/hal-00507632\" target=\"_blank\">https:\/\/hal.science\/hal-00507632<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Dealing+with+asynchronicity+in+parallel+Gaussian+Process+based+global+optimization&rft.atitle=&rft.aulast=Ginsbourger%2C+D.%3B+Janusevskis%2C+J.%3B+Le+Riche%2C+R.&rft.au=Ginsbourger%2C+D.%3B+Janusevskis%2C+J.%3B+Le+Riche%2C+R.&rft.date=2011&rft.pub=Mines+Saint-Etienne&rft_id=https%3A%2F%2Fhal.science%2Fhal-00507632&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-46\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-46\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation\" id=\"rdp-ebb-CITEREFJanusevskisLe_RicheGinsbourgerGirdziusas2012\">Janusevskis, Janis; Le Riche, Rodolphe; Ginsbourger, David; Girdziusas, Ramunas (2012), Hamadi, Youssef; Schoenauer, Marc, eds., <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/978-3-642-34413-8_37\" target=\"_blank\">\"Expected Improvements for the Asynchronous Parallel Global Optimization of Expensive Functions: Potentials and Challenges\"<\/a>, <i>Learning and Intelligent Optimization<\/i> (Berlin, Heidelberg: Springer Berlin Heidelberg) <b>7219<\/b>: 413\u2013418, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2F978-3-642-34413-8_37\" target=\"_blank\">10.1007\/978-3-642-34413-8_37<\/a>, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-3-642-34412-1<span class=\"printonly\">, <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/978-3-642-34413-8_37\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/978-3-642-34413-8_37<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 2023-09-18<\/span><\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Expected+Improvements+for+the+Asynchronous+Parallel+Global+Optimization+of+Expensive+Functions%3A+Potentials+and+Challenges&rft.jtitle=Learning+and+Intelligent+Optimization&rft.aulast=Janusevskis&rft.aufirst=Janis&rft.au=Janusevskis%2C%26%2332%3BJanis&rft.au=Le+Riche%2C%26%2332%3BRodolphe&rft.au=Ginsbourger%2C%26%2332%3BDavid&rft.au=Girdziusas%2C%26%2332%3BRamunas&rft.date=2012&rft.volume=7219&rft.pages=413%E2%80%93418&rft.place=Berlin%2C+Heidelberg&rft.pub=Springer+Berlin+Heidelberg&rft_id=info:doi\/10.1007%2F978-3-642-34413-8_37&rft.isbn=978-3-642-34412-1&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2F978-3-642-34413-8_37&rfr_id=info:sid\/en.wikipedia.org:Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<\/ol><\/div><\/div>\n<h2><span class=\"mw-headline\" id=\"Notes\">Notes<\/span><\/h2>\n<p>This presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added. In the original, there are multiple instances of citing research work using the last name of the last author listed, rather than the last name of the first author listed; this may have been a product of Japanese culture tending to read text from right to left. For this version, the last name of the first author was used to be consistent with research norms.\n<\/p>\n<!-- \nNewPP limit report\nCached time: 20231215134429\nCache expiry: 86400\nDynamic content: false\nComplications: []\nCPU time usage: 2.509 seconds\nReal time usage: 3.442 seconds\nPreprocessor visited node count: 49798\/1000000\nPost\u2010expand include size: 447181\/2097152 bytes\nTemplate argument size: 126011\/2097152 bytes\nHighest expansion depth: 25\/40\nExpensive parser function count: 0\/100\nUnstrip recursion depth: 0\/20\nUnstrip post\u2010expand size: 119172\/5000000 bytes\n-->\n<!--\nTransclusion expansion time report (%,ms,calls,template)\n100.00% 1131.802 1 -total\n 88.74% 1004.345 1 Template:Reflist\n 68.32% 773.244 46 Template:Citation\/core\n 68.13% 771.130 42 Template:Cite_journal\n 13.46% 152.338 46 Template:Date\n 8.21% 92.957 92 Template:Citation\/identifier\n 4.80% 54.305 1 Template:Infobox_journal_article\n 4.26% 48.267 1 Template:Infobox\n 3.09% 34.932 2 Template:Cite_web\n 2.68% 30.294 184 Template:Hide_in_print\n-->\n\n<!-- Saved in parser cache with key limswiki:pcache:idhash:14389-0!canonical!math=5 and timestamp 20231215134425 and revision id 53492. Serialized with JSON.\n -->\n<\/div><\/div><div class=\"printfooter\">Source: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.limswiki.org\/index.php\/Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science\">https:\/\/www.limswiki.org\/index.php\/Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science<\/a><\/div>\n<!-- end content --><div class=\"visualClear\"><\/div><\/div><\/div><div class=\"visualClear\"><\/div><\/div><!-- end of the left (by default at least) column --><div class=\"visualClear\"><\/div><\/div>\n\n\n\n<\/body>","9e1d6433c962801d5a75756c1046599c_images":["https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/c\/c0\/Fig1_Tamura_SciTechAdvMatMeth2023_3-1.jpeg","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/f\/f9\/Fig2_Tamura_SciTechAdvMatMeth2023_3-1.jpeg","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/b\/b4\/Fig3_Tamura_SciTechAdvMatMeth2023_3-1.jpeg","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/1\/1e\/Prog1_Tamura_SciTechAdvMatMeth2023_3-1.jpeg","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/d\/d1\/Fig4_Tamura_SciTechAdvMatMeth2023_3-1.jpeg","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/3\/34\/Fig5_Tamura_SciTechAdvMatMeth2023_3-1.jpeg"],"9e1d6433c962801d5a75756c1046599c_timestamp":1702682173,"e65552a701c8075f24aa45db5f398e2e_type":"article","e65552a701c8075f24aa45db5f398e2e_title":"Development of an integrated and comprehensive clinical trial process management system (Shen et al. 2023)","e65552a701c8075f24aa45db5f398e2e_url":"https:\/\/www.limswiki.org\/index.php\/Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system","e65552a701c8075f24aa45db5f398e2e_plaintext":"\n\nJournal:Development of an integrated and comprehensive clinical trial process management systemFrom LIMSWikiJump to navigationJump to searchFull article title\n \nDevelopment of an integrated and comprehensive clinical trial process management systemJournal\n \nBMC Medical Informatics and Decision MakingAuthor(s)\n \nShen, Liang; Zhai, You; Pan, AXiang; Zhao, Qingwei; Zhou, Min; Lio, JianAuthor affiliation(s)\n \nZhejiang University School of MedicinePrimary contact\n \nEmail: minzhou at zju dot edu dot cnYear published\n \n2023Volume and issue\n \n23Article #\n \n61DOI\n \n10.1186\/s12911-023-02158-8ISSN\n \n1472-6947Distribution license\n \nCreative Commons Attribution 4.0 InternationalWebsite\n \nhttps:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-023-02158-8Download\n \nhttps:\/\/bmcmedinformdecismak.biomedcentral.com\/counter\/pdf\/10.1186\/s12911-023-02158-8.pdf (PDF)\n\nContents \n\n1 Abstract \n2 Background \n3 Methods \n\n3.1 Overview \n3.2 System development process \n\n3.2.1 First stage \n3.2.2 Second stage \n3.2.3 Third stage \n3.2.4 Fourth stage \n\n\n\n\n4 Results \n\n4.1 System categories and features for clinical trial \n4.2 Enterprise process management for clinical trials \n4.3 Data integration with external systems \n4.4 Data security and privacy protection \n4.5 Evaluation \n\n\n5 Discussion \n6 Conclusion \n7 Abbreviations, acronyms, and initialisms \n8 Acknowledgements \n\n8.1 Author contributions \n8.2 Ethics approval and consent to participate \n8.3 Funding \n8.4 Availability of data and materials \n8.5 Software availability and requirements \n8.6 Competing interests \n\n\n9 References \n10 Notes \n\n\n\nAbstract \nBackground: The process of initiating and completing clinical drug trials in hospital settings is highly complex, with numerous institutional, technical, and record-keeping barriers. In this study, we independently developed an integrated clinical trial management system (CTMS) designed to comprehensively optimize the process management of clinical trials. The CTMS includes system development methods, efficient integration with external business systems, terminology, and standardization protocols, as well as data security and privacy protection.\nMethods: The development process proceeded through four stages, including demand analysis and problem collection, system design, system development and testing, system trial operation, and training the whole hospital to operate the system. The integrated CTMS comprises three modules: project approval and review management, clinical trial operations management, and background management modules. These are divided into seven subsystems and 59 internal processes, realizing all the functions necessary to comprehensively perform the process management of clinical trials. Efficient data integration is realized through extract-transform-load (ETL), message queue, and remote procedure call services with external systems such as the hospital information system (HIS), laboratory information system (LIS), electronic medical record (EMR), and clinical data repository (CDR). Data security is ensured by adopting corresponding policies for data storage and data access. Privacy protection complies with laws and regulations and de-identifies sensitive patient information.\nResults: The integrated CTMS was successfully developed in September 2015 and updated to version 4.2.5 in March 2021. During this period, 1,388 study projects were accepted, 43,051 electronic data files stored, and 12,144 subjects recruited in the First Affiliated Hospital, Zhejiang University School of Medicine.\nConclusion: The developed integrated CTMS realizes the data management of the entire clinical trials process, providing basic conditions for the efficient, high-quality, and standardized operation of clinical trials.\nKeywords: clinical trial, clinical trial management system (CTMS), information technology, integrated management system, medical informatics\n\nBackground \nClinical trials of drugs are an important stage in drug research and development and are means to improve the development of medical science and technology.[1][2][3][4][5] Standardizing the management of clinical research is key to simplifying the research process, improving quality, and ensuring the accuracy, reliability, and integrity of the results, thereby shortening the development cycle of new drugs and accelerating the process of drug registration.[6] \nIn China, each hospital has a dedicated clinical trial management department, called a clinical trial institution (CTI), responsible for the administrative management of clinical trial operation in the hospital. This involves determining the project leader, formulating research plans with sponsors, reviewing the plans with the ethics committee, signing research contracts, screening and enrolling subjects, originating records of trials, receiving drugs, distributing and recovering, processing or reporting adverse events (AE) or serious adverse events (SAE), managing in-hospital quality, accepting supervision and inspection, summarizing, and filing. The traditional management mode is based on paper documents and involves considerable manual labor, which often leads to untimely information updates, ineffective management, and unverifiable quality. \nToday, the rapid development of internet technology has profoundly impacted the mode of clinical trial management. Diverse companies have developed a series of network management systems for clinical trials.[7][8] Most of these systems have been developed based on the research and development (R&D) requirements of pharmaceutical companies. Clinical trials are managed in a networked manner, which only handles project-related data and excludes the process management links of the clinical trial organization. These commercialized clinical trial management systems (CTMSs) provide comprehensive clinical trial management services, supporting all types of clinical trials, from Phase I to Phase III, from a trial conducted in a single research center to multinational clinical trials; thus, they improve the quality of clinical trials and the efficiency of data management. However, from the perspective of drug clinical trial institutions, this type of CTMS cannot satisfy the practical requirements for real-time and effective management of all clinical trials carried out by these institutions. Hence, a new CTMS has been developed as an institutional management model in China. Its development and application are still in their infancy. The CTMS also faces certain incompatibility issues with the hospital database. It is difficult to achieve data sharing and docking. Moreover, the management method is still relatively primitive. The degree of digitalization, networking, and standardization of clinical trials is relatively low, and it is practically impossible for institutions to effectively manage clinical trials.\nIn recent years, owing to the increase in clinical trial projects in our hospital, professional and standardized information systems were urgently needed to assist in the management of clinical trials throughout the hospital. At present, the need for a centralized clinical trial project management platform reliant on the clinical data system of the hospital itself is intensifying. Based on the local area network (LAN) security architecture of our hospital, we have constructed a CTMS, which organically integrates the clinical trial organization management office, ethics committee, clinical trial center pharmacy, and clinical professional departments. The management system covers the entire process of clinical trials, including trial project establishment, ethical review, signing of agreements, trial implementation, trial conclusion, sponsor management, subject management, follow-up management, and centralized management of pharmacies supporting medication trials. The personnel involved include clinical departments, institutional management offices, central pharmacies, and ethics committees. With the help of the information system, a large number of personnel and complicated work processes are now more organically integrated to realize information sharing and collaborative work. With the help of this CTMS, the implementation of clinical trials can be standardized and the entire process can be traceable. \nAs a site of clinical trials, hospital management involves multi-party collaboration, as well as study project management, subject management, investigational product management, quality control, financial management, and other complex business processes. Hospitals must develop dedicated systems to assist the entire process management of clinical trials to ensure their efficient and high-quality operation. \nThe main objectives of this study are as follows. We aim (1) to develop an integrated CTMS as a dedicated database for clinical trials; (2) to achieve efficient data integration of the CTMS with hospital business systems such as the hospital information system (HIS), laboratory information system (LIS), clinical data repository (CDR), picture archiving and communication system (PACS), and electronic medical record (EMR); (3) to facilitate standardization and consistency of terminology in the development of database models and business processes; and (4) to ensure the safety and security of clinical data as well as protect patient privacy to comply with relevant regulations.\n\nMethods \nOverview \nThe First Affiliated Hospital, Zhejiang University School of Medicine (FAHZU) has six campuses with approximately 5,000 beds. In 2020, the institution recorded over 4.2 million outpatient and emergency visits, and 236,100 discharges. As one of the pioneering and earliest-founded National Drug CTIs in China, the first batch of clinical pharmacology bases under the Ministry of Health was established in 1998. As a first-class integrated service platform for clinical research in China, it operates 24 specialized groups and depends on an internationally recognized and independent ethics examination system, and it has undertaken more than 2,000 instances of foreign and domestic clinical trials since its establishment, including both Phase I\u2013IV drugs and medical devices.\n\nSystem development process \nThe development of an integrated CTMS began in March 2014, and after 18 months, the first version of the CTMS was launched in FAHZU in September 2015. The complete system development process was carried out in four stages, including two months of requirement analysis and problem collection, four months of system design, six months of system development and testing, and six months of system operation training and pilot runs. Based on the feedback from the use of the first version of CTMS and the continuous in-depth exploration of the business, the system\u2019s functionality and user experience continue to be iteratively updated. By March 2021, the updated version 4.2.5 was launched. Through the construction of an integrated CTMS, the entire process of clinical trial data management is realized, which provides the basic conditions for the efficient, high-quality, and standardized operation of clinical trials in the hospital.\n\nFirst stage \nThe collection and analysis of CTMS requirements defined the existing problem set to be solved. This stage is the most critical and determines the final goal direction of the system. The collection of requirements started in March 2014, and the participants included the CTI office director, CTI office secretary, ethics committee (EC) members, EC secretary, principal investigator (PI), sub-investigator, study nurse, clinical research associate (CRA), clinical research coordinator (CRC), investigational product custodians, financial officer, quality control expert, statisticians, data manager, pharmacovigilance associate, and information technology expert. After two months of formal and informal interviews, as well as an analysis of the collection requirements, the design goal of CTMS was determined. The overall goal is for the CTMS to realize the entire process of clinical trial management, including study project approval, ethical review, subject recruitment, subject management, investigational product management, financial management, and quality management. Table 1 lists the functional requirements for realizing the entire process management of clinical trials in hospitals based on seven dimensions. These requirements were used to construct the integrated CTMS.\n\n\n\n\n\n\n\nTable 1. List of functional requirements for realizing the entire process management workflow of clinical trials in hospitals\n\n\n\nNumber\n\nRequirement class\n\nRequirement description\n\n\n1\n\nStudy Project Approval Management\n\nThe application and review of clinical trials should adopt process automation management, including process customization configuration, remote submission of project application, uploading of project materials, automatic generation of to-do tasks, timely message transmission, material annotation, and other supporting functions.\n\n\n2\n\nEthical Review Management\n\nDefine the ethics committee review process and application contents (e.g., new protocols, protocol amendments, etc.).\n\n\n3\n\nSubject Management\n\nDefine the subject management model to ensure that the subjects complete the visit content of each cycle in strict accordance with the research plan.\n\n\n4\n\nInvestigational Product Management\n\nThe investigational product should adopt the central pharmacy model to achieve closed-loop management, including receiving, warehousing, distributing, recycling, returning, disposal, and early warning.\n\n\n5\n\nFinancial Management\n\nClinical trial finance requires independent accounting and management; the subjects\u2019 diagnosis and treatment processes can be exempted from payment, and the system should automatically record costs to achieve direct settlement between the hospital and the sponsor.\n\n\n6\n\nQuality Management\n\nDefine the elements and content of quality management, and the quality control of related data that must be collected during the operation of the system.\n\n\n7\n\nPrivilege Management\n\nDefine permissions and data access rules for different roles through multi-role collaboration.\n\n\n\nSecond stage \nIn the second stage, the CTMS design process included the definition of the system architecture and data interchange mechanisms, selection of storage media, database modeling and standardization, user interface (UI) and user experience (UX) design, and development of security and privacy policies based on analyzing and categorizing the requirements collected in the first phase, referring to good clinical practice (GCP) and the guidance documents of the National Medical Products Administration (NMPA). The functions and data required by each participant in the clinical trial are determined. Figure 1 shows the system architecture of the CTMS, which is divided into seven subsystems to meet the corresponding requirements of the first stage. Seven dimensions were put forward, including the clinical trial project management system (CTPMS), clinical trial ethical management system (CTEMS), clinical trial subject management system (CTSMS), clinical trial investigational product management system (CTIPMS), clinical trial financial management system (CTFMS), clinical trial quality management system (CTQMS), and permission management and maintenance system (PMMS). These systems include the functions of the whole-process management of clinical trials. The unified control of permissions is realized through single sign-on (SSO) between the systems. To ensure a more efficient operation of the CTMS, it is necessary to focus on the data integration mode with the hospital clinical and business systems (i.e., HIS, LIS, EMR, CDR, etc.). \nThe unified management of interface services is defined, as shown in Fig. 1, as a process of complete data integration, which primarily involves three types of data service functions: (1) basic data synchronization, which serves as the basis for system operation; (2) real-time data query to meet the needs of subject information retrieval and clinical trial data monitoring; and (3) diagnosis and treatment data generated by the CTMS, which are transmitted to the clinical business system of the hospital to meet the continuity requirement of diagnosis and treatment of the subjects. Database modeling and standardization are performed with reference to the Clinical Data Interchange Standards Consortium (CDISC) to obtain a standard vocabulary to ensure the integrity of the data model; in terms of system security and subject privacy protection, security protection strategies and development specifications were formulated with reference to the Health Insurance Portability and Accountability Act (HIPAA) and China\u2019s privacy protection laws. In addition, the CTMS implements fine-grained isolation and verification of permissions, and it records all events in a log to ensure the traceability of the data.\n\r\n\n\n\n\n\n\n\n\n\n\nFig. 1 System architecture for the clinical trial management system (CTMS). HIS: hospital information system; EMR: electronic medical record; LIS: laboratory information system; CDR: clinical data repository; PACS: picture archiving and communication systems.\n\n\n\nThird stage \nThe third stage focused on the development and testing of the CTMS. This was divided into two steps according to the definition of the design stage. The first step realizes clinical trial approval and review management, and the developed subsystems include CTPMS, CTEMS, and PMMS, which focus on the process management of the study project application and review stage, and do not involve data integration with external systems. The second step realizes clinical trial operational management. The developed subsystems include CTSMS, CTIPMS, CTFMS, and CTQMS, which complete internal function development and data integration with external systems, and realize the entire process of data management. The CTMS was developed based on the distributed architecture of the Java programming language, and its database is Oracle Database 11 g Release 2 on a server running the Linux CentOS 7.6 operating system; data integration with external systems is managed through an interface services unified management platform. The system adopts browser\/server architecture and supports current mainstream browsers, such as Google Chrome, Internet Explorer, and Firefox Browser for encrypted access via hypertext transfer protocol secure (HTTPS). System testing mainly includes verification of functional effectiveness, data security, system reliability, and the efficiency of data integration with external systems.\n\nFourth stage \nThis stage involved CTMS deployment and system operation training. In November 2014, CTPMS, CTEMS, and PMMS were officially launched, realizing clinical trial approval and review management in FAHZU. In March 2015, the development of CTSMS, CTMMS, CTFMS, and CTQMS was completed. Considering that the clinical trial operational management involves data integrity verification and data integration with external systems, a new pilot run phase was needed. By selecting a certain number of clinical trials in Phases I\u2013IV, the operating results were used to verify the compatibility of the system with different types of clinical trials. The CTMS pilot run system first selects Phase I clinical trials and runs a total of 20 projects. Through continuous updating of functions during operation, it can fully meet the needs of Phase I clinical trial operational management. Subsequently, it selects clinical trials in Phases II\u2013IV. A total of 10 projects have been run, and the functions have been stable and meet the requirements of various types of clinical trials. However, training hospital staff in the use of CTMS is critical. Through a variety of methods including face-to-face training and offline data learning, all staff related to clinical trials have been trained. The first version of CTMS was fully launched in FAHZU in September 2015.\n\nResults \nFAHZU is the first hospital in China to independently develop an integrated CTMS; the system has been successfully implemented in the hospital since September 2015. As of March 2021, the hospital now runs an updated version of CTMS known as V4.2.5. The data management of the entire process of clinical trials from project approval and review management to operational management has been fully realized. The FAHZU CTMS operates independently by design as a fully functional system at the application level (not as a component of the HIS), establishes a dedicated clinical trial database at the data level, and completes data integration with external systems through a unified interface system, to achieve process continuity and data integrity. Through the construction of an integrated CTMS, the flow of multi-party collaboration tasks is optimized, and non-research matters such as finance and data processing are simplified, to effectively improve the efficiency and quality of clinical trials. The results shown below are based on the latest stable version of FAHZU CTMS, V4.2.5.\n\nSystem categories and features for clinical trial \nThe first level involves clinical trial project approval and review management, and its functions cover the rationality review of clinical trial study projects and related affairs management by CTI and EC. Table 2 lists the categories and features of clinical trial approval and review management, and the services provided to users through CTPMS and CTEMS. Clinical trial project approval and review management are based on multi-role collaboration, focusing on improving efficiency as the core and supporting remote project application, full electronic project approval, review of process approval documents online, aggregation of reviews into review comments, and timely generation of tasks and notification. The main features of CTPMS include project approval management, to-do task list, project list, document management, contract management, initial meeting management, investigator management, and CRC management. Through the organic combination of these features, the CTI realizes the project approval review and daily management of study projects. The main features of CTEMS include EC management, ethics review management, and ethics conference management, which manages the continuous ethics review of study projects by the EC, including initial protocol review, protocol amendment review, SAE report and review, and violation\/deviation protocol review.\n\n\n\n\n\n\n\nTable 2. Categories and features for project approval and review management of clinical trials\n\n\n\nCategory\n\nFeatures\n\nDescription\n\n\nStudy project management\n\nProject approval management\n\nCTI defines the application and review procedures for study projects according to the standard operation procedure (SOP).\n\n\nTo-do tasks list\n\nLists the tasks that the user needs to complete, which are automatically generated by the system according to processes and user roles.\n\n\nProject list\n\nProject list contains basic information about the study, controlling different viewing scopes for different roles.\n\n\nDocument management\n\nElectronic management of documents, batch uploading, online review, and suggestion feedback.\n\n\nContract management\n\nManages the content and budget of the contract, and supervises the execution of the contract.\n\n\nInitial meeting management\n\nRecords the meeting contents and participants of the initial meeting.\n\n\nInvestigator management\n\nManages investigators\u2019 Good Clinical Practice (GCP) education, resume, etc.\n\n\nClinical research coordinator (CRC) management\n\nManages CRC personnel information, recruitment process, and workload reporting and review.\n\n\nEthics management\n\nEthics committee (EC)\n\nManages the organizational structure of the EC.\n\n\nEthics review management\n\nManages the ethics review process, including ethical review application, formal review, study assessment, ethics conference review, approval letter generation, etc.\n\n\nEthics conference management\n\nManages the project review agenda, meeting attendance, voting, meeting minutes, project review results, etc.\n\n\n\nThe second level comprises clinical trial operational management, which is a complex and continuous management process. Table 3 lists the categories and features of clinical trial operational management; the system provides services to users through the CTSMS, CTIPMS, CTFMS, and CTQMS. CTSMS is mainly composed of three stages: operational management pre-configuration, subject recruitment, and subject visit management. The features included in the operational management pre-configuration stage include study participant assignments, protocol configuration, electronic case report form (e-CRF) design, rule configuration, and global control over the access rights and visitation rules of subjects under the study project. The main features of the subject recruitment stage of the system include subject recruitment, subject violation verification (such as determining whether the subject is participating in another clinical trial and inputting incorrect information of subject), subject lists, and subject global labeling, to realize the registration of subjects under the corresponding study project, status labeling, and visualization. \nThere are three main events in the subject visit management stage, which are described as follows: \n\nObtain the visit content information in the corresponding study cycle according to the study plan and turn it into a to-do list. The visit content includes subject screening, inspection\/examination issuance, prescription issuance, treatment, randomization of subjects, e-CRF filling, and AE\/SAE reporting.\nEnter operational status changes of subjects, including switching protocols, admission, or discharge, entering the next visit stage, and status of subject updates (dropping out, withdrawal, failure to follow up, etc.).\nComplete data queries of subjects, including outpatient\/inpatient medical records, inspection\/examination results queries, subject fee queries, and e-CRF data filling.\nCTIPMS adopts the central management model, which is managed by qualified personnel designated by the CTI. Through the organic combination of stock management, prescription management, label management, and intelligent detection, closed-loop management of the entire process of trial investigational products is realized. CTFMS primarily combines the financial characteristics of clinical trials and the relevant requirements of the hospital\u2019s financial management, including payment and allocation, budget management, expenditure management, workload statistics, and project funding amounts to achieve orderly financial management based on greatly reducing researchers\u2019 time consumption. Similarly, CTQMS mainly includes regular quality control data report generation and reporting, as well as dynamic quality control based on collected data.\n\n\n\n\n\n\n\nTable 3. Categories and features for operational management of clinical trials\n\n\n\nCategory\n\nFeatures\n\nDescription\n\n\nSubject management\n\nStudy participants assignment\n\nAssigns study project participants (such as investigators, study nurses, clinical research coordinator [CRC], and clinical research associate [CRA]) and sets corresponding permissions.\n\n\nProtocol configuration\n\nThe protocol is mapped to computational executable events; the study plan can be automatically generated according to the visit cycle.\n\n\nRule configuration\n\nConfigures rules required for subject management (such as inclusion and exclusion criteria, various number generation rules, and drug randomization methods).\n\n\nRunning projects list\n\nContains responsible or participating running clinical trial projects, with access management.\n\n\nSubject list\n\nContains a list of subjects recruited for this study project, with different colors to distinguish the status of subjects.\n\n\nSubject recruitment\n\nSubjects\u2019 information can be linked by searching the hospital\u2019s patient database and alerted if they are enrolled in other clinical trials.\n\n\nSubject study process management\n\nManages the entire process of subjects from recruitment to the end of the visit, performs the study content required in the corresponding visit cycle (such as screening, randomization, inspection, and investigational product), and can query all the data required for clinical trials.\n\n\nStudy progress statistics\n\nGenerates statistics based on the distribution of subjects in different dimensions, including recruitment date, visit cycle, and different states (drop out, withdraw, completion, etc.).\n\n\nInvestigational product management\n\nStock management\n\nManages the inbound, storage, outbound, and refund of investigational product; the inventory quantity can be classified and counted by dictionary, batch, and random code.\n\n\nPrescription management\n\nManages the closed-loop process of prescription issuance, verification, distribution, use, and refund of investigational product, and supports different blinding methods (such as open, single-blinding, and double-blinding), which can be traced.\n\n\nLabel management\n\nAll circulation links of investigational product support automatic label scanning and verification (including inbound, outbound, use, etc.).\n\n\nIntelligent detection\n\nIntelligent early warning and verification of the management process of investigational product (such as near expiration date early warning, inventory early warning, and rule-based verification).\n\n\nFinancial management\n\nPayment and allocation management\n\nManages the financial process of payment addition, allocation (such as subject fee, investigator fee, and inspection fee), review, etc.\n\n\nBudget management\n\nManages the budget associated with the study project, including budget list, budget summary, and budget adjustment.\n\n\nExpenditure management\n\nManages all kinds of expenses and related study project funds.\n\n\nWorkload statistics\n\nCounts the corresponding workload in different dimensions (such as executive departments, project contracts, billing departments, and cost accounting) to produce financial statements.\n\n\nProject funding amount\n\nManages the summary and detailed list of various expenses of study projects (such as subject fee, investigator fee, and inspection fee).\n\n\nQuality management\n\nQuality control data reporting\n\nManages the reporting of various quality control data (such as mid-term study quality control, study monitoring, study audit, and site audit).\n\n\nQuality control element capture\n\nAutomatically obtains corresponding quality control elements during clinical trial running (such as deviation protocol, out of visit window, and adverse events).\n\n\nOperational data query\n\nWith authorization, the quality controller can query the running data of the study project in real-time.\n\n\n\nThe third level is backstage management, which is indispensable for the stable operation of the CTMS. Table 4 shows the categories and features for backstage management of clinical trials. In this level, the proposed system provides services to users through the PMMS. Unified management of all user information and data access permissions of the system through user and permission management provides a basis for the realization of the collaboration of users with different roles. Log management identifies the problems in the system operation and records all the data changes for verification. These two points are also an indispensable part of achieving secure data access. Data statistics are recorded according to the requirements of CTI to provide managers with statistical charts of study projects and operational data to provide decision support capability.\n\n\n\n\n\n\n\nTable 4. Categories and features for backstage management of clinical trials\n\n\n\nCategory\n\nFeatures\n\nDescription\n\n\nUser and permission management\n\nUser registration\n\nNon-hospital users apply for registration and approval mode (such as clinical research coordinator [CRC] and clinical research associate [CRA]).\n\n\nUser list\n\nUser list contains all the users in the system, managing login policies and permissions.\n\n\nPermission management\n\nRealizes the role-based authority management system.\n\n\nLog management\n\nSystem log\n\nIncludes system operation and data change logs, traceable to the source of changes.\n\n\nData dictionary management\n\nBasic data maintenance\n\nBasic data support for the operation of the clinical trial management system (CTMS) can be maintained through a graphical interface.\n\n\nBasic data synchronization\n\nSets up synchronization task; part of the basic data is automatically obtained from other business systems by means of timing synchronization.\n\n\nData statistics\n\nData statistics\n\nStatistical analysis capabilities of various data, support tables, and graphs.\n\n\nSystem settings\n\nSystem settings\n\nSystem customization functions to enable the system to operate more intelligently (such as messages and templates).\n\n\n\nEnterprise process management for clinical trials \nFAHZU CTMS realizes the entire process management of clinical trials through the organic combination of project approval and reviews management, clinical trial operational management, and backstage entire process management, which is further categorized into 59 internal processes by combining user and authority management (Fig. 2). Clinical trial project approval and review management consists of 14 main processes and eight auxiliary processes, which can efficiently complete the review and contract signature phases of the study project; a kick-off meeting is held to allow entry into clinical trial operational management. Clinical trial operational management is a complex and long-term process involving multiple factors such as subjects, diagnosis and treatment, medicine, and finance. We divide it into six stages: operational management pre-configuration, subject recruitment, subject visit management, investigational product management, financial management, and quality management, and then further subdivide these stages into 32 internal processes. Finally, backstage management, as the basic component, supports the stable operation of the entire clinical trial process.\n\r\n\n\n\n\n\n\n\n\n\n\nFig. 2 Process management system further categorized into 59 internal processes by combining user and authority management. CRA: clinical research associate; CRC: clinical research coordinator; CTI: clinical trial institutes; PI: principal investigator; EC: ethics committee.\n\n\n\nData integration with external systems \nTo use the CTMS as a dedicated database for clinical trials, data integration with external systems is indispensable. Figure 3 shows some of the critical data integration services between the CTMS and external systems. CTMS subsystems integrate with external systems such as HIS, LIS, EMR, and CDR through the interface service platform to achieve data service standardization and transmission process encryption. The data integration method supports extract-transform-load (ETL), message queue (MQ), and remote procedure call (RPC) services and is flexibly selected based on the data volume and business model. Data synchronization and data queries involve a large amount of data. The operations use ETL batch extraction and RPC service real-time queries, which is used for data dictionary maintenance, as well as queries of subjects\u2019 full diagnosis and treatment data, and e-CRF structured data filling. Subject information data and medical order data use the synchronous call of RPC service and asynchronous notification of MQ. Synchronous calling is used to obtain the patient information of the subject recruitment, the subject\u2019s diagnosis and treatment items, and prescription issuance, billing, etc. Asynchronous calling is used for notification of changes in the subject\u2019s status, obtaining medical order status, obtaining results, etc.\n\r\n\n\n\n\n\n\n\n\n\n\nFig. 3 Critical services for data integration between the clinical trial management system (CTMS) and external systems. HIS: hospital information system; EMR: electronic medical record; LIS: laboratory information system; CDR: clinical data repository; ETL: extract-transform-load; MQ: message queue; RPC: remote procedure call; CTMS: clinical trial management system; PMMS: permission management and maintenance system; CTSMS: clinical trial subject management system; CTIPMS: clinical trial investigational product management system; CTFMS: clinical trial financial management system; CTSMS: clinical trial subject management system; CTQMS: clinical trial quality management system.\n\n\n\nData security and privacy protection \nData security and privacy protection ensure that a mature process is developed and maintained throughout the design life cycle. Data security primarily involves data storage and data access security. Privacy protection focuses on ensuring data security while complying with laws and regulations to de-identify sensitive patient information.\nData security mainly includes the two aspects of data storage security and data access security. In terms of data storage security, FAHZU CTMS adopts two strategies: (1) multi-node storage of data to achieve high availability of data services and a data backup strategy to ensure that data will not be lost in case of failure, and (2) encrypted storage of sensitive data (e.g., passwords, ID numbers, and bank card numbers). Policies for data access security are:\n\nSecurity at the network level, through safety equipment (such as a firewall, host security protection, bastion host, and database\/log audit tools), to implement access control and prevent illegal access;\nSecurity at the application level through multiple roles of authority management system access controls on data content, especially including rules to ensure no open access to prescriptions (e.g., CRC non-prescription issuance authority), as well as finding and repairing the vulnerabilities of the application through penetration testing and completing the information security technology-evaluation requirement for classified protection of cybersecurity\u2014Level 3; and\nMaintaining a detailed data change log can be useful in tracing the record of the process of data recovery.\nSubject privacy protection aims to protect sensitive patient information through a series of de-identification processes, which are explicitly required in the GCP. The FAHZU CTMS strictly implements the requirements of subject privacy protection in the GCP guidelines and refers to the Information Security Technology Personal Information Security Specification issued by the Standardization Administration of China and the HIPAA Regulations issued by the U.S. Department of Health. Final compliance with HIPAA\u2019s implementation specifications requiring de-identification of protected health information excludes 18 personal health identifiers (PHIs) from the CTMS (e.g., name, address, cell phone number, and social security number) to comply with international and national laws. For CTMS users to identify subjects, we used the clinical trial protocol number, subject screening number, and subject name initials to uniquely identify them. At the level of data interaction, the patients\u2019 medical record numbers in the hospital are systematically encrypted and stored in the CTMS database, which is linked to the data of the external clinical and business systems (i.e., HIS, LIS, EMR, CDR, etc.) for automatic data transmission.\n\nEvaluation \nThe evaluation of the CTMS is ongoing and mainly conducted at two levels: clinical trial project approval and review management, and clinical trial operational management. The former focuses on the convenience of multi-party collaboration and the efficiency of project approval, whereas the latter focuses on the evaluation of subject-centered data management throughout the process and the improvement of the quality of clinical trials.\nThe improved clinical trial project approval and review management after applying the FAHZU CTMS, compared with that before the system was implemented, mainly embodies two distinct advantages. First, it realizes process automation management based on task and, combined with workflow technology, cooperates with user authority management systems, realizes automatic triggering of node events, and performs automatic assignment of processing personnel and message notifications such that personnel only need to process their personal to-do lists according to the message reminder, reducing the complexity of system use under multi-party collaboration. Second, clinical trial project approval supports remote application, and electronic document management enables online submission of project approval materials, as well as online review and summary of revision opinions in the review process, which reduces the workload of project approval materials review and improves the efficiency of project review. Since CTMS began to operate in the hospital in December 2014, 1,388 study projects have been accepted and 43,051 documents have been submitted through CTMS as of March 31, 2021.\nThe focus at the clinical trial operational management level is centered on patient outcomes to achieve full-cycle data management. Here, the advantages are more significant. The four core advantages are summarized as follows: \n\nThe CTMS requires that the research plan must be entered before the recruitment of subjects, and the visit content of the current visit cycle can be automatically correlated during the subject visit. It is transformed into to-do tasks in the current research stage, with timely reminders, reducing deviations from the protocol.\nAn independent billing model is adopted for clinical trial-related inspection and treatment expenses so that subjects can be exempted from expense reimbursement and be marked in the hospital business system to meet non-clinical trial diagnosis and treatment reminders and clinical trial inspection green special requirements such as channels.\nThe CTMS contains the data on each subject, including newly generated data during operation and data collected in the hospital business system. Through strict authority classification, the scope of the data queries, such as those issued by the CRC and other data query authorities, are limited to subjects who are responsible for the project.\nClinical trial drugs are label-based full-cycle closed-loop managed, and transfers are completed through label scanning, and support some special properties of clinical trial drugs, such as open, single-blind, and double-blind studies, and situations with other prescriptions, where drugs need to be random, and when the serial numbers and medicine need to be recalled.\nThe clinical trial operational management of CTMS began a pilot run at FAHZU in March 2015. Pilot runs are conducted to validate and improve the effectiveness of system functions and compatibility with different types of clinical trials. Since September 2015, all newly initiated clinical trials on the site have been managed through CTMS. According to statistics, as of March 31, 2021, a total of 12,144 subjects have been included in the management of CTMS, across 472 study projects. Figure 4 shows the change in the number of subjects and corresponding study projects enrolled in CTMS in the last two years. We observed that the monthly number of new subjects and the corresponding study projects remained stable, and the number of active subjects also remained stable, while the corresponding monthly number of active projects increased. Since the outbreak of COVID-19 in China in December 2019, with the support of the integrated CTMS and necessary measures (e.g., remote follow-up, express delivery, and remote inspection), it may be observed from Fig. 4 that the clinical trials have continued at a steady rate during the COVID-19 outbreak.\n\r\n\n\n\n\n\n\n\n\n\n\nFig. 4 Change in the number of subjects and corresponding study projects enrolled in the clinical trial management system (CTMS) over the last two years.\n\n\n\nDiscussion \nDigitalization is an inevitable trend in clinical trial management.[9] At present, there are many electronic clinical trial management systems in China; however, there are few systems that can connect with the information system of the clinical research center and unify the sponsor, data management department, and other parties into a platform for cooperation. The FAHZU CTMS has been researched and developed fully independently based on a distributed service architecture. It takes process management and trial data as the core; highly integrates, interconnects, and interfuses with hospital clinical business systems; and combines key contents such as data security and privacy protection to achieve independent application layers and interconnected data layers. Thus, comprehensive process management and dynamic real-time monitoring of clinical trials can be realized. Compared with commercial products, our self-developed CTMS is more cost-effective and highly customizable, fully fits the management needs of the hospital as a clinical trial administrative institution, and allows more timely system version updates, system operation, and maintenance response. Based on the deep understanding of hospital clinical business systems such as HIS, LIS, CDR, PACS, and EMR by information technology experts, a more comprehensive scheme was designed to achieve a high degree of integration between the CTMS and clinical business systems. The large amounts of medical data required for clinical trials are docked into the developed system and used as a CTMS-independent data collection and storage subsystem. In addition, based on the full understanding of data security and privacy protection, the security of the system was assigned great importance from the beginning of system development, and the privacy protection function of subjects was improved to better serve the entire process management of clinical trials in our institution. \nFrom an operational point of view, the use of the system has significantly improved the efficiency of researchers and institutional managers. For the institutional management, the system realizes real-time and efficient management of the entire process of clinical trials and ensures the reliability, authenticity, and integrity of clinical trial results. To ensure the safety of subjects, it realizes limited sharing of information through the data query, statistics, and tracking module. Institutional managers can view the project schedule and browse project-related node information, to accurately grasp the implementation of the project status, improve the quality of clinical trial data, reduce time consumption, and promote the standardized implementation of clinical trials according to the GCP guidelines and SOP of our hospital. On this basis, a full-cycle data quality monitoring and early warning platform for clinical trials can be constructed. Through artificial intelligence (AI) technologies such as text mining, natural language processing, machine learning, and knowledge mapping, intelligent full-cycle data analysis and early warning in clinical trials can be realized. These include the matching of inclusion and exclusion criteria, warning of inspection, warning of combined drug prohibition, warning of absence of visit activity or out of window, and intelligent warning of underreporting of AE or SAE.\nThe protection of private information is a more important consideration than the system construction.[10][11][12] The biggest problem with the establishment of the system is that the subject information may be exposed, which is a serious ethical problem. To solve potential risks from the level of laws and regulations, it is necessary to think deeply. Therefore, at the beginning of the design, we paid special attention to the concept of network data security and privacy protection, carried out the privacy impact assessment, and integrated the measures of privacy protection into the entire process of information system development. Institutional managers, researchers, quality controllers, CRCs, drug administrators, and inspectors are divided into different users. There is a strict permission management system for user accounts, to avoid the problem of account borrowing, certificate authority authentication or face recognition systems should be added to adopt fine-grained permission management. In addition, the SOP for risk assessment is particularly important, including the definition, classification, and rating of risks.[13] Only information management systems that meet the requirements of risk assessment can be operated. In the information age, everything is connected, and information is a trend.[14][15] In today's highly developed internet technology, we need to think about and solve the problems of data security and personal privacy, as well as the specific procedures to improve the efficiency of clinical research. At the same time, compliance with the Chinese GCP and International Conference on Harmonisation (ICH) requirements for GCP ensure compliance with ethical and legal provisions.\nCompared with the traditional paper CRF, e-CRF allows researchers to enter the CRF electronically based on the source data rather than fill it in manually.[12] However, source data verification (SDV) is still required to compare the CRF with original medical records and inspection lists to ensure the quality of data input. Our CTMS is now able to automate the collection of structured data, eliminate the SDV process, and further simplify the process. Thus, any modifications to the electronic source data can be recorded through an audit trail. In the future, our system will also study the complex natural language analysis involved in unstructured data. Simultaneously, we will build a way to connect electronic data acquisition (EDC) and hospital e-CRF to directly collect and transfer clinical data electronically, ensuring the quality and integrity of the data.\nIn the future, we also want to utilize the data in the clinical trial management platform to realize intelligent subject recruitment and improve the efficiency of subject recruitment. Recruiting subjects for some clinical trials is difficult, especially those involving rare diseases, stringent admission criteria, and special subgroups. Owing to information asymmetry, researchers only know the condition of the patients they are treating, and patients do not know that their disease may be under research at the hospital, which can seriously affect the progress of clinical trials.[16][17][18][19] To fully utilize the hospital CTMS system of clinical data, the central hospital and administering medical hospital can simultaneously be on the same clinical trial management platform, which will further enrich the patient resources. Additionally, the system can provide search functions, select exclusion criteria, fast-matching potential subjects, and realize intelligent recruitment of subjects, which can effectively help the sponsor accelerate the clinical trial process, reduce R&D costs, and successfully seize market opportunities. However, in the era of big data, these data have important scientific value in the field of clinical research. Applications such as large data analysis found that local residents\u2019 disease condition, and its influencing factors\u2014specific studies on key diseases\u2014have been successfully applied to determine the time of disease distribution, location distribution, population distribution, and analysis of risk factors of disease; furthermore, it has been used for the evaluation of the effectiveness of clinical screening and diagnosis methods, inspection treatment or drug treatment effect, optimization of individual diagnosis and treatment of disease, and research on the influencing factors of diseases after intervention. This is to provide the basis for the local health administration departments to make health management decisions.\nSince the outbreak of COVID-19, many jobs around the world have been stalled to a certain extent. In the event of a major public health emergency, carrying out clinical trials and ensuring the smooth implementation of monitoring work has become a problem for clinical trial practitioners, and remote monitoring has therefore been put on the agenda. The U.S. Food and Drug Administration (FDA) has long encouraged more centralized monitoring, where inspectors perform inspections in the office using relevant information tools rather than at research institutions (hospitals). The development trend of clinical trial monitoring is to replace on-site monitoring with centralized monitoring.[20] Remote monitoring can improve the quality and efficiency of clinical research and reduce its cost, and it is possible only when a series of electronic clinical trial products such as CTMS, electronic data acquisition, EMR, and clinical data management systems are widely used. The clinical trial information management system of our hospital is a CTMS that unites multiple teams on one platform and is highly integrated with all clinical systems of the hospital. As a web-based platform, remote data monitoring and cloud auditing can be included on the CTMS as a mature operation.\nCompared with other information systems, CTMSs are more professional and personalized. The realization of the effectiveness of a CTMS requires a significant amount of time, and the more clinical trials undertaken, the more significant the effect. Compared with the system design, the comprehensive and efficient application of the system takes longer to achieve. With the digitization of clinical research, the sharing and integration of research data will bring many management advantages, such as an increase in available management resources, scientific and data support for major decisions, the convenience of remote management, pertinacity, and pre-operation. The CTMS developed by our hospital will be constantly updated and upgraded, and its functions will constantly improve. The system update will keep pace with developments in international drug clinical trial management, promote the development of clinical trials in a more standardized direction, and promote the disciplinary status of the hospital regarding high-level clinical trials.\n\nConclusion \nThe FAHZU CTMS, as the first integrated CTMS independently developed by a hospital in China, can better adapt to the institutional needs for individualized, whole-process, and dynamically comprehensive evaluation and supervision of clinical trials. The integrated CTMS contains three levels and seven subsystems, which fully realizes the whole-process data management of clinical trials from project approval and review management to operational management. Through the unified interface system, the developed CTMS provides a variety of access methods to complete efficient data integration with the clinical business systems, and applies multiple security policies combined with privacy protection methods to effectively ensure the security of data and the privacy of subjects during clinical trial operation. The operation results based on the integrated CTMS show that it can effectively control the risks in the clinical trial process, so as to improve the science, safety, and timeliness of the new drug development process.\n\n Abbreviations, acronyms, and initialisms \nAE: adverse event\nAI: artificial intelligence\nCDISC: Clinical Data Interchange Standards Consortium\nCDR: clinical data repository\nCRA: clinical research associate\nCRC: clinical research coordinator\nCTEMS: clinical trial ethical management system\nCTFMS: clinical trial financial management system\nCTI: clinical trial institution\nCTIPMS: clinical trial investigational product management system\nCTMS: clinical trial management system\nCTPMS: clinical trial project management system\nCTQMS: clinical trial quality management system\nCTSMS: clinical trial subject management system\ne-CRF: electronic case report form\nEC: ethics committee\nEMR: electronic medical record\nETL: extract-transform-load\nFAHZU: First Affiliated Hospital, Zhejiang University School of Medicine\nCP: good clinical practice\nHIPAA: Health Insurance Portability and Accountability Act\nHIS: hospital information system\nHTTPS: hypertext transfer protocol secure\nICH: International Conference on Harmonisation\nLAN: local area network\nLIS: laboratory information system\nPACS: picture archiving and communication systems\nPHI: personal health identifier\nPI: principal investigator\nPMMS: permission management and maintenance system\nR&D: research and development\nRPC: remote procedure call\nSAE: serious adverse events\nSDV: source data verification\nSSO: single sign-on\nUI: user interface\nUX: user experience\nAcknowledgements \nAuthor contributions \nLS conceived the concept of the project, was responsible for the architecture and integration concept, designed and developed the system, and wrote the manuscript. YZ collected and sorted out the requirements for the construction of the clinical trial management system and collected references. AXP and QWZ designed and organized the tables and figures in the article. MZ and JL were responsible for conceptualization and formal analysis, reviewed the manuscript, and supplemented the discussion section. All authors read and approved the final manuscript.\n\nEthics approval and consent to participate \nThere were no individual-level data for the study, so ethics committee approval was not required. All executive trials included in this study system were the Clinical Trial Ethics Committee (EC) of The First Affiliated Hospital, Zhejiang University School of Medicine (FAHZU) reviewed and approved. All subjects in these trials signed informed consents prior to enrollment.\n\nFunding \nThis study was funded by the New Drug Creation Project of The 13th Five-Year National Science and Technology Major Special Project (2020ZX09201-003).\n\nAvailability of data and materials \nFAHZU CTMS consists of several subsystems. Considering data security and privacy protection, subsystems associated with subject data are deployed based on the hospital\u2019s internal network, and domain names are resolved by a self-built domain name systemserver. The following four links show part of the web pages of the core subsystem, which can show the function design and data volume of the system. For additional information please contact the corresponding author.\n1. https:\/\/doi.org\/10.5281\/zenodo.5880663\n2. https:\/\/doi.org\/10.5281\/zenodo.5880783\n3. https:\/\/doi.org\/10.5281\/zenodo.5880799\n4. https:\/\/doi.org\/10.5281\/zenodo.5880826\n\nSoftware availability and requirements \nProject name: FAHZU CTMS\nProject home page: https:\/\/ctms.zy91.com\nOperating system(s): Platform-independent\nProgramming language: Java\nOther requirements: Java 1.7.1 or higher, Tomcat 7.0 or higher, Nginx 1.14.2 or higher, Oracle Database 11 g Release 2, Redis 4.0\nLicense: Free for academics\nAny restrictions to use by non-academics: Contact authors\n\nCompeting interests \nThe authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.\n\nReferences \n\n\n\u2191 Kruizinga, M. D.; Stuurman, F. E.; Exadaktylos, V.; Doll, R. J.; Stephenson, D. T.; Groeneveld, G. J.; Driessen, G. J. A.; Cohen, A. F. (1 October 2020). Michel, Martin. ed. \"Development of Novel, Value-Based, Digital Endpoints for Clinical Trials: A Structured Approach Toward Fit-for-Purpose Validation\" (in en). Pharmacological Reviews 72 (4): 899\u2013909. doi:10.1124\/pr.120.000028. ISSN 0031-6997. http:\/\/pharmrev.aspetjournals.org\/lookup\/doi\/10.1124\/pr.120.000028 .   \n \n\n\u2191 Davi, Ruthie; Mahendraratnam, Nirosha; Chatterjee, Arnaub; Dawson, C. Jill; Sherman, Rachel (1 December 2020). \"Informing single-arm clinical trials with external controls\" (in en). Nature Reviews Drug Discovery 19 (12): 821\u2013822. doi:10.1038\/d41573-020-00146-5. ISSN 1474-1776. https:\/\/www.nature.com\/articles\/d41573-020-00146-5 .   \n \n\n\u2191 the COVID19 and Cancer Clinical Trials Working Group; Desai, Aakash; Gainor, Justin F.; Hegde, Aparna; Schram, Alison M.; Curigliano, Giuseppe; Pal, Sumanta; Liu, Stephen V. et al. (1 May 2021). \"Author Correction: COVID-19 vaccine guidance for patients with cancer participating in oncology clinical trials\" (in en). Nature Reviews Clinical Oncology 18 (5): 320\u2013320. doi:10.1038\/s41571-021-00503-2. ISSN 1759-4774. PMC PMC7985918. PMID 33758378. https:\/\/www.nature.com\/articles\/s41571-021-00503-2 .   \n \n\n\u2191 Manem, Venkata S.K.; Salgado, Roberto; Aftimos, Philippe; Sotiriou, Christos; Haibe-Kains, Benjamin (1 October 2018). \"Network science in clinical trials: A patient-centered approach\" (in en). Seminars in Cancer Biology 52: 135\u2013150. doi:10.1016\/j.semcancer.2017.12.006. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1044579X17302365 .   \n \n\n\u2191 Tan, Eng-King (1 January 2021). \"Movement disorders in 2020: clinical trials, genetic discoveries, and COVID-19\" (in en). The Lancet Neurology 20 (1): 10\u201312. doi:10.1016\/S1474-4422(20)30448-8. PMC PMC7833604. PMID 33340472. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1474442220304488 .   \n \n\n\u2191 Bhagat, Seema; Kapatkar, Vaibhavi K.; Mane, Ashish; Pinto, Colette; Parikh, Devang; Mittal, Gaurav; Jain, Rishi (14 February 2020). \"An Industry Perspective on Risks and Mitigation Strategies Associated with Post Conduct Phase of Clinical Trial\" (in en). Reviews on Recent Clinical Trials 15 (1): 28\u201333. doi:10.2174\/1574887114666191016103332. http:\/\/www.eurekaselect.com\/175736\/article .   \n \n\n\u2191 Nourani, Aynaz; Ayatollahi, Haleh; Dodaran, Masoud Solaymani (30 January 2019). \"A Review of Clinical Data Management Systems Used in Clinical Trials\" (in en). Reviews on Recent Clinical Trials 14 (1): 10\u201323. doi:10.2174\/1574887113666180924165230. http:\/\/www.eurekaselect.com\/165619\/article .   \n \n\n\u2191 Park, Yu Rang; Yoon, Young Jo; Koo, HaYeong; Yoo, Soyoung; Choi, Chang-Min; Beck, Sung-Ho; Kim, Tae Won (24 April 2018). \"Utilization of a Clinical Trial Management System for the Whole Clinical Trial Process as an Integrated Database: System Development\" (in en). Journal of Medical Internet Research 20 (4): e103. doi:10.2196\/jmir.9312. ISSN 1438-8871. PMC PMC5941091. PMID 29691212. http:\/\/www.jmir.org\/2018\/4\/e103\/ .   \n \n\n\u2191 Nourani, Aynaz; Ayatollahi, Haleh; Dodaran, Masoud Solaymani (21 August 2019). \"Clinical Trial Data Management Software: A Review of the Technical Features\" (in en). Reviews on Recent Clinical Trials 14 (3): 160\u2013172. doi:10.2174\/1574887114666190207151500. http:\/\/www.eurekaselect.com\/169754\/article .   \n \n\n\u2191 Barlow, Candida (1 April 2020). \"Human Subjects Protection and Federal Regulations of Clinical Trials\" (in en). Seminars in Oncology Nursing 36 (2): 151001. doi:10.1016\/j.soncn.2020.151001. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0749208120300164 .   \n \n\n\u2191 Barlow, Candida (1 April 2020). \"Oncology Research: Clinical Trial Management Systems, Electronic Medical Record, and Artificial Intelligence\" (in en). Seminars in Oncology Nursing 36 (2): 151005. doi:10.1016\/j.soncn.2020.151005. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0749208120300206 .   \n \n\n\u2191 12.0 12.1 Finniss, Damien G; Kaptchuk, Ted J; Miller, Franklin; Benedetti, Fabrizio (1 February 2010). \"Biological, clinical, and ethical advances of placebo effects\" (in en). The Lancet 375 (9715): 686\u2013695. doi:10.1016\/S0140-6736(09)61706-2. PMC PMC2832199. PMID 20171404. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0140673609617062 .   \n \n\n\u2191 J\u00f8rgensen, Lars; Paludan-M\u00fcller, Asger S.; Laursen, David R. T.; Savovi\u0107, Jelena; Boutron, Isabelle; Sterne, Jonathan A. C.; Higgins, Julian P. T.; Hr\u00f3bjartsson, Asbj\u00f8rn (1 December 2016). \"Evaluation of the Cochrane tool for assessing risk of bias in randomized clinical trials: overview of published comments and analysis of user practice in Cochrane and non-Cochrane reviews\" (in en). Systematic Reviews 5 (1): 80. doi:10.1186\/s13643-016-0259-8. ISSN 2046-4053. PMC PMC4862216. PMID 27160280. http:\/\/systematicreviewsjournal.biomedcentral.com\/articles\/10.1186\/s13643-016-0259-8 .   \n \n\n\u2191 Cowie, Martin R.; Blomster, Juuso I.; Curtis, Lesley H.; Duclaux, Sylvie; Ford, Ian; Fritz, Fleur; Goldman, Samantha; Janmohamed, Salim et al. (1 January 2017). \"Electronic health records to facilitate clinical research\" (in en). Clinical Research in Cardiology 106 (1): 1\u20139. doi:10.1007\/s00392-016-1025-6. ISSN 1861-0684. PMC PMC5226988. PMID 27557678. http:\/\/link.springer.com\/10.1007\/s00392-016-1025-6 .   \n \n\n\u2191 Sharma, Abhinav; Harrington, Robert A.; McClellan, Mark B.; Turakhia, Mintu P.; Eapen, Zubin J.; Steinhubl, Steven; Mault, James R.; Majmudar, Maulik D. et al. (1 June 2018). \"Using Digital Health Technology to Better Generate Evidence and Deliver Evidence-Based Care\" (in en). Journal of the American College of Cardiology 71 (23): 2680\u20132690. doi:10.1016\/j.jacc.2018.03.523. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0735109718344139 .   \n \n\n\u2191 Kempf, Lucas; Goldsmith, Jonathan C.; Temple, Robert (1 April 2018). \"Challenges of developing and conducting clinical trials in rare disorders\" (in en). American Journal of Medical Genetics Part A 176 (4): 773\u2013783. doi:10.1002\/ajmg.a.38413. ISSN 1552-4825. https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/ajmg.a.38413 .   \n \n\n\u2191 Brasil, Sandra; Pascoal, Carlota; Francisco, Rita; dos Reis Ferreira, Vanessa; A. Videira, Paula; Valad\u00e3o, Gon\u00e7alo (27 November 2019). \"Artificial Intelligence (AI) in Rare Diseases: Is the Future Brighter?\" (in en). Genes 10 (12): 978. doi:10.3390\/genes10120978. ISSN 2073-4425. PMC PMC6947640. PMID 31783696. https:\/\/www.mdpi.com\/2073-4425\/10\/12\/978 .   \n \n\n\u2191 Wu, Jasmanda; Wang, Cunlin; Toh, Sengwee; Pisa, Federica Edith; Bauer, Larry (1 October 2020). \"Use of real\u2010world evidence in regulatory decisions for rare diseases in the United States\u2014Current status and future directions\" (in en). Pharmacoepidemiology and Drug Safety 29 (10): 1213\u20131218. doi:10.1002\/pds.4962. ISSN 1053-8569. https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/pds.4962 .   \n \n\n\u2191 Groft, Stephen C.; Posada de la Paz, Manuel (2017), Posada de la Paz, Manuel; Taruscio, Domenica; Groft, Stephen C., eds., \"Preparing for the Future of Rare Diseases\", Rare Diseases Epidemiology: Update and Overview (Cham: Springer International Publishing) 1031: 641\u2013648, doi:10.1007\/978-3-319-67144-4_34, ISBN 978-3-319-67142-0, http:\/\/link.springer.com\/10.1007\/978-3-319-67144-4_34 . Retrieved 2023-06-28   \n \n\n\u2191 Hurley, Caroline; Shiely, Frances; Power, Jessica; Clarke, Mike; Eustace, Joseph A.; Flanagan, Evelyn; Kearney, Patricia M. (1 November 2016). \"Risk based monitoring (RBM) tools for clinical trials: A systematic review\" (in en). Contemporary Clinical Trials 51: 15\u201327. doi:10.1016\/j.cct.2016.09.003. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1551714416302877 .   \n \n\n\nNotes \nThis presentation is faithful to the original, with only a few minor changes to presentation, grammar, and punctuation. In some cases important information was missing from the references, and that information was added.\n\n\n\n\n\nSource: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.limswiki.org\/index.php\/Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system\">https:\/\/www.limswiki.org\/index.php\/Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system<\/a>\nCategories: LIMSwiki journal articles (added in 2023)LIMSwiki journal articles (all)LIMSwiki journal articles on clinical informaticsLIMSwiki journal articles on clinical researchNavigation menuPage actionsJournalDiscussionView sourceHistoryPage actionsJournalDiscussionMoreToolsIn other languagesPersonal toolsLog inNavigationMain pageEncyclopedic articlesRecent changesRandom pageHelp about MediaWikiSearch\u00a0 ToolsWhat links hereRelated changesSpecial pagesPermanent linkPage informationPopular publications\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\nPrint\/exportCreate a bookDownload as PDFDownload as PDFDownload as Plain textPrintable version This page was last edited on 28 June 2023, at 17:50.Content is available under a Creative Commons Attribution-ShareAlike 4.0 International License unless otherwise noted.This page has been accessed 532 times.Privacy policyAbout LIMSWikiDisclaimers\n\n\n\n","e65552a701c8075f24aa45db5f398e2e_html":"<body class=\"mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-206 ns-subject page-Journal_Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system rootpage-Journal_Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system skin-monobook action-view skin--responsive\"><div id=\"rdp-ebb-globalWrapper\"><div id=\"rdp-ebb-column-content\"><div id=\"rdp-ebb-content\" class=\"mw-body\" role=\"main\"><a id=\"rdp-ebb-top\"><\/a>\n<h1 id=\"rdp-ebb-firstHeading\" class=\"firstHeading\" lang=\"en\">Journal:Development of an integrated and comprehensive clinical trial process management system<\/h1><div id=\"rdp-ebb-bodyContent\" class=\"mw-body-content\"><!-- start content --><div id=\"rdp-ebb-mw-content-text\" lang=\"en\" dir=\"ltr\" class=\"mw-content-ltr\"><div class=\"mw-parser-output\">\n\n\n<h2><span class=\"mw-headline\" id=\"Abstract\">Abstract<\/span><\/h2>\n<p><b>Background<\/b>: The process of initiating and completing clinical drug trials in <a href=\"https:\/\/www.limswiki.org\/index.php\/Hospital\" title=\"Hospital\" class=\"wiki-link\" data-key=\"b8f070c66d8123fe91063594befebdff\">hospital<\/a> settings is highly complex, with numerous institutional, technical, and record-keeping barriers. In this study, we independently developed an integrated <a href=\"https:\/\/www.limswiki.org\/index.php\/Clinical_trial_management_system\" title=\"Clinical trial management system\" class=\"wiki-link\" data-key=\"69c3d457afb8e96412b08403b7bfcccb\">clinical trial management system<\/a> (CTMS) designed to comprehensively optimize the process management of clinical trials. The CTMS includes system development methods, efficient integration with external business systems, terminology, and standardization protocols, as well as <a href=\"https:\/\/www.limswiki.org\/index.php\/Information_security\" title=\"Information security\" class=\"wiki-link\" data-key=\"9eff362d944224ff1d4ffe3a149d7cff\">data security<\/a> and <a href=\"https:\/\/www.limswiki.org\/index.php\/Information_privacy\" title=\"Information privacy\" class=\"wiki-link\" data-key=\"185f6d9f874e48914b5789317408f782\">privacy<\/a> protection.\n<\/p><p><b>Methods<\/b>: The development process proceeded through four stages, including demand analysis and problem collection, system design, system development and testing, system trial operation, and training the whole hospital to operate the system. The integrated CTMS comprises three modules: project approval and review management, clinical trial operations management, and background management modules. These are divided into seven subsystems and 59 internal processes, realizing all the functions necessary to comprehensively perform the process management of clinical trials. Efficient data integration is realized through extract-transform-load (ETL), message queue, and remote procedure call services with external systems such as the <a href=\"https:\/\/www.limswiki.org\/index.php\/Hospital_information_system\" title=\"Hospital information system\" class=\"wiki-link\" data-key=\"d8385de7b1f39a39d793f8ce349b448d\">hospital information system<\/a> (HIS), <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory_information_system\" title=\"Laboratory information system\" class=\"wiki-link\" data-key=\"37add65b4d1c678b382a7d4817a9cf64\">laboratory information system<\/a> (LIS), <a href=\"https:\/\/www.limswiki.org\/index.php\/Electronic_medical_record\" title=\"Electronic medical record\" class=\"wiki-link\" data-key=\"99a695d2af23397807da0537d29d0be7\">electronic medical record<\/a> (EMR), and <a href=\"https:\/\/www.limswiki.org\/index.php\/Clinical_data_management_system\" title=\"Clinical data management system\" class=\"wiki-link\" data-key=\"398ab2671bc637e95d720977c6762190\">clinical data repository<\/a> (CDR). Data security is ensured by adopting corresponding policies for data storage and data access. Privacy protection complies with laws and regulations and de-identifies sensitive patient information.\n<\/p><p><b>Results<\/b>: The integrated CTMS was successfully developed in September 2015 and updated to version 4.2.5 in March 2021. During this period, 1,388 study projects were accepted, 43,051 electronic data files stored, and 12,144 subjects recruited in the First Affiliated Hospital, Zhejiang University School of Medicine.\n<\/p><p><b>Conclusion<\/b>: The developed integrated CTMS realizes the <a href=\"https:\/\/www.limswiki.org\/index.php\/Information_management\" title=\"Information management\" class=\"wiki-link\" data-key=\"f8672d270c0750a858ed940158ca0a73\">data management<\/a> of the entire clinical trials process, providing basic conditions for the efficient, high-quality, and standardized operation of clinical trials.\n<\/p><p><b>Keywords<\/b>: clinical trial, clinical trial management system (CTMS), information technology, integrated management system, medical informatics\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Background\">Background<\/span><\/h2>\n<p>Clinical trials of drugs are an important stage in drug <a href=\"https:\/\/www.limswiki.org\/index.php\/Research\" title=\"Research\" class=\"wiki-link\" data-key=\"409634fd90113f119362927fe222f549\">research<\/a> and development and are means to improve the development of medical science and technology.<sup id=\"rdp-ebb-cite_ref-1\" class=\"reference\"><a href=\"#cite_note-1\">[1]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-2\" class=\"reference\"><a href=\"#cite_note-2\">[2]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-3\" class=\"reference\"><a href=\"#cite_note-3\">[3]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-4\" class=\"reference\"><a href=\"#cite_note-4\">[4]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-5\" class=\"reference\"><a href=\"#cite_note-5\">[5]<\/a><\/sup> Standardizing the management of <a href=\"https:\/\/www.limswiki.org\/index.php\/Medical_research\" title=\"Medical research\" class=\"wiki-link\" data-key=\"0ee7e4e2a32a422d78fe6bd1ab0d1cbc\">clinical research<\/a> is key to simplifying the research process, improving <a href=\"https:\/\/www.limswiki.org\/index.php\/Quality_(business)\" title=\"Quality (business)\" class=\"wiki-link\" data-key=\"c4ac43430d1c3a3a15d1255257aaea37\">quality<\/a>, and ensuring the accuracy, reliability, and integrity of the results, thereby shortening the development cycle of new drugs and accelerating the process of drug registration.<sup id=\"rdp-ebb-cite_ref-6\" class=\"reference\"><a href=\"#cite_note-6\">[6]<\/a><\/sup> \n<\/p><p>In China, each <a href=\"https:\/\/www.limswiki.org\/index.php\/Hospital\" title=\"Hospital\" class=\"wiki-link\" data-key=\"b8f070c66d8123fe91063594befebdff\">hospital<\/a> has a dedicated clinical trial management department, called a clinical trial institution (CTI), responsible for the administrative management of clinical trial operation in the hospital. This involves determining the project leader, formulating research plans with sponsors, reviewing the plans with the ethics committee, signing research contracts, screening and enrolling subjects, originating records of trials, receiving drugs, distributing and recovering, processing or reporting adverse events (AE) or serious adverse events (SAE), managing in-hospital quality, accepting supervision and inspection, summarizing, and filing. The traditional management mode is based on paper documents and involves considerable manual labor, which often leads to untimely <a href=\"https:\/\/www.limswiki.org\/index.php\/Information\" title=\"Information\" class=\"wiki-link\" data-key=\"6300a14d9c2776dcca0999b5ed940e7d\">information<\/a> updates, ineffective management, and unverifiable quality. \n<\/p><p>Today, the rapid development of internet technology has profoundly impacted the mode of clinical trial management. Diverse companies have developed a series of network management systems for clinical trials.<sup id=\"rdp-ebb-cite_ref-7\" class=\"reference\"><a href=\"#cite_note-7\">[7]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-8\" class=\"reference\"><a href=\"#cite_note-8\">[8]<\/a><\/sup> Most of these systems have been developed based on the research and development (R&D) requirements of pharmaceutical companies. Clinical trials are managed in a networked manner, which only handles project-related data and excludes the process management links of the clinical trial organization. These commercialized <a href=\"https:\/\/www.limswiki.org\/index.php\/Clinical_trial_management_system\" title=\"Clinical trial management system\" class=\"wiki-link\" data-key=\"69c3d457afb8e96412b08403b7bfcccb\">clinical trial management systems<\/a> (CTMSs) provide comprehensive clinical trial management services, supporting all types of clinical trials, from Phase I to Phase III, from a trial conducted in a single research center to multinational clinical trials; thus, they improve the quality of clinical trials and the efficiency of <a href=\"https:\/\/www.limswiki.org\/index.php\/Information_management\" title=\"Information management\" class=\"wiki-link\" data-key=\"f8672d270c0750a858ed940158ca0a73\">data management<\/a>. However, from the perspective of drug clinical trial institutions, this type of CTMS cannot satisfy the practical requirements for real-time and effective management of all clinical trials carried out by these institutions. Hence, a new CTMS has been developed as an institutional management model in China. Its development and application are still in their infancy. The CTMS also faces certain incompatibility issues with the hospital database. It is difficult to achieve <a href=\"https:\/\/www.limswiki.org\/index.php\/Data_sharing\" title=\"Data sharing\" class=\"wiki-link\" data-key=\"a99d5fda27f755c693c65864d9286130\">data sharing<\/a> and docking. Moreover, the management method is still relatively primitive. The degree of digitalization, networking, and standardization of clinical trials is relatively low, and it is practically impossible for institutions to effectively manage clinical trials.\n<\/p><p>In recent years, owing to the increase in clinical trial projects in our hospital, professional and standardized information systems were urgently needed to assist in the management of clinical trials throughout the hospital. At present, the need for a centralized clinical trial project management platform reliant on the clinical data system of the hospital itself is intensifying. Based on the local area network (LAN) security architecture of our hospital, we have constructed a CTMS, which organically integrates the clinical trial organization management office, ethics committee, clinical trial center pharmacy, and clinical professional departments. The management system covers the entire process of clinical trials, including trial project establishment, ethical review, signing of agreements, trial implementation, trial conclusion, sponsor management, subject management, follow-up management, and centralized management of pharmacies supporting medication trials. The personnel involved include clinical departments, institutional management offices, central pharmacies, and ethics committees. With the help of the information system, a large number of personnel and complicated work processes are now more organically integrated to realize information sharing and collaborative work. With the help of this CTMS, the implementation of clinical trials can be standardized and the entire process can be traceable. \n<\/p><p>As a site of clinical trials, hospital management involves multi-party collaboration, as well as study project management, subject management, investigational product management, <a href=\"https:\/\/www.limswiki.org\/index.php\/Quality_control\" title=\"Quality control\" class=\"wiki-link\" data-key=\"1e0e0c2eb3e45aff02f5d61799821f0f\">quality control<\/a>, financial management, and other complex business processes. Hospitals must develop dedicated systems to assist the entire process management of clinical trials to ensure their efficient and high-quality operation. \n<\/p><p>The main objectives of this study are as follows. We aim (1) to develop an integrated CTMS as a dedicated database for clinical trials; (2) to achieve efficient data integration of the CTMS with hospital business systems such as the <a href=\"https:\/\/www.limswiki.org\/index.php\/Hospital_information_system\" title=\"Hospital information system\" class=\"wiki-link\" data-key=\"d8385de7b1f39a39d793f8ce349b448d\">hospital information system<\/a> (HIS), <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory_information_system\" title=\"Laboratory information system\" class=\"wiki-link\" data-key=\"37add65b4d1c678b382a7d4817a9cf64\">laboratory information system<\/a> (LIS), <a href=\"https:\/\/www.limswiki.org\/index.php\/Clinical_data_management_system\" title=\"Clinical data management system\" class=\"wiki-link\" data-key=\"398ab2671bc637e95d720977c6762190\">clinical data repository<\/a> (CDR), <a href=\"https:\/\/www.limswiki.org\/index.php\/Picture_archiving_and_communication_system\" title=\"Picture archiving and communication system\" class=\"wiki-link\" data-key=\"523b73ff51fa83663dc0b1d59e6d0f05\">picture archiving and communication system<\/a> (PACS), and <a href=\"https:\/\/www.limswiki.org\/index.php\/Electronic_medical_record\" title=\"Electronic medical record\" class=\"wiki-link\" data-key=\"99a695d2af23397807da0537d29d0be7\">electronic medical record<\/a> (EMR); (3) to facilitate standardization and consistency of terminology in the development of database models and business processes; and (4) to ensure the safety and <a href=\"https:\/\/www.limswiki.org\/index.php\/Information_security\" title=\"Information security\" class=\"wiki-link\" data-key=\"9eff362d944224ff1d4ffe3a149d7cff\">security<\/a> of clinical data as well as protect patient <a href=\"https:\/\/www.limswiki.org\/index.php\/Information_privacy\" title=\"Information privacy\" class=\"wiki-link\" data-key=\"185f6d9f874e48914b5789317408f782\">privacy<\/a> to comply with relevant regulations.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Methods\">Methods<\/span><\/h2>\n<h3><span class=\"mw-headline\" id=\"Overview\">Overview<\/span><\/h3>\n<p>The First Affiliated Hospital, Zhejiang University School of Medicine (FAHZU) has six campuses with approximately 5,000 beds. In 2020, the institution recorded over 4.2 million outpatient and emergency visits, and 236,100 discharges. As one of the pioneering and earliest-founded National Drug CTIs in China, the first batch of clinical pharmacology bases under the Ministry of Health was established in 1998. As a first-class integrated service platform for clinical research in China, it operates 24 specialized groups and depends on an internationally recognized and independent ethics examination system, and it has undertaken more than 2,000 instances of foreign and domestic clinical trials since its establishment, including both Phase I\u2013IV drugs and <a href=\"https:\/\/www.limswiki.org\/index.php\/Medical_device\" title=\"Medical device\" class=\"wiki-link\" data-key=\"8e821122daa731f0fa8782fae57831fa\">medical devices<\/a>.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"System_development_process\">System development process<\/span><\/h3>\n<p>The development of an integrated CTMS began in March 2014, and after 18 months, the first version of the CTMS was launched in FAHZU in September 2015. The complete system development process was carried out in four stages, including two months of requirement analysis and problem collection, four months of system design, six months of system development and testing, and six months of system operation training and pilot runs. Based on the feedback from the use of the first version of CTMS and the continuous in-depth exploration of the business, the system\u2019s functionality and user experience continue to be iteratively updated. By March 2021, the updated version 4.2.5 was launched. Through the construction of an integrated CTMS, the entire process of clinical trial data management is realized, which provides the basic conditions for the efficient, high-quality, and standardized operation of clinical trials in the hospital.\n<\/p>\n<h4><span class=\"mw-headline\" id=\"First_stage\">First stage<\/span><\/h4>\n<p>The collection and analysis of CTMS requirements defined the existing problem set to be solved. This stage is the most critical and determines the final goal direction of the system. The collection of requirements started in March 2014, and the participants included the CTI office director, CTI office secretary, ethics committee (EC) members, EC secretary, <a href=\"https:\/\/www.limswiki.org\/index.php\/Principal_investigator\" title=\"Principal investigator\" class=\"wiki-link\" data-key=\"fc46a2d8fd6731c64dab335a424a06dc\">principal investigator<\/a> (PI), sub-investigator, study nurse, clinical research associate (CRA), clinical research coordinator (CRC), investigational product custodians, financial officer, quality control expert, statisticians, data manager, pharmacovigilance associate, and information technology expert. After two months of formal and informal interviews, as well as an analysis of the collection requirements, the design goal of CTMS was determined. The overall goal is for the CTMS to realize the entire process of clinical trial management, including study project approval, ethical review, subject recruitment, subject management, investigational product management, financial management, and quality management. Table 1 lists the functional requirements for realizing the entire process management of clinical trials in hospitals based on seven dimensions. These requirements were used to construct the integrated CTMS.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"3\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 1.<\/b> List of functional requirements for realizing the entire process management workflow of clinical trials in hospitals\n<\/td><\/tr>\n\n<tr>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Number\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Requirement class\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Requirement description\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Study Project Approval Management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">The application and review of clinical trials should adopt process automation management, including process customization configuration, remote submission of project application, uploading of project materials, automatic generation of to-do tasks, timely message transmission, material annotation, and other supporting functions.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Ethical Review Management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Define the ethics committee review process and application contents (e.g., new protocols, protocol amendments, etc.).\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Subject Management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Define the subject management model to ensure that the subjects complete the visit content of each cycle in strict accordance with the research plan.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">4\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Investigational Product Management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">The investigational product should adopt the central pharmacy model to achieve closed-loop management, including receiving, warehousing, distributing, recycling, returning, disposal, and early warning.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">5\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Financial Management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Clinical trial finance requires independent accounting and management; the subjects\u2019 diagnosis and treatment processes can be exempted from payment, and the system should automatically record costs to achieve direct settlement between the hospital and the sponsor.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">6\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Quality Management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Define the elements and content of quality management, and the quality control of related data that must be collected during the operation of the system.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">7\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Privilege Management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Define permissions and data access rules for different roles through multi-role collaboration.\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h4><span class=\"mw-headline\" id=\"Second_stage\">Second stage<\/span><\/h4>\n<p>In the second stage, the CTMS design process included the definition of the system architecture and data interchange mechanisms, selection of storage media, database modeling and standardization, user interface (UI) and user experience (UX) design, and development of security and privacy policies based on analyzing and categorizing the requirements collected in the first phase, referring to good clinical practice (GCP) and the guidance documents of the National Medical Products Administration (NMPA). The functions and data required by each participant in the clinical trial are determined. Figure 1 shows the system architecture of the CTMS, which is divided into seven subsystems to meet the corresponding requirements of the first stage. Seven dimensions were put forward, including the clinical trial project management system (CTPMS), clinical trial ethical management system (CTEMS), clinical trial subject management system (CTSMS), clinical trial investigational product management system (CTIPMS), clinical trial financial management system (CTFMS), clinical trial quality management system (CTQMS), and permission management and maintenance system (PMMS). These systems include the functions of the whole-process management of clinical trials. The unified control of permissions is realized through single sign-on (SSO) between the systems. To ensure a more efficient operation of the CTMS, it is necessary to focus on the data integration mode with the hospital clinical and business systems (i.e., HIS, LIS, EMR, CDR, etc.). \n<\/p><p>The unified management of interface services is defined, as shown in Fig. 1, as a process of complete data integration, which primarily involves three types of data service functions: (1) basic data synchronization, which serves as the basis for system operation; (2) real-time data query to meet the needs of subject information retrieval and clinical trial data monitoring; and (3) diagnosis and treatment data generated by the CTMS, which are transmitted to the clinical business system of the hospital to meet the continuity requirement of diagnosis and treatment of the subjects. Database modeling and standardization are performed with reference to the Clinical Data Interchange Standards Consortium (CDISC) to obtain a standard vocabulary to ensure the integrity of the data model; in terms of system security and subject privacy protection, security protection strategies and development specifications were formulated with reference to the <a href=\"https:\/\/www.limswiki.org\/index.php\/Health_Insurance_Portability_and_Accountability_Act\" title=\"Health Insurance Portability and Accountability Act\" class=\"wiki-link\" data-key=\"b70673a0117c21576016cb7498867153\">Health Insurance Portability and Accountability Act<\/a> (HIPAA) and China\u2019s privacy protection laws. In addition, the CTMS implements fine-grained isolation and verification of permissions, and it records all events in a <a href=\"https:\/\/www.limswiki.org\/index.php\/Audit_trail\" title=\"Audit trail\" class=\"wiki-link\" data-key=\"96a617b543c5b2f26617288ba923c0f0\">log<\/a> to ensure the traceability of the data.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig1_Shen_BMCMedInfoDecMak23_23.png\" class=\"image wiki-link\" data-key=\"1b93c208b3d73d0ee03bc8cbee85dcec\"><img alt=\"Fig1 Shen BMCMedInfoDecMak23 23.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/e\/e2\/Fig1_Shen_BMCMedInfoDecMak23_23.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Fig. 1<\/b> System architecture for the clinical trial management system (CTMS). HIS: hospital information system; EMR: electronic medical record; LIS: laboratory information system; CDR: clinical data repository; PACS: picture archiving and communication systems.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h4><span class=\"mw-headline\" id=\"Third_stage\">Third stage<\/span><\/h4>\n<p>The third stage focused on the development and testing of the CTMS. This was divided into two steps according to the definition of the design stage. The first step realizes clinical trial approval and review management, and the developed subsystems include CTPMS, CTEMS, and PMMS, which focus on the process management of the study project application and review stage, and do not involve data integration with external systems. The second step realizes clinical trial operational management. The developed subsystems include CTSMS, CTIPMS, CTFMS, and CTQMS, which complete internal function development and data integration with external systems, and realize the entire process of data management. The CTMS was developed based on the distributed architecture of the Java programming language, and its database is Oracle Database 11 g Release 2 on a server running the Linux CentOS 7.6 operating system; data integration with external systems is managed through an interface services unified management platform. The system adopts browser\/server architecture and supports current mainstream browsers, such as Google Chrome, Internet Explorer, and Firefox Browser for encrypted access via hypertext transfer protocol secure (HTTPS). System testing mainly includes verification of functional effectiveness, data security, system reliability, and the efficiency of data integration with external systems.\n<\/p>\n<h4><span class=\"mw-headline\" id=\"Fourth_stage\">Fourth stage<\/span><\/h4>\n<p>This stage involved CTMS deployment and system operation training. In November 2014, CTPMS, CTEMS, and PMMS were officially launched, realizing clinical trial approval and review management in FAHZU. In March 2015, the development of CTSMS, CTMMS, CTFMS, and CTQMS was completed. Considering that the clinical trial operational management involves <a href=\"https:\/\/www.limswiki.org\/index.php\/Data_integrity\" title=\"Data integrity\" class=\"wiki-link\" data-key=\"382a9bb77ee3e36bb3b37c79ed813167\">data integrity<\/a> verification and <a href=\"https:\/\/www.limswiki.org\/index.php\/Data_integration\" title=\"Data integration\" class=\"wiki-link\" data-key=\"fd01c635859e1d5b9583e43e31ef6718\">data integration<\/a> with external systems, a new pilot run phase was needed. By selecting a certain number of clinical trials in Phases I\u2013IV, the operating results were used to verify the compatibility of the system with different types of clinical trials. The CTMS pilot run system first selects Phase I clinical trials and runs a total of 20 projects. Through continuous updating of functions during operation, it can fully meet the needs of Phase I clinical trial operational management. Subsequently, it selects clinical trials in Phases II\u2013IV. A total of 10 projects have been run, and the functions have been stable and meet the requirements of various types of clinical trials. However, training hospital staff in the use of CTMS is critical. Through a variety of methods including face-to-face training and offline data learning, all staff related to clinical trials have been trained. The first version of CTMS was fully launched in FAHZU in September 2015.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Results\">Results<\/span><\/h2>\n<p>FAHZU is the first hospital in China to independently develop an integrated CTMS; the system has been successfully implemented in the hospital since September 2015. As of March 2021, the hospital now runs an updated version of CTMS known as V4.2.5. The data management of the entire process of clinical trials from project approval and review management to operational management has been fully realized. The FAHZU CTMS operates independently by design as a fully functional system at the application level (not as a component of the HIS), establishes a dedicated clinical trial database at the data level, and completes data integration with external systems through a unified interface system, to achieve process continuity and data integrity. Through the construction of an integrated CTMS, the flow of multi-party collaboration tasks is optimized, and non-research matters such as finance and data processing are simplified, to effectively improve the efficiency and quality of clinical trials. The results shown below are based on the latest stable version of FAHZU CTMS, V4.2.5.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"System_categories_and_features_for_clinical_trial\">System categories and features for clinical trial<\/span><\/h3>\n<p>The first level involves clinical trial project approval and review management, and its functions cover the rationality review of clinical trial study projects and related affairs management by CTI and EC. Table 2 lists the categories and features of clinical trial approval and review management, and the services provided to users through CTPMS and CTEMS. Clinical trial project approval and review management are based on multi-role collaboration, focusing on improving efficiency as the core and supporting remote project application, full electronic project approval, review of process approval documents online, aggregation of reviews into review comments, and timely generation of tasks and notification. The main features of CTPMS include project approval management, to-do task list, project list, document management, contract management, initial meeting management, investigator management, and CRC management. Through the organic combination of these features, the CTI realizes the project approval review and daily management of study projects. The main features of CTEMS include EC management, ethics review management, and ethics conference management, which manages the continuous ethics review of study projects by the EC, including initial protocol review, protocol amendment review, SAE report and review, and violation\/deviation protocol review.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"3\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 2.<\/b> Categories and features for project approval and review management of clinical trials\n<\/td><\/tr>\n\n<tr>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Category\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Features\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Description\n<\/th><\/tr>\n<tr>\n<td rowspan=\"8\" style=\"background-color:white; padding-left:10px; padding-right:10px;\">Study project management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Project approval management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">CTI defines the application and review procedures for study projects according to the standard operation procedure (SOP).\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">To-do tasks list\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Lists the tasks that the user needs to complete, which are automatically generated by the system according to processes and user roles.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Project list\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Project list contains basic information about the study, controlling different viewing scopes for different roles.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Document management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Electronic management of documents, batch uploading, online review, and suggestion feedback.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Contract management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Manages the content and budget of the contract, and supervises the execution of the contract.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Initial meeting management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Records the meeting contents and participants of the initial meeting.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Investigator management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Manages investigators\u2019 Good Clinical Practice (GCP) education, resume, etc.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Clinical research coordinator (CRC) management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Manages CRC personnel information, recruitment process, and workload reporting and review.\n<\/td><\/tr>\n<tr>\n<td rowspan=\"3\" style=\"background-color:white; padding-left:10px; padding-right:10px;\">Ethics management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Ethics committee (EC)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Manages the organizational structure of the EC.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Ethics review management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Manages the ethics review process, including ethical review application, formal review, study assessment, ethics conference review, approval letter generation, etc.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Ethics conference management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Manages the project review agenda, meeting attendance, voting, meeting minutes, project review results, etc.\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>The second level comprises clinical trial operational management, which is a complex and continuous management process. Table 3 lists the categories and features of clinical trial operational management; the system provides services to users through the CTSMS, CTIPMS, CTFMS, and CTQMS. CTSMS is mainly composed of three stages: operational management pre-configuration, subject recruitment, and subject visit management. The features included in the operational management pre-configuration stage include study participant assignments, protocol configuration, electronic case report form (e-CRF) design, rule configuration, and global control over the access rights and visitation rules of subjects under the study project. The main features of the subject recruitment stage of the system include subject recruitment, subject violation verification (such as determining whether the subject is participating in another clinical trial and inputting incorrect information of subject), subject lists, and subject global labeling, to realize the of subjects under the corresponding study project, status labeling, and visualization. \n<\/p><p>There are three main events in the subject visit management stage, which are described as follows: \n<\/p>\n<ol><li>Obtain the visit content information in the corresponding study cycle according to the study plan and turn it into a to-do list. The visit content includes subject screening, inspection\/examination issuance, prescription issuance, treatment, randomization of subjects, e-CRF filling, and AE\/SAE reporting.<\/li>\n<li>Enter operational status changes of subjects, including switching protocols, admission, or discharge, entering the next visit stage, and status of subject updates (dropping out, withdrawal, failure to follow up, etc.).<\/li>\n<li>Complete data queries of subjects, including outpatient\/inpatient medical records, inspection\/examination results queries, subject fee queries, and e-CRF data filling.<\/li><\/ol>\n<p>CTIPMS adopts the central management model, which is managed by qualified personnel designated by the CTI. Through the organic combination of stock management, prescription management, label management, and intelligent detection, closed-loop management of the entire process of trial investigational products is realized. CTFMS primarily combines the financial characteristics of clinical trials and the relevant requirements of the hospital\u2019s financial management, including payment and allocation, budget management, expenditure management, workload statistics, and project funding amounts to achieve orderly financial management based on greatly reducing researchers\u2019 time consumption. Similarly, CTQMS mainly includes regular quality control data report generation and reporting, as well as dynamic quality control based on collected data.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"3\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 3.<\/b> Categories and features for operational management of clinical trials\n<\/td><\/tr>\n\n<tr>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Category\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Features\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Description\n<\/th><\/tr>\n<tr>\n<td rowspan=\"8\" style=\"background-color:white; padding-left:10px; padding-right:10px;\">Subject management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Study participants assignment\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Assigns study project participants (such as investigators, study nurses, clinical research coordinator [CRC], and clinical research associate [CRA]) and sets corresponding permissions.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Protocol configuration\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">The protocol is mapped to computational executable events; the study plan can be automatically generated according to the visit cycle.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Rule configuration\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Configures rules required for subject management (such as inclusion and exclusion criteria, various number generation rules, and drug randomization methods).\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Running projects list\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Contains responsible or participating running clinical trial projects, with access management.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Subject list\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Contains a list of subjects recruited for this study project, with different colors to distinguish the status of subjects.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Subject recruitment\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Subjects\u2019 information can be linked by searching the hospital\u2019s patient database and alerted if they are enrolled in other clinical trials.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Subject study process management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Manages the entire process of subjects from recruitment to the end of the visit, performs the study content required in the corresponding visit cycle (such as screening, randomization, inspection, and investigational product), and can query all the data required for clinical trials.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Study progress statistics\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Generates statistics based on the distribution of subjects in different dimensions, including recruitment date, visit cycle, and different states (drop out, withdraw, completion, etc.).\n<\/td><\/tr>\n<tr>\n<td rowspan=\"4\" style=\"background-color:white; padding-left:10px; padding-right:10px;\">Investigational product management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Stock management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Manages the inbound, storage, outbound, and refund of investigational product; the inventory quantity can be classified and counted by dictionary, batch, and random code.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Prescription management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Manages the closed-loop process of prescription issuance, verification, distribution, use, and refund of investigational product, and supports different blinding methods (such as open, single-blinding, and double-blinding), which can be traced.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Label management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">All circulation links of investigational product support automatic label scanning and verification (including inbound, outbound, use, etc.).\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Intelligent detection\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Intelligent early warning and verification of the management process of investigational product (such as near expiration date early warning, inventory early warning, and rule-based verification).\n<\/td><\/tr>\n<tr>\n<td rowspan=\"5\" style=\"background-color:white; padding-left:10px; padding-right:10px;\">Financial management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Payment and allocation management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Manages the financial process of payment addition, allocation (such as subject fee, investigator fee, and inspection fee), review, etc.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Budget management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Manages the budget associated with the study project, including budget list, budget summary, and budget adjustment.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Expenditure management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Manages all kinds of expenses and related study project funds.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Workload statistics\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Counts the corresponding workload in different dimensions (such as executive departments, project contracts, billing departments, and cost accounting) to produce financial statements.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Project funding amount\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Manages the summary and detailed list of various expenses of study projects (such as subject fee, investigator fee, and inspection fee).\n<\/td><\/tr>\n<tr>\n<td rowspan=\"3\" style=\"background-color:white; padding-left:10px; padding-right:10px;\">Quality management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Quality control data reporting\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Manages the reporting of various quality control data (such as mid-term study quality control, study monitoring, study audit, and site audit).\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Quality control element capture\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Automatically obtains corresponding quality control elements during clinical trial running (such as deviation protocol, out of visit window, and adverse events).\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Operational data query\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">With authorization, the quality controller can query the running data of the study project in real-time.\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>The third level is backstage management, which is indispensable for the stable operation of the CTMS. Table 4 shows the categories and features for backstage management of clinical trials. In this level, the proposed system provides services to users through the PMMS. Unified management of all user information and data access permissions of the system through user and permission management provides a basis for the realization of the collaboration of users with different roles. Log management identifies the problems in the system operation and records all the data changes for verification. These two points are also an indispensable part of achieving secure data access. Data statistics are recorded according to the requirements of CTI to provide managers with statistical charts of study projects and operational data to provide decision support capability.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"3\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 4.<\/b> Categories and features for backstage management of clinical trials\n<\/td><\/tr>\n\n<tr>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Category\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Features\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Description\n<\/th><\/tr>\n<tr>\n<td rowspan=\"3\" style=\"background-color:white; padding-left:10px; padding-right:10px;\">User and permission management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">User registration\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Non-hospital users apply for registration and approval mode (such as clinical research coordinator [CRC] and clinical research associate [CRA]).\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">User list\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">User list contains all the users in the system, managing login policies and permissions.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Permission management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Realizes the role-based authority management system.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Log management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">System log\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Includes system operation and data change logs, traceable to the source of changes.\n<\/td><\/tr>\n<tr>\n<td rowspan=\"2\" style=\"background-color:white; padding-left:10px; padding-right:10px;\">Data dictionary management\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Basic data maintenance\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Basic data support for the operation of the clinical trial management system (CTMS) can be maintained through a graphical interface.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Basic data synchronization\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Sets up synchronization task; part of the basic data is automatically obtained from other business systems by means of timing synchronization.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Data statistics\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Data statistics\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Statistical analysis capabilities of various data, support tables, and graphs.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">System settings\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">System settings\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">System customization functions to enable the system to operate more intelligently (such as messages and templates).\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h3><span class=\"mw-headline\" id=\"Enterprise_process_management_for_clinical_trials\">Enterprise process management for clinical trials<\/span><\/h3>\n<p>FAHZU CTMS realizes the entire process management of clinical trials through the organic combination of project approval and reviews management, clinical trial operational management, and backstage entire process management, which is further categorized into 59 internal processes by combining user and authority management (Fig. 2). Clinical trial project approval and review management consists of 14 main processes and eight auxiliary processes, which can efficiently complete the review and contract signature phases of the study project; a kick-off meeting is held to allow entry into clinical trial operational management. Clinical trial operational management is a complex and long-term process involving multiple factors such as subjects, diagnosis and treatment, medicine, and finance. We divide it into six stages: operational management pre-configuration, subject recruitment, subject visit management, investigational product management, financial management, and quality management, and then further subdivide these stages into 32 internal processes. Finally, backstage management, as the basic component, supports the stable operation of the entire clinical trial process.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig2_Shen_BMCMedInfoDecMak23_23.png\" class=\"image wiki-link\" data-key=\"ca007dfb86270520350e53ca4663178b\"><img alt=\"Fig2 Shen BMCMedInfoDecMak23 23.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/8\/87\/Fig2_Shen_BMCMedInfoDecMak23_23.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Fig. 2<\/b> Process management system further categorized into 59 internal processes by combining user and authority management. CRA: clinical research associate; CRC: clinical research coordinator; CTI: clinical trial institutes; PI: principal investigator; EC: ethics committee.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h3><span class=\"mw-headline\" id=\"Data_integration_with_external_systems\">Data integration with external systems<\/span><\/h3>\n<p>To use the CTMS as a dedicated database for clinical trials, data integration with external systems is indispensable. Figure 3 shows some of the critical data integration services between the CTMS and external systems. CTMS subsystems integrate with external systems such as HIS, LIS, EMR, and CDR through the interface service platform to achieve data service standardization and transmission process <a href=\"https:\/\/www.limswiki.org\/index.php\/Encryption\" title=\"Encryption\" class=\"wiki-link\" data-key=\"86a503652ed5cc9d8e2b0252a480b5e1\">encryption<\/a>. The data integration method supports extract-transform-load (ETL), message queue (MQ), and remote procedure call (RPC) services and is flexibly selected based on the data volume and business model. Data synchronization and data queries involve a large amount of data. The operations use ETL batch extraction and RPC service real-time queries, which is used for data dictionary maintenance, as well as queries of subjects\u2019 full diagnosis and treatment data, and e-CRF structured data filling. Subject information data and medical order data use the synchronous call of RPC service and asynchronous notification of MQ. Synchronous calling is used to obtain the patient information of the subject recruitment, the subject\u2019s diagnosis and treatment items, and prescription issuance, billing, etc. Asynchronous calling is used for notification of changes in the subject\u2019s status, obtaining medical order status, obtaining results, etc.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig3_Shen_BMCMedInfoDecMak23_23.png\" class=\"image wiki-link\" data-key=\"9194d146d2b4b4afccc3fd121c0a2b28\"><img alt=\"Fig3 Shen BMCMedInfoDecMak23 23.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/9\/92\/Fig3_Shen_BMCMedInfoDecMak23_23.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Fig. 3<\/b> Critical services for data integration between the clinical trial management system (CTMS) and external systems. HIS: hospital information system; EMR: electronic medical record; LIS: laboratory information system; CDR: clinical data repository; ETL: extract-transform-load; MQ: message queue; RPC: remote procedure call; CTMS: clinical trial management system; PMMS: permission management and maintenance system; CTSMS: clinical trial subject management system; CTIPMS: clinical trial investigational product management system; CTFMS: clinical trial financial management system; CTSMS: clinical trial subject management system; CTQMS: clinical trial quality management system.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h3><span class=\"mw-headline\" id=\"Data_security_and_privacy_protection\">Data security and privacy protection<\/span><\/h3>\n<p>Data security and privacy protection ensure that a mature process is developed and maintained throughout the design life cycle. Data security primarily involves data storage and data access security. Privacy protection focuses on ensuring data security while complying with laws and regulations to de-identify sensitive patient information.\n<\/p><p>Data security mainly includes the two aspects of data storage security and data access security. In terms of data storage security, FAHZU CTMS adopts two strategies: (1) multi-node storage of data to achieve high availability of data services and a data <a href=\"https:\/\/www.limswiki.org\/index.php\/Backup\" title=\"Backup\" class=\"wiki-link\" data-key=\"e12548e6bf5f28bfee99099fe8662dde\">backup<\/a> strategy to ensure that data will not be lost in case of failure, and (2) encrypted storage of sensitive data (e.g., passwords, ID numbers, and bank card numbers). Policies for data access security are:\n<\/p>\n<ol><li>Security at the network level, through safety equipment (such as a firewall, host security protection, bastion host, and database\/log audit tools), to implement access control and prevent illegal access;<\/li>\n<li>Security at the application level through multiple roles of authority management system access controls on data content, especially including rules to ensure no open access to prescriptions (e.g., CRC non-prescription issuance authority), as well as finding and repairing the vulnerabilities of the application through penetration testing and completing the information security technology-evaluation requirement for classified protection of <a href=\"https:\/\/www.limswiki.org\/index.php\/Cybersecurity\" class=\"mw-redirect wiki-link\" title=\"Cybersecurity\" data-key=\"ba653dc2a1384e5f9f6ac9dc1a740109\">cybersecurity<\/a>\u2014Level 3; and<\/li>\n<li>Maintaining a detailed data change log can be useful in tracing the record of the process of data recovery.<\/li><\/ol>\n<p>Subject privacy protection aims to protect sensitive patient information through a series of de-identification processes, which are explicitly required in the GCP. The FAHZU CTMS strictly implements the requirements of subject privacy protection in the GCP guidelines and refers to the Information Security Technology Personal Information Security Specification issued by the Standardization Administration of China and the HIPAA Regulations issued by the U.S. Department of Health. Final compliance with HIPAA\u2019s implementation specifications requiring de-identification of protected health information excludes 18 personal health identifiers (PHIs) from the CTMS (e.g., name, address, cell phone number, and social security number) to comply with international and national laws. For CTMS users to identify subjects, we used the clinical trial protocol number, subject screening number, and subject name initials to uniquely identify them. At the level of data interaction, the patients\u2019 medical record numbers in the hospital are systematically encrypted and stored in the CTMS database, which is linked to the data of the external clinical and business systems (i.e., HIS, LIS, EMR, CDR, etc.) for automatic data transmission.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Evaluation\">Evaluation<\/span><\/h3>\n<p>The evaluation of the CTMS is ongoing and mainly conducted at two levels: clinical trial project approval and review management, and clinical trial operational management. The former focuses on the convenience of multi-party collaboration and the efficiency of project approval, whereas the latter focuses on the evaluation of subject-centered data management throughout the process and the improvement of the quality of clinical trials.\n<\/p><p>The improved clinical trial project approval and review management after applying the FAHZU CTMS, compared with that before the system was implemented, mainly embodies two distinct advantages. First, it realizes process automation management based on task and, combined with workflow technology, cooperates with user authority management systems, realizes automatic triggering of node events, and performs automatic assignment of processing personnel and message notifications such that personnel only need to process their personal to-do lists according to the message reminder, reducing the complexity of system use under multi-party collaboration. Second, clinical trial project approval supports remote application, and electronic document management enables online submission of project approval materials, as well as online review and summary of revision opinions in the review process, which reduces the workload of project approval materials review and improves the efficiency of project review. Since CTMS began to operate in the hospital in December 2014, 1,388 study projects have been accepted and 43,051 documents have been submitted through CTMS as of March 31, 2021.\n<\/p><p>The focus at the clinical trial operational management level is centered on patient outcomes to achieve full-cycle data management. Here, the advantages are more significant. The four core advantages are summarized as follows: \n<\/p>\n<ol><li>The CTMS requires that the research plan must be entered before the recruitment of subjects, and the visit content of the current visit cycle can be automatically correlated during the subject visit. It is transformed into to-do tasks in the current research stage, with timely reminders, reducing deviations from the protocol.<\/li>\n<li>An independent billing model is adopted for clinical trial-related inspection and treatment expenses so that subjects can be exempted from expense reimbursement and be marked in the hospital business system to meet non-clinical trial diagnosis and treatment reminders and clinical trial inspection green special requirements such as channels.<\/li>\n<li>The CTMS contains the data on each subject, including newly generated data during operation and data collected in the hospital business system. Through strict authority classification, the scope of the data queries, such as those issued by the CRC and other data query authorities, are limited to subjects who are responsible for the project.<\/li>\n<li>Clinical trial drugs are label-based full-cycle closed-loop managed, and transfers are completed through label scanning, and support some special properties of clinical trial drugs, such as open, single-blind, and double-blind studies, and situations with other prescriptions, where drugs need to be random, and when the serial numbers and medicine need to be recalled.<\/li><\/ol>\n<p>The clinical trial operational management of CTMS began a pilot run at FAHZU in March 2015. Pilot runs are conducted to validate and improve the effectiveness of system functions and compatibility with different types of clinical trials. Since September 2015, all newly initiated clinical trials on the site have been managed through CTMS. According to statistics, as of March 31, 2021, a total of 12,144 subjects have been included in the management of CTMS, across 472 study projects. Figure 4 shows the change in the number of subjects and corresponding study projects enrolled in CTMS in the last two years. We observed that the monthly number of new subjects and the corresponding study projects remained stable, and the number of active subjects also remained stable, while the corresponding monthly number of active projects increased. Since the outbreak of <a href=\"https:\/\/www.limswiki.org\/index.php\/COVID-19\" class=\"mw-redirect wiki-link\" title=\"COVID-19\" data-key=\"da9bd20c492b2a17074ad66c2fe25652\">COVID-19<\/a> in China in December 2019, with the support of the integrated CTMS and necessary measures (e.g., remote follow-up, express delivery, and remote inspection), it may be observed from Fig. 4 that the clinical trials have continued at a steady rate during the COVID-19 outbreak.\n<\/p><p><br \/>\n<\/p><p><a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig4_Shen_BMCMedInfoDecMak23_23.png\" class=\"image wiki-link\" data-key=\"6e9f581a48c6b8766432eed2afbe2b4f\"><img alt=\"Fig4 Shen BMCMedInfoDecMak23 23.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/c\/cf\/Fig4_Shen_BMCMedInfoDecMak23_23.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Fig. 4<\/b> Change in the number of subjects and corresponding study projects enrolled in the clinical trial management system (CTMS) over the last two years.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h2><span class=\"mw-headline\" id=\"Discussion\">Discussion<\/span><\/h2>\n<p>Digitalization is an inevitable trend in clinical trial management.<sup id=\"rdp-ebb-cite_ref-9\" class=\"reference\"><a href=\"#cite_note-9\">[9]<\/a><\/sup> At present, there are many electronic clinical trial management systems in China; however, there are few systems that can connect with the information system of the clinical research center and unify the sponsor, data management department, and other parties into a platform for cooperation. The FAHZU CTMS has been researched and developed fully independently based on a distributed service architecture. It takes process management and trial data as the core; highly integrates, interconnects, and interfuses with hospital clinical business systems; and combines key contents such as data security and privacy protection to achieve independent application layers and interconnected data layers. Thus, comprehensive process management and dynamic real-time monitoring of clinical trials can be realized. Compared with commercial products, our self-developed CTMS is more cost-effective and highly customizable, fully fits the management needs of the hospital as a clinical trial administrative institution, and allows more timely system version updates, system operation, and maintenance response. Based on the deep understanding of hospital clinical business systems such as HIS, LIS, CDR, PACS, and EMR by information technology experts, a more comprehensive scheme was designed to achieve a high degree of integration between the CTMS and clinical business systems. The large amounts of medical data required for clinical trials are docked into the developed system and used as a CTMS-independent data collection and storage subsystem. In addition, based on the full understanding of data security and privacy protection, the security of the system was assigned great importance from the beginning of system development, and the privacy protection function of subjects was improved to better serve the entire process management of clinical trials in our institution. \n<\/p><p>From an operational point of view, the use of the system has significantly improved the efficiency of researchers and institutional managers. For the institutional management, the system realizes real-time and efficient management of the entire process of clinical trials and ensures the reliability, authenticity, and integrity of clinical trial results. To ensure the safety of subjects, it realizes limited sharing of information through the data query, statistics, and tracking module. Institutional managers can view the project schedule and browse project-related node information, to accurately grasp the implementation of the project status, improve the quality of clinical trial data, reduce time consumption, and promote the standardized implementation of clinical trials according to the GCP guidelines and SOP of our hospital. On this basis, a full-cycle data quality monitoring and early warning platform for clinical trials can be constructed. Through <a href=\"https:\/\/www.limswiki.org\/index.php\/Artificial_intelligence\" title=\"Artificial intelligence\" class=\"wiki-link\" data-key=\"0c45a597361ca47e1cd8112af676276e\">artificial intelligence<\/a> (AI) technologies such as text mining, natural language processing, <a href=\"https:\/\/www.limswiki.org\/index.php\/Machine_learning\" title=\"Machine learning\" class=\"wiki-link\" data-key=\"79aab39cfa124c958cd1dbcab3dde122\">machine learning<\/a>, and knowledge mapping, intelligent full-cycle <a href=\"https:\/\/www.limswiki.org\/index.php\/Data_analysis\" title=\"Data analysis\" class=\"wiki-link\" data-key=\"545c95e40ca67c9e63cd0a16042a5bd1\">data analysis<\/a> and early warning in clinical trials can be realized. These include the matching of inclusion and exclusion criteria, warning of inspection, warning of combined drug prohibition, warning of absence of visit activity or out of window, and intelligent warning of underreporting of AE or SAE.\n<\/p><p>The protection of private information is a more important consideration than the system construction.<sup id=\"rdp-ebb-cite_ref-10\" class=\"reference\"><a href=\"#cite_note-10\">[10]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-11\" class=\"reference\"><a href=\"#cite_note-11\">[11]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:0_12-0\" class=\"reference\"><a href=\"#cite_note-:0-12\">[12]<\/a><\/sup> The biggest problem with the establishment of the system is that the subject information may be exposed, which is a serious ethical problem. To solve potential risks from the level of laws and regulations, it is necessary to think deeply. Therefore, at the beginning of the design, we paid special attention to the concept of network data security and privacy protection, carried out the privacy impact assessment, and integrated the measures of privacy protection into the entire process of information system development. Institutional managers, researchers, quality controllers, CRCs, drug administrators, and inspectors are divided into different users. There is a strict permission management system for user accounts, to avoid the problem of account borrowing, certificate authority authentication or face recognition systems should be added to adopt fine-grained permission management. In addition, the SOP for risk assessment is particularly important, including the definition, classification, and rating of risks.<sup id=\"rdp-ebb-cite_ref-13\" class=\"reference\"><a href=\"#cite_note-13\">[13]<\/a><\/sup> Only information management systems that meet the requirements of risk assessment can be operated. In the information age, everything is connected, and information is a trend.<sup id=\"rdp-ebb-cite_ref-14\" class=\"reference\"><a href=\"#cite_note-14\">[14]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-15\" class=\"reference\"><a href=\"#cite_note-15\">[15]<\/a><\/sup> In today's highly developed internet technology, we need to think about and solve the problems of data security and personal privacy, as well as the specific procedures to improve the efficiency of clinical research. At the same time, compliance with the Chinese GCP and International Conference on Harmonisation (ICH) requirements for GCP ensure compliance with ethical and legal provisions.\n<\/p><p>Compared with the traditional paper CRF, e-CRF allows researchers to enter the CRF electronically based on the source data rather than fill it in manually.<sup id=\"rdp-ebb-cite_ref-:0_12-1\" class=\"reference\"><a href=\"#cite_note-:0-12\">[12]<\/a><\/sup> However, source data verification (SDV) is still required to compare the CRF with original medical records and inspection lists to ensure the quality of data input. Our CTMS is now able to automate the collection of structured data, eliminate the SDV process, and further simplify the process. Thus, any modifications to the electronic source data can be recorded through an <a href=\"https:\/\/www.limswiki.org\/index.php\/Audit_trail\" title=\"Audit trail\" class=\"wiki-link\" data-key=\"96a617b543c5b2f26617288ba923c0f0\">audit trail<\/a>. In the future, our system will also study the complex natural language analysis involved in unstructured data. Simultaneously, we will build a way to connect <a href=\"https:\/\/www.limswiki.org\/index.php\/Electronic_data_capture\" title=\"Electronic data capture\" class=\"wiki-link\" data-key=\"121ee10293d97335609c50b94f7b5394\">electronic data acquisition<\/a> (EDC) and hospital e-CRF to directly collect and transfer clinical data electronically, ensuring the quality and integrity of the data.\n<\/p><p>In the future, we also want to utilize the data in the clinical trial management platform to realize intelligent subject recruitment and improve the efficiency of subject recruitment. Recruiting subjects for some clinical trials is difficult, especially those involving rare diseases, stringent admission criteria, and special subgroups. Owing to information asymmetry, researchers only know the condition of the patients they are treating, and patients do not know that their disease may be under research at the hospital, which can seriously affect the progress of clinical trials.<sup id=\"rdp-ebb-cite_ref-16\" class=\"reference\"><a href=\"#cite_note-16\">[16]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-17\" class=\"reference\"><a href=\"#cite_note-17\">[17]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-18\" class=\"reference\"><a href=\"#cite_note-18\">[18]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-19\" class=\"reference\"><a href=\"#cite_note-19\">[19]<\/a><\/sup> To fully utilize the hospital CTMS system of clinical data, the central hospital and administering medical hospital can simultaneously be on the same clinical trial management platform, which will further enrich the patient resources. Additionally, the system can provide search functions, select exclusion criteria, fast-matching potential subjects, and realize intelligent recruitment of subjects, which can effectively help the sponsor accelerate the clinical trial process, reduce R&D costs, and successfully seize market opportunities. However, in the era of big data, these data have important scientific value in the field of clinical research. Applications such as large data analysis found that local residents\u2019 disease condition, and its influencing factors\u2014specific studies on key diseases\u2014have been successfully applied to determine the time of disease distribution, location distribution, population distribution, and analysis of risk factors of disease; furthermore, it has been used for the evaluation of the effectiveness of clinical screening and diagnosis methods, inspection treatment or drug treatment effect, optimization of individual diagnosis and treatment of disease, and research on the influencing factors of diseases after intervention. This is to provide the basis for the local health administration departments to make health management decisions.\n<\/p><p>Since the outbreak of COVID-19, many jobs around the world have been stalled to a certain extent. In the event of a major public health emergency, carrying out clinical trials and ensuring the smooth implementation of monitoring work has become a problem for clinical trial practitioners, and remote monitoring has therefore been put on the agenda. The U.S. <a href=\"https:\/\/www.limswiki.org\/index.php\/Food_and_Drug_Administration\" title=\"Food and Drug Administration\" class=\"wiki-link\" data-key=\"e2be8927071ac419c0929f7aa1ede7fe\">Food and Drug Administration<\/a> (FDA) has long encouraged more centralized monitoring, where inspectors perform inspections in the office using relevant information tools rather than at research institutions (hospitals). The development trend of clinical trial monitoring is to replace on-site monitoring with centralized monitoring.<sup id=\"rdp-ebb-cite_ref-20\" class=\"reference\"><a href=\"#cite_note-20\">[20]<\/a><\/sup> Remote monitoring can improve the quality and efficiency of clinical research and reduce its cost, and it is possible only when a series of electronic clinical trial products such as CTMS, electronic data acquisition, EMR, and clinical data management systems are widely used. The clinical trial information management system of our hospital is a CTMS that unites multiple teams on one platform and is highly integrated with all clinical systems of the hospital. As a web-based platform, remote data monitoring and cloud auditing can be included on the CTMS as a mature operation.\n<\/p><p>Compared with other information systems, CTMSs are more professional and personalized. The realization of the effectiveness of a CTMS requires a significant amount of time, and the more clinical trials undertaken, the more significant the effect. Compared with the system design, the comprehensive and efficient application of the system takes longer to achieve. With the digitization of clinical research, the sharing and integration of research data will bring many management advantages, such as an increase in available management resources, scientific and data support for major decisions, the convenience of remote management, pertinacity, and pre-operation. The CTMS developed by our hospital will be constantly updated and upgraded, and its functions will constantly improve. The system update will keep pace with developments in international drug clinical trial management, promote the development of clinical trials in a more standardized direction, and promote the disciplinary status of the hospital regarding high-level clinical trials.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Conclusion\">Conclusion<\/span><\/h2>\n<p>The FAHZU CTMS, as the first integrated CTMS independently developed by a hospital in China, can better adapt to the institutional needs for individualized, whole-process, and dynamically comprehensive evaluation and supervision of clinical trials. The integrated CTMS contains three levels and seven subsystems, which fully realizes the whole-process data management of clinical trials from project approval and review management to operational management. Through the unified interface system, the developed CTMS provides a variety of access methods to complete efficient data integration with the clinical business systems, and applies multiple security policies combined with privacy protection methods to effectively ensure the security of data and the privacy of subjects during clinical trial operation. The operation results based on the integrated CTMS show that it can effectively control the risks in the clinical trial process, so as to improve the science, safety, and timeliness of the new drug development process.\n<\/p>\n<h2><span id=\"rdp-ebb-Abbreviations,_acronyms,_and_initialisms\"><\/span><span class=\"mw-headline\" id=\"Abbreviations.2C_acronyms.2C_and_initialisms\">Abbreviations, acronyms, and initialisms<\/span><\/h2>\n<ul><li><b>AE<\/b>: adverse event<\/li>\n<li><b>AI<\/b>: artificial intelligence<\/li>\n<li><b>CDISC<\/b>: Clinical Data Interchange Standards Consortium<\/li>\n<li><b>CDR<\/b>: clinical data repository<\/li>\n<li><b>CRA<\/b>: clinical research associate<\/li>\n<li><b>CRC<\/b>: clinical research coordinator<\/li>\n<li><b>CTEMS<\/b>: clinical trial ethical management system<\/li>\n<li><b>CTFMS<\/b>: clinical trial financial management system<\/li>\n<li><b>CTI<\/b>: clinical trial institution<\/li>\n<li><b>CTIPMS<\/b>: clinical trial investigational product management system<\/li>\n<li><b>CTMS<\/b>: clinical trial management system<\/li>\n<li><b>CTPMS<\/b>: clinical trial project management system<\/li>\n<li><b>CTQMS<\/b>: clinical trial quality management system<\/li>\n<li><b>CTSMS<\/b>: clinical trial subject management system<\/li>\n<li><b>e-CRF<\/b>: electronic case report form<\/li>\n<li><b>EC<\/b>: ethics committee<\/li>\n<li><b>EMR<\/b>: electronic medical record<\/li>\n<li><b>ETL<\/b>: extract-transform-load<\/li>\n<li><b>FAHZU<\/b>: First Affiliated Hospital, Zhejiang University School of Medicine<\/li>\n<li><b>CP<\/b>: good clinical practice<\/li>\n<li><b>HIPAA<\/b>: Health Insurance Portability and Accountability Act<\/li>\n<li><b>HIS<\/b>: hospital information system<\/li>\n<li><b>HTTPS<\/b>: hypertext transfer protocol secure<\/li>\n<li><b>ICH<\/b>: International Conference on Harmonisation<\/li>\n<li><b>LAN<\/b>: local area network<\/li>\n<li><b>LIS<\/b>: laboratory information system<\/li>\n<li><b>PACS<\/b>: picture archiving and communication systems<\/li>\n<li><b>PHI<\/b>: personal health identifier<\/li>\n<li><b>PI<\/b>: principal investigator<\/li>\n<li><b>PMMS<\/b>: permission management and maintenance system<\/li>\n<li><b>R&D<\/b>: research and development<\/li>\n<li><b>RPC<\/b>: remote procedure call<\/li>\n<li><b>SAE<\/b>: serious adverse events<\/li>\n<li><b>SDV<\/b>: source data verification<\/li>\n<li><b>SSO<\/b>: single sign-on<\/li>\n<li><b>UI<\/b>: user interface<\/li>\n<li><b>UX<\/b>: user experience<\/li><\/ul>\n<h2><span class=\"mw-headline\" id=\"Acknowledgements\">Acknowledgements<\/span><\/h2>\n<h3><span class=\"mw-headline\" id=\"Author_contributions\">Author contributions<\/span><\/h3>\n<p>LS conceived the concept of the project, was responsible for the architecture and integration concept, designed and developed the system, and wrote the manuscript. YZ collected and sorted out the requirements for the construction of the clinical trial management system and collected references. AXP and QWZ designed and organized the tables and figures in the article. MZ and JL were responsible for conceptualization and formal analysis, reviewed the manuscript, and supplemented the discussion section. All authors read and approved the final manuscript.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Ethics_approval_and_consent_to_participate\">Ethics approval and consent to participate<\/span><\/h3>\n<p>There were no individual-level data for the study, so ethics committee approval was not required. All executive trials included in this study system were the Clinical Trial Ethics Committee (EC) of The First Affiliated Hospital, Zhejiang University School of Medicine (FAHZU) reviewed and approved. All subjects in these trials signed informed consents prior to enrollment.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Funding\">Funding<\/span><\/h3>\n<p>This study was funded by the New Drug Creation Project of The 13th Five-Year National Science and Technology Major Special Project (2020ZX09201-003).\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Availability_of_data_and_materials\">Availability of data and materials<\/span><\/h3>\n<p>FAHZU CTMS consists of several subsystems. Considering data security and privacy protection, subsystems associated with subject data are deployed based on the hospital\u2019s internal network, and domain names are resolved by a self-built domain name systemserver. The following four links show part of the web pages of the core subsystem, which can show the function design and data volume of the system. For additional information please contact the corresponding author.\n<\/p><p>1. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/doi.org\/10.5281\/zenodo.5880663\" target=\"_blank\">https:\/\/doi.org\/10.5281\/zenodo.5880663<\/a>\n2. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/doi.org\/10.5281\/zenodo.5880783\" target=\"_blank\">https:\/\/doi.org\/10.5281\/zenodo.5880783<\/a>\n3. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/doi.org\/10.5281\/zenodo.5880799\" target=\"_blank\">https:\/\/doi.org\/10.5281\/zenodo.5880799<\/a>\n4. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/doi.org\/10.5281\/zenodo.5880826\" target=\"_blank\">https:\/\/doi.org\/10.5281\/zenodo.5880826<\/a>\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Software_availability_and_requirements\">Software availability and requirements<\/span><\/h3>\n<p>Project name: FAHZU CTMS\nProject home page: <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ctms.zy91.com\" target=\"_blank\">https:\/\/ctms.zy91.com<\/a>\nOperating system(s): Platform-independent\nProgramming language: Java\nOther requirements: Java 1.7.1 or higher, Tomcat 7.0 or higher, Nginx 1.14.2 or higher, Oracle Database 11 g Release 2, Redis 4.0\nLicense: Free for academics\nAny restrictions to use by non-academics: Contact authors\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Competing_interests\">Competing interests<\/span><\/h3>\n<p>The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"References\">References<\/span><\/h2>\n<div class=\"reflist references-column-width\" style=\"-moz-column-width: 30em; -webkit-column-width: 30em; column-width: 30em; list-style-type: decimal;\">\n<div class=\"mw-references-wrap mw-references-columns\"><ol class=\"references\">\n<li id=\"cite_note-1\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-1\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Kruizinga, M. D.; Stuurman, F. E.; Exadaktylos, V.; Doll, R. J.; Stephenson, D. T.; Groeneveld, G. J.; Driessen, G. J. A.; Cohen, A. F. (1 October 2020). Michel, Martin. ed. <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/pharmrev.aspetjournals.org\/lookup\/doi\/10.1124\/pr.120.000028\" target=\"_blank\">\"Development of Novel, Value-Based, Digital Endpoints for Clinical Trials: A Structured Approach Toward Fit-for-Purpose Validation\"<\/a> (in en). <i>Pharmacological Reviews<\/i> <b>72<\/b> (4): 899\u2013909. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1124%2Fpr.120.000028\" target=\"_blank\">10.1124\/pr.120.000028<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0031-6997\" target=\"_blank\">0031-6997<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/pharmrev.aspetjournals.org\/lookup\/doi\/10.1124\/pr.120.000028\" target=\"_blank\">http:\/\/pharmrev.aspetjournals.org\/lookup\/doi\/10.1124\/pr.120.000028<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Development+of+Novel%2C+Value-Based%2C+Digital+Endpoints+for+Clinical+Trials%3A+A+Structured+Approach+Toward+Fit-for-Purpose+Validation&rft.jtitle=Pharmacological+Reviews&rft.aulast=Kruizinga&rft.aufirst=M.+D.&rft.au=Kruizinga%2C%26%2332%3BM.+D.&rft.au=Stuurman%2C%26%2332%3BF.+E.&rft.au=Exadaktylos%2C%26%2332%3BV.&rft.au=Doll%2C%26%2332%3BR.+J.&rft.au=Stephenson%2C%26%2332%3BD.+T.&rft.au=Groeneveld%2C%26%2332%3BG.+J.&rft.au=Driessen%2C%26%2332%3BG.+J.+A.&rft.au=Cohen%2C%26%2332%3BA.+F.&rft.date=1+October+2020&rft.volume=72&rft.issue=4&rft.pages=899%E2%80%93909&rft_id=info:doi\/10.1124%2Fpr.120.000028&rft.issn=0031-6997&rft_id=http%3A%2F%2Fpharmrev.aspetjournals.org%2Flookup%2Fdoi%2F10.1124%2Fpr.120.000028&rfr_id=info:sid\/en.wikipedia.org:Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-2\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-2\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Davi, Ruthie; Mahendraratnam, Nirosha; Chatterjee, Arnaub; Dawson, C. Jill; Sherman, Rachel (1 December 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/d41573-020-00146-5\" target=\"_blank\">\"Informing single-arm clinical trials with external controls\"<\/a> (in en). <i>Nature Reviews Drug Discovery<\/i> <b>19<\/b> (12): 821\u2013822. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fd41573-020-00146-5\" target=\"_blank\">10.1038\/d41573-020-00146-5<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1474-1776\" target=\"_blank\">1474-1776<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/d41573-020-00146-5\" target=\"_blank\">https:\/\/www.nature.com\/articles\/d41573-020-00146-5<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Informing+single-arm+clinical+trials+with+external+controls&rft.jtitle=Nature+Reviews+Drug+Discovery&rft.aulast=Davi&rft.aufirst=Ruthie&rft.au=Davi%2C%26%2332%3BRuthie&rft.au=Mahendraratnam%2C%26%2332%3BNirosha&rft.au=Chatterjee%2C%26%2332%3BArnaub&rft.au=Dawson%2C%26%2332%3BC.+Jill&rft.au=Sherman%2C%26%2332%3BRachel&rft.date=1+December+2020&rft.volume=19&rft.issue=12&rft.pages=821%E2%80%93822&rft_id=info:doi\/10.1038%2Fd41573-020-00146-5&rft.issn=1474-1776&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fd41573-020-00146-5&rfr_id=info:sid\/en.wikipedia.org:Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-3\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-3\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">the COVID19 and Cancer Clinical Trials Working Group; Desai, Aakash; Gainor, Justin F.; Hegde, Aparna; Schram, Alison M.; Curigliano, Giuseppe; Pal, Sumanta; Liu, Stephen V. <i>et al.<\/i> (1 May 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41571-021-00503-2\" target=\"_blank\">\"Author Correction: COVID-19 vaccine guidance for patients with cancer participating in oncology clinical trials\"<\/a> (in en). <i>Nature Reviews Clinical Oncology<\/i> <b>18<\/b> (5): 320\u2013320. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41571-021-00503-2\" target=\"_blank\">10.1038\/s41571-021-00503-2<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1759-4774\" target=\"_blank\">1759-4774<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC7985918\/\" target=\"_blank\">PMC7985918<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/33758378\" target=\"_blank\">33758378<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41571-021-00503-2\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41571-021-00503-2<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Author+Correction%3A+COVID-19+vaccine+guidance+for+patients+with+cancer+participating+in+oncology+clinical+trials&rft.jtitle=Nature+Reviews+Clinical+Oncology&rft.aulast=the+COVID19+and+Cancer+Clinical+Trials+Working+Group&rft.au=the+COVID19+and+Cancer+Clinical+Trials+Working+Group&rft.au=Desai%2C%26%2332%3BAakash&rft.au=Gainor%2C%26%2332%3BJustin+F.&rft.au=Hegde%2C%26%2332%3BAparna&rft.au=Schram%2C%26%2332%3BAlison+M.&rft.au=Curigliano%2C%26%2332%3BGiuseppe&rft.au=Pal%2C%26%2332%3BSumanta&rft.au=Liu%2C%26%2332%3BStephen+V.&rft.au=Halmos%2C%26%2332%3BBalazs&rft.date=1+May+2021&rft.volume=18&rft.issue=5&rft.pages=320%E2%80%93320&rft_id=info:doi\/10.1038%2Fs41571-021-00503-2&rft.issn=1759-4774&rft_id=info:pmc\/PMC7985918&rft_id=info:pmid\/33758378&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41571-021-00503-2&rfr_id=info:sid\/en.wikipedia.org:Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-4\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-4\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Manem, Venkata S.K.; Salgado, Roberto; Aftimos, Philippe; Sotiriou, Christos; Haibe-Kains, Benjamin (1 October 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1044579X17302365\" target=\"_blank\">\"Network science in clinical trials: A patient-centered approach\"<\/a> (in en). <i>Seminars in Cancer Biology<\/i> <b>52<\/b>: 135\u2013150. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.semcancer.2017.12.006\" target=\"_blank\">10.1016\/j.semcancer.2017.12.006<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1044579X17302365\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1044579X17302365<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Network+science+in+clinical+trials%3A+A+patient-centered+approach&rft.jtitle=Seminars+in+Cancer+Biology&rft.aulast=Manem&rft.aufirst=Venkata+S.K.&rft.au=Manem%2C%26%2332%3BVenkata+S.K.&rft.au=Salgado%2C%26%2332%3BRoberto&rft.au=Aftimos%2C%26%2332%3BPhilippe&rft.au=Sotiriou%2C%26%2332%3BChristos&rft.au=Haibe-Kains%2C%26%2332%3BBenjamin&rft.date=1+October+2018&rft.volume=52&rft.pages=135%E2%80%93150&rft_id=info:doi\/10.1016%2Fj.semcancer.2017.12.006&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS1044579X17302365&rfr_id=info:sid\/en.wikipedia.org:Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-5\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-5\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Tan, Eng-King (1 January 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1474442220304488\" target=\"_blank\">\"Movement disorders in 2020: clinical trials, genetic discoveries, and COVID-19\"<\/a> (in en). <i>The Lancet Neurology<\/i> <b>20<\/b> (1): 10\u201312. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2FS1474-4422%2820%2930448-8\" target=\"_blank\">10.1016\/S1474-4422(20)30448-8<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC7833604\/\" target=\"_blank\">PMC7833604<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/33340472\" target=\"_blank\">33340472<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1474442220304488\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1474442220304488<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Movement+disorders+in+2020%3A+clinical+trials%2C+genetic+discoveries%2C+and+COVID-19&rft.jtitle=The+Lancet+Neurology&rft.aulast=Tan&rft.aufirst=Eng-King&rft.au=Tan%2C%26%2332%3BEng-King&rft.date=1+January+2021&rft.volume=20&rft.issue=1&rft.pages=10%E2%80%9312&rft_id=info:doi\/10.1016%2FS1474-4422%2820%2930448-8&rft_id=info:pmc\/PMC7833604&rft_id=info:pmid\/33340472&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS1474442220304488&rfr_id=info:sid\/en.wikipedia.org:Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-6\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-6\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Bhagat, Seema; Kapatkar, Vaibhavi K.; Mane, Ashish; Pinto, Colette; Parikh, Devang; Mittal, Gaurav; Jain, Rishi (14 February 2020). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.eurekaselect.com\/175736\/article\" target=\"_blank\">\"An Industry Perspective on Risks and Mitigation Strategies Associated with Post Conduct Phase of Clinical Trial\"<\/a> (in en). <i>Reviews on Recent Clinical Trials<\/i> <b>15<\/b> (1): 28\u201333. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.2174%2F1574887114666191016103332\" target=\"_blank\">10.2174\/1574887114666191016103332<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.eurekaselect.com\/175736\/article\" target=\"_blank\">http:\/\/www.eurekaselect.com\/175736\/article<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=An+Industry+Perspective+on+Risks+and+Mitigation+Strategies+Associated+with+Post+Conduct+Phase+of+Clinical+Trial&rft.jtitle=Reviews+on+Recent+Clinical+Trials&rft.aulast=Bhagat&rft.aufirst=Seema&rft.au=Bhagat%2C%26%2332%3BSeema&rft.au=Kapatkar%2C%26%2332%3BVaibhavi+K.&rft.au=Mane%2C%26%2332%3BAshish&rft.au=Pinto%2C%26%2332%3BColette&rft.au=Parikh%2C%26%2332%3BDevang&rft.au=Mittal%2C%26%2332%3BGaurav&rft.au=Jain%2C%26%2332%3BRishi&rft.date=14+February+2020&rft.volume=15&rft.issue=1&rft.pages=28%E2%80%9333&rft_id=info:doi\/10.2174%2F1574887114666191016103332&rft_id=http%3A%2F%2Fwww.eurekaselect.com%2F175736%2Farticle&rfr_id=info:sid\/en.wikipedia.org:Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-7\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-7\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Nourani, Aynaz; Ayatollahi, Haleh; Dodaran, Masoud Solaymani (30 January 2019). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.eurekaselect.com\/165619\/article\" target=\"_blank\">\"A Review of Clinical Data Management Systems Used in Clinical Trials\"<\/a> (in en). <i>Reviews on Recent Clinical Trials<\/i> <b>14<\/b> (1): 10\u201323. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.2174%2F1574887113666180924165230\" target=\"_blank\">10.2174\/1574887113666180924165230<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.eurekaselect.com\/165619\/article\" target=\"_blank\">http:\/\/www.eurekaselect.com\/165619\/article<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Review+of+Clinical+Data+Management+Systems+Used+in+Clinical+Trials&rft.jtitle=Reviews+on+Recent+Clinical+Trials&rft.aulast=Nourani&rft.aufirst=Aynaz&rft.au=Nourani%2C%26%2332%3BAynaz&rft.au=Ayatollahi%2C%26%2332%3BHaleh&rft.au=Dodaran%2C%26%2332%3BMasoud+Solaymani&rft.date=30+January+2019&rft.volume=14&rft.issue=1&rft.pages=10%E2%80%9323&rft_id=info:doi\/10.2174%2F1574887113666180924165230&rft_id=http%3A%2F%2Fwww.eurekaselect.com%2F165619%2Farticle&rfr_id=info:sid\/en.wikipedia.org:Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-8\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-8\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Park, Yu Rang; Yoon, Young Jo; Koo, HaYeong; Yoo, Soyoung; Choi, Chang-Min; Beck, Sung-Ho; Kim, Tae Won (24 April 2018). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.jmir.org\/2018\/4\/e103\/\" target=\"_blank\">\"Utilization of a Clinical Trial Management System for the Whole Clinical Trial Process as an Integrated Database: System Development\"<\/a> (in en). <i>Journal of Medical Internet Research<\/i> <b>20<\/b> (4): e103. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.2196%2Fjmir.9312\" target=\"_blank\">10.2196\/jmir.9312<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1438-8871\" target=\"_blank\">1438-8871<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC5941091\/\" target=\"_blank\">PMC5941091<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/29691212\" target=\"_blank\">29691212<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.jmir.org\/2018\/4\/e103\/\" target=\"_blank\">http:\/\/www.jmir.org\/2018\/4\/e103\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Utilization+of+a+Clinical+Trial+Management+System+for+the+Whole+Clinical+Trial+Process+as+an+Integrated+Database%3A+System+Development&rft.jtitle=Journal+of+Medical+Internet+Research&rft.aulast=Park&rft.aufirst=Yu+Rang&rft.au=Park%2C%26%2332%3BYu+Rang&rft.au=Yoon%2C%26%2332%3BYoung+Jo&rft.au=Koo%2C%26%2332%3BHaYeong&rft.au=Yoo%2C%26%2332%3BSoyoung&rft.au=Choi%2C%26%2332%3BChang-Min&rft.au=Beck%2C%26%2332%3BSung-Ho&rft.au=Kim%2C%26%2332%3BTae+Won&rft.date=24+April+2018&rft.volume=20&rft.issue=4&rft.pages=e103&rft_id=info:doi\/10.2196%2Fjmir.9312&rft.issn=1438-8871&rft_id=info:pmc\/PMC5941091&rft_id=info:pmid\/29691212&rft_id=http%3A%2F%2Fwww.jmir.org%2F2018%2F4%2Fe103%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-9\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-9\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Nourani, Aynaz; Ayatollahi, Haleh; Dodaran, Masoud Solaymani (21 August 2019). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.eurekaselect.com\/169754\/article\" target=\"_blank\">\"Clinical Trial Data Management Software: A Review of the Technical Features\"<\/a> (in en). <i>Reviews on Recent Clinical Trials<\/i> <b>14<\/b> (3): 160\u2013172. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.2174%2F1574887114666190207151500\" target=\"_blank\">10.2174\/1574887114666190207151500<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.eurekaselect.com\/169754\/article\" target=\"_blank\">http:\/\/www.eurekaselect.com\/169754\/article<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Clinical+Trial+Data+Management+Software%3A+A+Review+of+the+Technical+Features&rft.jtitle=Reviews+on+Recent+Clinical+Trials&rft.aulast=Nourani&rft.aufirst=Aynaz&rft.au=Nourani%2C%26%2332%3BAynaz&rft.au=Ayatollahi%2C%26%2332%3BHaleh&rft.au=Dodaran%2C%26%2332%3BMasoud+Solaymani&rft.date=21+August+2019&rft.volume=14&rft.issue=3&rft.pages=160%E2%80%93172&rft_id=info:doi\/10.2174%2F1574887114666190207151500&rft_id=http%3A%2F%2Fwww.eurekaselect.com%2F169754%2Farticle&rfr_id=info:sid\/en.wikipedia.org:Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-10\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-10\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Barlow, Candida (1 April 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0749208120300164\" target=\"_blank\">\"Human Subjects Protection and Federal Regulations of Clinical Trials\"<\/a> (in en). <i>Seminars in Oncology Nursing<\/i> <b>36<\/b> (2): 151001. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.soncn.2020.151001\" target=\"_blank\">10.1016\/j.soncn.2020.151001<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0749208120300164\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0749208120300164<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Human+Subjects+Protection+and+Federal+Regulations+of+Clinical+Trials&rft.jtitle=Seminars+in+Oncology+Nursing&rft.aulast=Barlow&rft.aufirst=Candida&rft.au=Barlow%2C%26%2332%3BCandida&rft.date=1+April+2020&rft.volume=36&rft.issue=2&rft.pages=151001&rft_id=info:doi\/10.1016%2Fj.soncn.2020.151001&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0749208120300164&rfr_id=info:sid\/en.wikipedia.org:Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-11\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-11\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Barlow, Candida (1 April 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0749208120300206\" target=\"_blank\">\"Oncology Research: Clinical Trial Management Systems, Electronic Medical Record, and Artificial Intelligence\"<\/a> (in en). <i>Seminars in Oncology Nursing<\/i> <b>36<\/b> (2): 151005. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.soncn.2020.151005\" target=\"_blank\">10.1016\/j.soncn.2020.151005<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0749208120300206\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0749208120300206<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Oncology+Research%3A+Clinical+Trial+Management+Systems%2C+Electronic+Medical+Record%2C+and+Artificial+Intelligence&rft.jtitle=Seminars+in+Oncology+Nursing&rft.aulast=Barlow&rft.aufirst=Candida&rft.au=Barlow%2C%26%2332%3BCandida&rft.date=1+April+2020&rft.volume=36&rft.issue=2&rft.pages=151005&rft_id=info:doi\/10.1016%2Fj.soncn.2020.151005&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0749208120300206&rfr_id=info:sid\/en.wikipedia.org:Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:0-12\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:0_12-0\">12.0<\/a><\/sup> <sup><a href=\"#cite_ref-:0_12-1\">12.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Finniss, Damien G; Kaptchuk, Ted J; Miller, Franklin; Benedetti, Fabrizio (1 February 2010). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0140673609617062\" target=\"_blank\">\"Biological, clinical, and ethical advances of placebo effects\"<\/a> (in en). <i>The Lancet<\/i> <b>375<\/b> (9715): 686\u2013695. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2FS0140-6736%2809%2961706-2\" target=\"_blank\">10.1016\/S0140-6736(09)61706-2<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC2832199\/\" target=\"_blank\">PMC2832199<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/20171404\" target=\"_blank\">20171404<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0140673609617062\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0140673609617062<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Biological%2C+clinical%2C+and+ethical+advances+of+placebo+effects&rft.jtitle=The+Lancet&rft.aulast=Finniss&rft.aufirst=Damien+G&rft.au=Finniss%2C%26%2332%3BDamien+G&rft.au=Kaptchuk%2C%26%2332%3BTed+J&rft.au=Miller%2C%26%2332%3BFranklin&rft.au=Benedetti%2C%26%2332%3BFabrizio&rft.date=1+February+2010&rft.volume=375&rft.issue=9715&rft.pages=686%E2%80%93695&rft_id=info:doi\/10.1016%2FS0140-6736%2809%2961706-2&rft_id=info:pmc\/PMC2832199&rft_id=info:pmid\/20171404&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0140673609617062&rfr_id=info:sid\/en.wikipedia.org:Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-13\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-13\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">J\u00f8rgensen, Lars; Paludan-M\u00fcller, Asger S.; Laursen, David R. T.; Savovi\u0107, Jelena; Boutron, Isabelle; Sterne, Jonathan A. C.; Higgins, Julian P. T.; Hr\u00f3bjartsson, Asbj\u00f8rn (1 December 2016). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/systematicreviewsjournal.biomedcentral.com\/articles\/10.1186\/s13643-016-0259-8\" target=\"_blank\">\"Evaluation of the Cochrane tool for assessing risk of bias in randomized clinical trials: overview of published comments and analysis of user practice in Cochrane and non-Cochrane reviews\"<\/a> (in en). <i>Systematic Reviews<\/i> <b>5<\/b> (1): 80. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1186%2Fs13643-016-0259-8\" target=\"_blank\">10.1186\/s13643-016-0259-8<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2046-4053\" target=\"_blank\">2046-4053<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC4862216\/\" target=\"_blank\">PMC4862216<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/27160280\" target=\"_blank\">27160280<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/systematicreviewsjournal.biomedcentral.com\/articles\/10.1186\/s13643-016-0259-8\" target=\"_blank\">http:\/\/systematicreviewsjournal.biomedcentral.com\/articles\/10.1186\/s13643-016-0259-8<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Evaluation+of+the+Cochrane+tool+for+assessing+risk+of+bias+in+randomized+clinical+trials%3A+overview+of+published+comments+and+analysis+of+user+practice+in+Cochrane+and+non-Cochrane+reviews&rft.jtitle=Systematic+Reviews&rft.aulast=J%C3%B8rgensen&rft.aufirst=Lars&rft.au=J%C3%B8rgensen%2C%26%2332%3BLars&rft.au=Paludan-M%C3%BCller%2C%26%2332%3BAsger+S.&rft.au=Laursen%2C%26%2332%3BDavid+R.+T.&rft.au=Savovi%C4%87%2C%26%2332%3BJelena&rft.au=Boutron%2C%26%2332%3BIsabelle&rft.au=Sterne%2C%26%2332%3BJonathan+A.+C.&rft.au=Higgins%2C%26%2332%3BJulian+P.+T.&rft.au=Hr%C3%B3bjartsson%2C%26%2332%3BAsbj%C3%B8rn&rft.date=1+December+2016&rft.volume=5&rft.issue=1&rft.pages=80&rft_id=info:doi\/10.1186%2Fs13643-016-0259-8&rft.issn=2046-4053&rft_id=info:pmc\/PMC4862216&rft_id=info:pmid\/27160280&rft_id=http%3A%2F%2Fsystematicreviewsjournal.biomedcentral.com%2Farticles%2F10.1186%2Fs13643-016-0259-8&rfr_id=info:sid\/en.wikipedia.org:Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-14\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-14\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Cowie, Martin R.; Blomster, Juuso I.; Curtis, Lesley H.; Duclaux, Sylvie; Ford, Ian; Fritz, Fleur; Goldman, Samantha; Janmohamed, Salim <i>et al.<\/i> (1 January 2017). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/s00392-016-1025-6\" target=\"_blank\">\"Electronic health records to facilitate clinical research\"<\/a> (in en). <i>Clinical Research in Cardiology<\/i> <b>106<\/b> (1): 1\u20139. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2Fs00392-016-1025-6\" target=\"_blank\">10.1007\/s00392-016-1025-6<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1861-0684\" target=\"_blank\">1861-0684<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC5226988\/\" target=\"_blank\">PMC5226988<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/27557678\" target=\"_blank\">27557678<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/s00392-016-1025-6\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/s00392-016-1025-6<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Electronic+health+records+to+facilitate+clinical+research&rft.jtitle=Clinical+Research+in+Cardiology&rft.aulast=Cowie&rft.aufirst=Martin+R.&rft.au=Cowie%2C%26%2332%3BMartin+R.&rft.au=Blomster%2C%26%2332%3BJuuso+I.&rft.au=Curtis%2C%26%2332%3BLesley+H.&rft.au=Duclaux%2C%26%2332%3BSylvie&rft.au=Ford%2C%26%2332%3BIan&rft.au=Fritz%2C%26%2332%3BFleur&rft.au=Goldman%2C%26%2332%3BSamantha&rft.au=Janmohamed%2C%26%2332%3BSalim&rft.au=Kreuzer%2C%26%2332%3BJ%C3%B6rg&rft.date=1+January+2017&rft.volume=106&rft.issue=1&rft.pages=1%E2%80%939&rft_id=info:doi\/10.1007%2Fs00392-016-1025-6&rft.issn=1861-0684&rft_id=info:pmc\/PMC5226988&rft_id=info:pmid\/27557678&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2Fs00392-016-1025-6&rfr_id=info:sid\/en.wikipedia.org:Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-15\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-15\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Sharma, Abhinav; Harrington, Robert A.; McClellan, Mark B.; Turakhia, Mintu P.; Eapen, Zubin J.; Steinhubl, Steven; Mault, James R.; Majmudar, Maulik D. <i>et al.<\/i> (1 June 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0735109718344139\" target=\"_blank\">\"Using Digital Health Technology to Better Generate Evidence and Deliver Evidence-Based Care\"<\/a> (in en). <i>Journal of the American College of Cardiology<\/i> <b>71<\/b> (23): 2680\u20132690. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.jacc.2018.03.523\" target=\"_blank\">10.1016\/j.jacc.2018.03.523<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0735109718344139\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0735109718344139<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Using+Digital+Health+Technology+to+Better+Generate+Evidence+and+Deliver+Evidence-Based+Care&rft.jtitle=Journal+of+the+American+College+of+Cardiology&rft.aulast=Sharma&rft.aufirst=Abhinav&rft.au=Sharma%2C%26%2332%3BAbhinav&rft.au=Harrington%2C%26%2332%3BRobert+A.&rft.au=McClellan%2C%26%2332%3BMark+B.&rft.au=Turakhia%2C%26%2332%3BMintu+P.&rft.au=Eapen%2C%26%2332%3BZubin+J.&rft.au=Steinhubl%2C%26%2332%3BSteven&rft.au=Mault%2C%26%2332%3BJames+R.&rft.au=Majmudar%2C%26%2332%3BMaulik+D.&rft.au=Roessig%2C%26%2332%3BLothar&rft.date=1+June+2018&rft.volume=71&rft.issue=23&rft.pages=2680%E2%80%932690&rft_id=info:doi\/10.1016%2Fj.jacc.2018.03.523&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0735109718344139&rfr_id=info:sid\/en.wikipedia.org:Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-16\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-16\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Kempf, Lucas; Goldsmith, Jonathan C.; Temple, Robert (1 April 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/ajmg.a.38413\" target=\"_blank\">\"Challenges of developing and conducting clinical trials in rare disorders\"<\/a> (in en). <i>American Journal of Medical Genetics Part A<\/i> <b>176<\/b> (4): 773\u2013783. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1002%2Fajmg.a.38413\" target=\"_blank\">10.1002\/ajmg.a.38413<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1552-4825\" target=\"_blank\">1552-4825<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/ajmg.a.38413\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/ajmg.a.38413<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Challenges+of+developing+and+conducting+clinical+trials+in+rare+disorders&rft.jtitle=American+Journal+of+Medical+Genetics+Part+A&rft.aulast=Kempf&rft.aufirst=Lucas&rft.au=Kempf%2C%26%2332%3BLucas&rft.au=Goldsmith%2C%26%2332%3BJonathan+C.&rft.au=Temple%2C%26%2332%3BRobert&rft.date=1+April+2018&rft.volume=176&rft.issue=4&rft.pages=773%E2%80%93783&rft_id=info:doi\/10.1002%2Fajmg.a.38413&rft.issn=1552-4825&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1002%2Fajmg.a.38413&rfr_id=info:sid\/en.wikipedia.org:Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-17\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-17\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Brasil, Sandra; Pascoal, Carlota; Francisco, Rita; dos Reis Ferreira, Vanessa; A. Videira, Paula; Valad\u00e3o, Gon\u00e7alo (27 November 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.mdpi.com\/2073-4425\/10\/12\/978\" target=\"_blank\">\"Artificial Intelligence (AI) in Rare Diseases: Is the Future Brighter?\"<\/a> (in en). <i>Genes<\/i> <b>10<\/b> (12): 978. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3390%2Fgenes10120978\" target=\"_blank\">10.3390\/genes10120978<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2073-4425\" target=\"_blank\">2073-4425<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC6947640\/\" target=\"_blank\">PMC6947640<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/31783696\" target=\"_blank\">31783696<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.mdpi.com\/2073-4425\/10\/12\/978\" target=\"_blank\">https:\/\/www.mdpi.com\/2073-4425\/10\/12\/978<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Artificial+Intelligence+%28AI%29+in+Rare+Diseases%3A+Is+the+Future+Brighter%3F&rft.jtitle=Genes&rft.aulast=Brasil&rft.aufirst=Sandra&rft.au=Brasil%2C%26%2332%3BSandra&rft.au=Pascoal%2C%26%2332%3BCarlota&rft.au=Francisco%2C%26%2332%3BRita&rft.au=dos+Reis+Ferreira%2C%26%2332%3BVanessa&rft.au=A.+Videira%2C%26%2332%3BPaula&rft.au=Valad%C3%A3o%2C%26%2332%3BGon%C3%A7alo&rft.date=27+November+2019&rft.volume=10&rft.issue=12&rft.pages=978&rft_id=info:doi\/10.3390%2Fgenes10120978&rft.issn=2073-4425&rft_id=info:pmc\/PMC6947640&rft_id=info:pmid\/31783696&rft_id=https%3A%2F%2Fwww.mdpi.com%2F2073-4425%2F10%2F12%2F978&rfr_id=info:sid\/en.wikipedia.org:Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-18\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-18\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Wu, Jasmanda; Wang, Cunlin; Toh, Sengwee; Pisa, Federica Edith; Bauer, Larry (1 October 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/pds.4962\" target=\"_blank\">\"Use of real\u2010world evidence in regulatory decisions for rare diseases in the United States\u2014Current status and future directions\"<\/a> (in en). <i>Pharmacoepidemiology and Drug Safety<\/i> <b>29<\/b> (10): 1213\u20131218. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1002%2Fpds.4962\" target=\"_blank\">10.1002\/pds.4962<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1053-8569\" target=\"_blank\">1053-8569<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/pds.4962\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/pds.4962<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Use+of+real%E2%80%90world+evidence+in+regulatory+decisions+for+rare+diseases+in+the+United+States%E2%80%94Current+status+and+future+directions&rft.jtitle=Pharmacoepidemiology+and+Drug+Safety&rft.aulast=Wu&rft.aufirst=Jasmanda&rft.au=Wu%2C%26%2332%3BJasmanda&rft.au=Wang%2C%26%2332%3BCunlin&rft.au=Toh%2C%26%2332%3BSengwee&rft.au=Pisa%2C%26%2332%3BFederica+Edith&rft.au=Bauer%2C%26%2332%3BLarry&rft.date=1+October+2020&rft.volume=29&rft.issue=10&rft.pages=1213%E2%80%931218&rft_id=info:doi\/10.1002%2Fpds.4962&rft.issn=1053-8569&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1002%2Fpds.4962&rfr_id=info:sid\/en.wikipedia.org:Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-19\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-19\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation\" id=\"rdp-ebb-CITEREFGroftPosada_de_la_Paz2017\">Groft, Stephen C.; Posada de la Paz, Manuel (2017), Posada de la Paz, Manuel; Taruscio, Domenica; Groft, Stephen C., eds., <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/978-3-319-67144-4_34\" target=\"_blank\">\"Preparing for the Future of Rare Diseases\"<\/a>, <i>Rare Diseases Epidemiology: Update and Overview<\/i> (Cham: Springer International Publishing) <b>1031<\/b>: 641\u2013648, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2F978-3-319-67144-4_34\" target=\"_blank\">10.1007\/978-3-319-67144-4_34<\/a>, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-3-319-67142-0<span class=\"printonly\">, <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/978-3-319-67144-4_34\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/978-3-319-67144-4_34<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 2023-06-28<\/span><\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Preparing+for+the+Future+of+Rare+Diseases&rft.jtitle=Rare+Diseases+Epidemiology%3A+Update+and+Overview&rft.aulast=Groft&rft.aufirst=Stephen+C.&rft.au=Groft%2C%26%2332%3BStephen+C.&rft.au=Posada+de+la+Paz%2C%26%2332%3BManuel&rft.date=2017&rft.volume=1031&rft.pages=641%E2%80%93648&rft.place=Cham&rft.pub=Springer+International+Publishing&rft_id=info:doi\/10.1007%2F978-3-319-67144-4_34&rft.isbn=978-3-319-67142-0&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2F978-3-319-67144-4_34&rfr_id=info:sid\/en.wikipedia.org:Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-20\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-20\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Hurley, Caroline; Shiely, Frances; Power, Jessica; Clarke, Mike; Eustace, Joseph A.; Flanagan, Evelyn; Kearney, Patricia M. (1 November 2016). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1551714416302877\" target=\"_blank\">\"Risk based monitoring (RBM) tools for clinical trials: A systematic review\"<\/a> (in en). <i>Contemporary Clinical Trials<\/i> <b>51<\/b>: 15\u201327. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.cct.2016.09.003\" target=\"_blank\">10.1016\/j.cct.2016.09.003<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1551714416302877\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1551714416302877<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Risk+based+monitoring+%28RBM%29+tools+for+clinical+trials%3A+A+systematic+review&rft.jtitle=Contemporary+Clinical+Trials&rft.aulast=Hurley&rft.aufirst=Caroline&rft.au=Hurley%2C%26%2332%3BCaroline&rft.au=Shiely%2C%26%2332%3BFrances&rft.au=Power%2C%26%2332%3BJessica&rft.au=Clarke%2C%26%2332%3BMike&rft.au=Eustace%2C%26%2332%3BJoseph+A.&rft.au=Flanagan%2C%26%2332%3BEvelyn&rft.au=Kearney%2C%26%2332%3BPatricia+M.&rft.date=1+November+2016&rft.volume=51&rft.pages=15%E2%80%9327&rft_id=info:doi\/10.1016%2Fj.cct.2016.09.003&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS1551714416302877&rfr_id=info:sid\/en.wikipedia.org:Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<\/ol><\/div><\/div>\n<h2><span class=\"mw-headline\" id=\"Notes\">Notes<\/span><\/h2>\n<p>This presentation is faithful to the original, with only a few minor changes to presentation, grammar, and punctuation. In some cases important information was missing from the references, and that information was added.\n<\/p>\n<!-- \nNewPP limit report\nCached time: 20231215050118\nCache expiry: 86400\nDynamic content: false\nComplications: []\nCPU time usage: 0.537 seconds\nReal time usage: 0.728 seconds\nPreprocessor visited node count: 23385\/1000000\nPost\u2010expand include size: 211740\/2097152 bytes\nTemplate argument size: 62054\/2097152 bytes\nHighest expansion depth: 25\/40\nExpensive parser function count: 0\/100\nUnstrip recursion depth: 0\/20\nUnstrip post\u2010expand size: 53315\/5000000 bytes\n-->\n<!--\nTransclusion expansion time report (%,ms,calls,template)\n100.00% 361.372 1 -total\n 86.09% 311.088 1 Template:Reflist\n 64.71% 233.851 19 Template:Cite_journal\n 63.51% 229.519 20 Template:Citation\/core\n 11.69% 42.236 20 Template:Date\n 9.55% 34.526 1 Template:Infobox_journal_article\n 8.54% 30.844 44 Template:Citation\/identifier\n 8.53% 30.813 1 Template:Infobox\n 5.04% 18.225 1 Template:Citation\n 4.65% 16.795 80 Template:Infobox\/row\n-->\n\n<!-- Saved in parser cache with key limswiki:pcache:idhash:14270-0!canonical and timestamp 20231215050117 and revision id 52371. Serialized with JSON.\n -->\n<\/div><\/div><div class=\"printfooter\">Source: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.limswiki.org\/index.php\/Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system\">https:\/\/www.limswiki.org\/index.php\/Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system<\/a><\/div>\n<!-- end content --><div class=\"visualClear\"><\/div><\/div><\/div><div class=\"visualClear\"><\/div><\/div><!-- end of the left (by default at least) column --><div class=\"visualClear\"><\/div><\/div>\n\n\n\n<\/body>","e65552a701c8075f24aa45db5f398e2e_images":["https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/e\/e2\/Fig1_Shen_BMCMedInfoDecMak23_23.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/8\/87\/Fig2_Shen_BMCMedInfoDecMak23_23.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/9\/92\/Fig3_Shen_BMCMedInfoDecMak23_23.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/c\/cf\/Fig4_Shen_BMCMedInfoDecMak23_23.png"],"e65552a701c8075f24aa45db5f398e2e_timestamp":1702682173,"b283f4f5c78061a91491bc16c8576d36_type":"article","b283f4f5c78061a91491bc16c8576d36_title":"A web application to support the coordination of reflexive, interpretative toxicology testing (Pablo et al. 2023)","b283f4f5c78061a91491bc16c8576d36_url":"https:\/\/www.limswiki.org\/index.php\/Journal:A_web_application_to_support_the_coordination_of_reflexive,_interpretative_toxicology_testing","b283f4f5c78061a91491bc16c8576d36_plaintext":"\n\nJournal:A web application to support the coordination of reflexive, interpretative toxicology testingFrom LIMSWikiJump to navigationJump to searchFull article title\n \nA web application to support the coordination of reflexive, interpretative toxicology testingJournal\n \nJournal of Pathology InformaticsAuthor(s)\n \nPablo, Abed; Laha, Thomas J.; Breit, Nathan; Hoffman, Noah G.; Hoofnagle, Andrew N.; Baird, Geoffrey S.; Mathias, Patrick C.Author affiliation(s)\n \nUniversity of Washington School of MedicinePrimary contact\n \nEmail: pcm10 at uw dot eduYear published\n \n2023Volume and issue\n \n14Article #\n \n100303DOI\n \n10.1016\/j.jpi.2023.100303ISSN\n \n2153-3539Distribution license\n \nCreative Commons Attribution 4.0 InternationalWebsite\n \nhttps:\/\/www.sciencedirect.com\/science\/article\/pii\/S2153353923001177Download\n \nhttps:\/\/www.sciencedirect.com\/science\/article\/pii\/S2153353923001177\/pdfft (PDF)\n\nContents \n\n1 Abstract \n2 Background \n3 Methods \n\n3.1 Pre-application testing and workflows \n3.2 Requirements \n3.3 Technical requirements \n3.4 Performance measurement and user feedback \n\n\n4 Results \n\n4.1 Laboratory workflows prior to implementation \n4.2 Technical requirements \n4.3 Requirements \n4.4 Performance measurement and user feedback \n\n\n5 Discussion \n6 Abbreviations, acronyms, and initialisms \n7 Appendix A. Supplementary data \n8 Acknowledgements \n\n8.1 Conflict of interest \n\n\n9 References \n10 Notes \n\n\n\nAbstract \nBackground: Reflexive laboratory testing workflows can improve the assessment of patients receiving pain medications chronically, but complex workflows requiring pathologist input and interpretation may not be well-supported by traditional laboratory information systems (LISs). In this work, we describe the development of a web application that improves the efficiency of pathologists and laboratory staff in delivering actionable toxicology results.\nMethod: Before designing the application, we set out to understand the entire workflow, including the laboratory workflow and pathologist review. Additionally, we gathered requirements and specifications from stakeholders. Finally, to assess the performance of the implementation of the application, we surveyed stakeholders and documented the approximate amount of time that is required in each step of the workflow.\nResults: A web-based application was chosen for the ease of access for users. Relevant clinical data was routinely received and displayed in the application. The workflows in the laboratory and during the interpretation process served as the basis of the user interface (UI). With the addition of auto-filing software, the return on investment (ROI) was significant. The laboratory saved the equivalent of one full-time employee in time by automating file management and result entry.\nDiscussion: Implementation of a purpose-built application to support reflex and interpretation workflows in a clinical pathology practice has led to a significant improvement in laboratory efficiency. Custom- and purpose-built applications can help reduce staff burnout, reduce transcription errors, and allow staff to focus on more critical issues around quality.\nKeywords: Python, laboratory workflows, custom web application, quality control, mass spectrometry\n\nBackground \nUrine drug monitoring is an increasingly important tool for primary care physicians and pain specialists to monitor patients on chronic opioid therapy.[1] Routine and random monitoring for all patients on long-term opioid therapy is recommended prior to starting and throughout drug therapy.[2] There are various analytical methods and matrices available for patient monitoring. Urine drug testing (UDT) has become the \u201cgold-standard\u201d for detecting illicit drug use and monitoring ongoing therapy. Urine is non-invasive, readily available, and contains higher concentrations of drugs and metabolites compared with other matrices. Immunoassay testing, commonly referred to as urine drug screening, detects the presence of selected drugs and\/or metabolites based on a detection threshold. Typically, the immunoassay functions as the initial evaluation for the potential detection of drugs, but because it lacks specificity for analytes of the same drug class, it can lead to false-positive and -negative results. Therefore, drug screens are often coupled with a confirmatory detection method for more specific testing. Chromatography is generally reserved for confirmatory or definitive testing following immunoassay screening. Historically, gas chromatography\u2013mass spectrometry (GC\u2013MS) was the standard for confirmatory testing. However, liquid chromatography\u2013mass spectrometry\u2013tandem mass spectrometry (LC-MS\/MS) has gained favor over GC-MS due to reduced complexity of sample preparation methods and the broader applicability of LC-MS\/MS to different drug classes.[3][4]\nMany laboratories are equipped to support UDT.[1][3][4][5][6] A major challenge of staged drug testing (i.e., screening and confirmatory assays) is the review and interpretation of the large number of quantitative and qualitative results that is generated by screening and confirmatory methods.[7][8][9] Pharmacokinetic mechanisms dictate how quickly and how much of a drug and its metabolites appear in an individual's urine, which can be challenging for physicians who have limited understanding due to reviewing a limited number of these results.[7] An incorrect interpretation of the LC-MS\/MS results could mistakenly suggest that a patient used a non-prescribed medication. Additionally, the potential for analytes to co-elute could affect the signal of the analytes, influencing the overall accuracy of the LC-MS\/MS test.[4] \nYang et al.[10] described that adding interpretative comments from a trained laboratory director to urine drug screening results can greatly impact both the clinician and patient. Clinicians may not understand the analytical method limitations, and any resulting misinterpretation of the results may lead to a patient's stigmatization or inappropriate termination from care. The authors further reported an increase in urine drug screen orders with interpretive comments, indicating that clinicians value the interpretation made by a trained director. Because accurate interpretation of results often requires the integration of documented clinical information, medication data, test results, pharmacokinetics, and test limitations and interferences, assigning a trained pathologist or clinical chemist who is familiar with interpreting clinical information, as well as the biochemistry and testing methods to provide an interpretation of the results, is a valuable tool in ensuring the correct diagnosis for patients.\nOne challenge laboratories and laboratory directors face with UDT is that there may be results from several tests that inform whether additional testing is required and need to be integrated to provide an interpretation given the specific clinical context. Within the Department of Laboratory Medicine and Pathology at University of Washington Medicine (UW Medicine), we have formulated a urine drug testing ordering protocol and interpretation workflow. Clinicians may select from a menu of panels that are based on the assessed patient's risk of treatment non-compliance. Panels include an immunoassay drug screen and confirmation LC-MS\/MS assays for opioids, amphetamines, benzodiazepines, and alcohol. For example, a low-risk patient panel consists of an immunoassay drug screen, and aberrant or unexpected results can lead the reviewing pathologist to order one or more of the confirmation tests. On top of the interpretation challenges, turnaround times (TAT) for the different tests are unevenly distributed. Most laboratory information systems (LISs) are not equipped to track, collect, and display the different results in an effective and informative manner to support this workflow.\nThe aim of this project was to design and implement a web-based software application that centralizes test results for pathologist review, performs the quality control (QC) calculations for the complex LC-MS\/MS analysis of opioids and their metabolites, functions as a user input form for pathologist interpretation entry, and auto-files results into the LIS.\n\nMethods \nThe initial step before designing the application was to understand the entire workflow, then gather requirements and specifications from stakeholders.\n\nPre-application testing and workflows \nAt UW Medicine, for each patient specimen undergoing the opiate confirmation assay, two extractions are prepared, an undiluted and a diluted 10-fold urine sample. To help improve the complex LC-MS\/MS data analysis of opioids and opioid metabolites, a command-line software application was developed.[11] The software application known as \u201cSMACK\u201d was shown to improve the data analysis process by automating a QC algorithm and calculations to improve consistency of analysis as well as reduce the amount of time medical laboratory scientists (MLS) spent reviewing the data.\nThe algorithm was part of an intricate workflow that involved several steps, spreadsheets, and individuals. The goal of the spreadsheets was to collect and consolidate the relevant patient information so that the reviewing pathologist had the necessary information to generate an interpretation for the overall case. The information with the relevant patient data was spread across several spreadsheets, the creation of which relied on laboratory staff and pathologists copying and pasting various fields from one spreadsheet to another. Pathologists and laboratory staff communicated the completion of steps via email while using a naming convention for the relevant files.\n\nRequirements \nWe set to gather information on how clinicians ordered the different drug monitoring tests and how the information arrived at the laboratory. We investigated the LIS data workflow that was in place and how this information could feed the application, including assay results and patient demographics. Additionally, we worked with the laboratory to understand how test results were produced and how this information should be included in the application. Lastly, we investigated what the requirements were for progressing from one step of the workflow to the next along with the data associated with the progression. In addition to the workflow in the laboratory, we needed to understand what data elements are needed for each step of the workflow. With this information, we would create a map of the workflow to visualize and better understand the workflow. We gathered necessary components from stakeholders, pathologists, directors, pathologists in training, and laboratory staff by interviewing stakeholders and observing workflows. During this process, we gathered user interface (UI) requirements from stakeholders to shape functionality of the optimized web application.\n\nTechnical requirements \nThe previously described QC software SMACK had been implemented as a command-line application in Python, with no dependencies outside of the standard library.[11] Input data is provided as an XML format file exported from Waters' TargetLynx software. The outputs of the software are two CSV-formatted reports, one containing assay results and the other a detailed report of the QC calculations. Appendix A, Supplemental Fig. 1 displays the flow chart defining the QC algorithm, which has been simplified since its initial implementation. For this project, we needed to incorporate the software into the application to perform the same QC calculations on the opiate assay raw data and then display the output to the user in a meaningful manner. Moreover, our group has a defined technical software stack for developing and deploying applications in Python. For these reasons, writing the application in Python was the most efficient choice. An additional requirement was to automatically file results into the LIS (from CliniSys, Inc.). To achieve this requirement, we had to work with Data Innovations' Instrument Manager software.\n\nPerformance measurement and user feedback \nFinally, to assess the performance, we surveyed stakeholders and documented the rough amount of time that is required in each step of the workflow. Input was solicited from several laboratory staff members rotating through the area regarding how much time was spent on each step before and after the implementation of the application. We also surveyed the pathologists in training (chemistry fellows and rotating residents), known as trainees. Finally, we extracted data from the LIS to capture the number of occurrences in which a testing order had a correction or modification after the initial data entry so we could compare correction rates prior to and after implementation.\n\nResults \nLaboratory workflows prior to implementation \nProviders order UDTs for the patient based on an assessment of the patient's risk for compliance. Table 1 displays the patient risk levels, the intended population, and the tests associated with each risk panel.\n\n\n\n\nTable 1. Urine drug testing ordering strategy based on risk. Providers assess risk, review clinical patient care data, and order tests as described.\n\n\nRisk\n\nIntended population\n\nTests\n\n\nLow\n\nMorphine equivalent dose <90\u202fmg\/day; Stable on chronic opioid therapy or buprenorphine therapy\n\nDrug screen immunoassay\n\n\nMedium to high\n\nMorphine equivalent dose >=90\u202fmg\/day; Aberrant behavior (including early prescriptions, lost prescriptions, outbursts in office)\n\nDrug screen immunoassay; Enzymatic assay for alcohol; Opiate confirmation LCMS\n\n\nHigh\n\nSame as medium to high; added concern for clonazepam or lorazepam\n\nDrug screen immunoassay; Enzymatic assay for alcohol; Opiate confirmation LCMS; Benzodiazepine confirmation LCMS\n\n\nFor each panel, providers have the option to enter any drug the patient is expected to be taking which may influence the test results. Each test in the panel is conducted separately and the results are entered into the LIS. Once all the tests have been completed and results entered, the pathologist may begin generating an interpretation.\nWhen pathologists are reviewing the results, they consider the patient's history in the electronic health records (EHRs) for information that may influence the test results such as medications not specified by the ordering provider. These notes would be entered in dedicated fields in the spreadsheet. For training purposes, a pathologist trainee will often review the results first, then review their work with a director who may discuss the case, edit the preliminary interpretation, then make the final approval. Upon review of cases that initially only received the immunoassay drug screen, a probable outcome is for the reviewing pathologist to deem additional confirmation testing necessary. Therefore, the workflow is the summation of two separate parts, cases which receive opiate confirmation testing versus those that do not. The consideration to separate the workflow into two parts was partly because many orders require additional reflexive orders which influences how quickly the final interpretations will be released. For this reason, it was best to group cases by risk level to provide an interpretation as quickly as possible. Laboratory staff worked with departmental software engineers to create and deliver spreadsheets for each workflow which contain the patient's demographics (medical record number, age, sex, name, and order number) along with the results of the tests.\nThe opiate assay is performed on a Waters Xevo TQ-MS tandem mass spectrometer with an Aquity UPLC system. Data acquisition is controlled by Waters MassLynx and chromatography review is completed using Waters TargetLynx software. When the data acquisition is complete, the technologist completes a cursory review of the auto-integration using TargetLynx. Once the preliminary review is complete, an XML file with the raw data is created. The XML file is the input of the application that performs QC calculations from the LC-MS\/MS raw data. The output was two detailed tabular reports; the first was the calculated results after the algorithm was applied and another with a detailed report of the QC calculations. These reports would be reviewed in Microsoft Excel by two laboratory staff members to address any values that did not meet QC standards before entering the results into the LIS. Once the results have been reviewed, staff would send an email to the reviewing pathologists that reports were ready for their review. The pathologist would then combine the calculated results with the spreadsheet containing other test results and patient demographics via a copy-paste method. During the pathologist review, they may identify aberrant results stemming from interfering compounds or possible adulterations based on the data. They would then create an email thread with the laboratory to communicate and resolve the issue.\nOne possible outcome for those cases where patient risk levels are associated with low to medium risk, a reviewing pathologist orders a confirmatory test. Pathologists would type the test codes for the orders they wished to place in the spreadsheet and the reviewing MLS would place the orders in the LIS. Therefore, a third spreadsheet was created to capture the results for the cases that received additional tests. To transfer the notes captured during the initial review, a Microsoft Excel macro was created which would parse the spreadsheets and bring the pathologist's saved work from the initial spreadsheet into the final spreadsheet.\nThese spreadsheets were created and delivered via a dedicated website where staff and faculty were able to download the spreadsheets from any computer in the laboratory. After the spreadsheet was initially downloaded and edited, the file was stored on a shared drive accessible through a protected network that was accessible to the laboratory and pathologists.\nA naming convention for these files was created to help indicate the stage in the workflow.\nOnce the reviewing pathologist had completed their review, they would send an email to staff so that they could enter the interpretation result into the LIS manually. If the technologist identified any issues with the interpretation, they would email the pathologist to edit\/review the interpretation.\n\nTechnical requirements \nThe application was implemented in Python using the Flask web framework[12] and other open-source libraries. Appendix A, Supplemental Table 1 lists the required libraries used for the application.\nThe software was developed on an Apple Macintosh running Mac OS 10.15 and is used in production on Amazon Web Services (AWS) server running Ubuntu (18.04 LTS). The data from the LIS is gathered using Cache Object Script that collects information every 30\u202fminutes from 6 a.m. to 6 p.m. and is delivered to a secure cloud storage resource (S3 bucket) on AWS. Once the results are finalized, the application sends a tab-delimited file to a transfer Linux (4.15.0) server which is picked up by Data Innovations instrument manager that sends the data to the LIS.\nAn important component of the software development and deployment process was the use of the Git version control software. Git is a free, open-source, distributed version control software that can be used to store \u201csnapshots\u201d of source code files in which modifications are captured to the code repository. Each change is attributed to a specific author and can be associated with a comment describing the intent of the change. A unique tag identifies each change, and the application captures the tag in the software version number and records this identifier to every data output. In this manner, results of each analysis can be traced to the exact version of the software used to collect and generate those results. Additionally, GitLab is a Git-based fully integrated platform which allows users to create \u201cissues\u201d where enhancements, development, and other topics may be discussed collectively. This feature of GitLab can also attribute labels to each issue so that larger topics or concepts can be collectively viewed and labeled.\nThe application is a web-based application that can be launched from any web browser. Access control is managed using our institutional identity provider with single sign-on with two-factor authentication. For auto-resulting, once the results reach their terminal status (pathologists and laboratory staff have completed their collective review), a tab-separated-values (TSV) is sent to Data Innovations' Instrument Manager where it files the results to the LIS.\n\nRequirements \nThe data collected and workflows described serviced as the foundation of the application's design. The application needed to gather the necessary data from the LIS and display the information in a manner that is useful to the user. A system needed to be designed to progress a case through the necessary steps for the appropriate reviewer. Entry fields needed to be available for all members to be able to communicate problems, capture notes by the pathologists during their review, and for the final interpretation result. The application needed to ingest the XML file from TargetLynx of the opiate test, apply the SMACK algorithm, display both the QC calculations and results, and save all edits. Lastly, the application needed a system to communicate any additional confirmation tests the pathologist wanted to order.\nA new component within the laboratory workflow was the introduction of an automated liquid handler to prepare the urine samples for the opiate assay. The laboratory had been manually preparing the samples and entering the patient identifiers into the acquisition system, MassLynx. When creating the extraction method of the liquid handler, the decision was made to use the barcode on the specimen label as the sample identifier for the assay. To associate the result back to the patient, all specimen IDs also needed to be gathered.\nThe decision to create a web-based application was to provide an easy-access entry point for the laboratory and pathologists. Also, a web-based application provided the flexibility needed to create a custom site that fit the needs of the workflow. Fig. 1 shows the home page of the application.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 1. Home page of the opiate sign-out application. The landing page lists all pending samples in the application along with sample information such as case number, container ID (LCMS), medical record number, patient name, and status of each test.\n\n\n\nWe provided a tabular pending list that indicates the case status, order status, order number, LCMS results page, medical record number, patient name, and the status of each test. Cases were provided status IDs to progress each case through the different steps of the workflow. While investigating the overall workflows we determined that workflows could be categorized by the different stakeholders, the pathologists and the MLSs, and the different tasks associated with each personnel. From this, we created filters for the pending lists so that each user could easily jump into their associated workflow and collectively work on cases based on the case status (Fig. 2). We added a search bar so that a user could search any pending or historical case by name, order number, hospital ID, or specimen ID.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 2. General workflow outline. The status of each case drives the case from one step in the workflow to the next until the result is filed into the LIS.\n\n\n\nFor each case, a new page is created where the patient demographics, test results, and entry fields are displayed. Fig. 3 shows what the case page looks like in the application for a given patient, and Table 2 lists the separate features displayed on the page.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 3. Case page example. The page displays the fields and layout of the case page of a given patient.\n\n\n\n\n\n\nTable 2. Listed case page features. Each listed feature is an object that supports the process of generating the test interpretation by a pathologist.\n\n\nNumber\n\nFeature\n\nDescription\n\n\n1\n\nPending case list\n\nA list of all the pending cases for review\n\n\n2\n\nIcon definitions\n\nThe key maps out the status symbols and their definitions\n\n\n3\n\nOrder information\n\nPatient and order information of the overall case\n\n\n4\n\nScreening results\n\nThis card displays the results of the tests by test component. If a test is still pending or incomplete in the LIS, a gray \u201cpending\u201d box will be displayed in the \u201cTests\u201d column.\n\n\n5\n\nMedication list\n\nEntry field where medications found in the patient\u2019s chart are entered\n\n\n6\n\nChart reviewed\n\nFor billing purposes, pathologists indicate whether the patient\u2019s chart was reviewed during review, a value that is entered, as a result, into the LIS.\n\n\n7\n\nReflex to\n\nThe pathologist will mark these boxes to indicate which additional tests will be ordered by the MLS in the LIS.\n\n\n8\n\nRequest urine specific gravity\n\nThe pathologist will check this box for MLSs to perform a specific gravity test and they will populate the entry box with the result.\n\n\n9\n\nCase interpretation (UINTDS)\n\nEntry field used to capture the pathologist generated interpretation\n\n\n10\n\nCase status\n\nThis is a dropdown box used to communicate and move the case through the sign-out process. Statuses are changed after the person has completed their duties for the case.\n\n\n11\n\nSignoff check boxes\n\nWhen a director checks this box, it will gather their name and associate it with the additional order or result for billing purposes.\n\n\n12\n\nComment\n\nEntry field used to gather comments, questions, concerns regarding the overall case\n\n\n13\n\nSave\n\nThis button will save all input and changes made to the page\n\n\n14\n\nHistory\n\nHistorical changes and comments are displayed with the employee ID of the person who made the change and timestamp of the change.\n\n\nFor orders where the opiate LCMS confirmation test was placed, a separate page is created known as the LCMS page. Fig. 4 displays the LCMS page for a given patient. This page holds much of the same information as the case page but with dedicated space for the result of the QC calculations. In this manner, we avoid cluttering a single page, and if a case did not have the opiate confirmation test ordered, the page only shows relevant information. When the LCMS page exists, the specimen ID is the basis of the URL of the LCMS page. This page duplicates many of the features displayed on the case page to help highlight key information and provide ease of access to resources for the reviewing pathologists.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 4. LCMS page example. The figure displays the several features of the LCMS page for a given patient including the output table of the SMACK QC calculations.\n\n\n\nWhen an XML file is uploaded to the application, the application applies the algorithm and automatically associates the result with a patient using the unique specimen ID. The result table can be viewed in one of three configurations:\n\n1. Metabolite \u2013 Compounds are listed together by metabolic groups.\n2. Worksheet \u2013 Compounds are listed in the order seen in the LIS.\n3. Sequential \u2013 Compounds are listed in the order seen in TargetLynx.\nThe opiate confirmation assay is quantitative\/qualitative. Values that require review by the laboratory staff for QC purposes are highlighted in red. Positive values are highlighted in green. Either a positive numerical value is displayed for quantitative results or \u201cPOS\u201d is displayed for positive qualitative results. Negatives are displayed in black and denoted by \u201cNRN.\u201d\nOnce the interpretations have been created, the laboratory staff will review it for accuracy (ensuring that each drug\/analyte is mentioned) before submitting it to the LIS. If they identify a problem, they will change the status of the case to \u201cProblem for Director.\u201d This status captures all cases that require special attention by the attending pathologist, also known as the Director. Once the problem has been addressed, the pathologist may change the status to the appropriate selection.\nThe status has been the main factor for progressing a case through the workflow. We recognized that the overall workflow could be split by personnel, the MLS staff, and pathologists, and could be further broken up by the steps in those workflows. Therefore, we developed separate tabular views (Table 3) that collectively group pending cases based on the step in the workflow the case currently resides.\n\n\n\n\nTable 3. Tabular views of pending cases. Each tab holds a list of cases based on the state the case resides in the workflow.\n\n\nNumber\n\nTable filter\n\nDescription\n\n\n1\n\nUPDRS results entry\n\nFor these cases, the immunoassay results have been reviewed and an interpretation has been finalized by an attending pathologist. The interpretation of these cases requires a review by an MLS to ensure that there are no expected compounds or medications that were not mentioned in the interpretation.\n\n\n2\n\nReflexes\n\nThis is a list of cases that has been reviewed by an attending pathologist; the cases have been requested\/approved for a reflex test ordered by an MLS in the LIS.\n\n\n3\n\nOpiate interpretation entry\n\nFor these cases, the opiate assay results have been reviewed and an interpretation has been finalized by an attending pathologist. The interpretation of these cases requires a review by an MLS to ensure that there are no expected compounds or medications that were not mentioned in the interpretation.\n\n\n4\n\nSG requests\n\nThis tab holds a list of cases which has been marked by an attending pathologist for a request to MLSs to conduct a specific gravity (SG) test. This test will not be billed or appear in the LIS.\n\n\n5\n\nMLS problems\n\nThese cases have been marked as \u201cProblem for MLS\u201d by a trainee or attending. The comment section will indicate more about the problem. These cases require a review by MLSs.\n\n\n6\n\nUPDRS review\n\nThese cases require a review of the immunoassay results and medications for the patient to be collected. Once complete, a decision to either order reflex tests or make an interpretation needs to be made.\n\n\n7\n\nUOPIAC review\n\nThese cases require a review of the final opiate LCMS confirmation assay results and medications for the patient to be collected, potentially, based on the initial order. Once complete, a decision to either order reflex tests or make an interpretation needs to be made.\n\n\n8\n\nDirector problem\n\nThese cases have been marked as \u201cProblem for Director\u201d by a trainee or MLS. The comment section will indicate more about the problem. These cases require an additional review or resolution by the attending pathologist(s).\n\n\nPerformance measurement and user feedback \nSeveral new cases are ordered every day. In 2020, there were roughly 150 interpretations generated per week. Having dedicated queues for each step in the workflow helped staff and pathologist coordinate the work. Also, the queues helped pathologists prioritize and schedule the work based on the amount of time each pathologist had at a given time.\nTo investigate the return on investment, we surveyed the users on the amount of time each step in the workflow took to complete both before and after the application. The workflow no longer involved worksheets. Before the application, users reported they often spent one to two hours per day managing and combining files. This number came down to zero as they no longer had to use worksheets. Additionally, the introduction of the liquid handler to automatically prepare samples for the assay helped gain back an hour of time the MLS spent preparing the samples manually. The greatest return for staff was moving from manual result-entry to automated result-entry. The test is conducted five times per week and requires six days of staffing to process and manage data. Each batch requires the review of two laboratory staff members, and that practice continues but they would manually enter the results into the LIS. Staff reported they would spend around one hour per batch entering the results. This number came down to virtually zero as the result entry was automated via the flat file transfer to Data Innovations' Instrument Manager. For pathologists, they reported the interface was easier to navigate the multitude of values in a spreadsheet. The organization helped keep better track of cases and organize the workflow to be more efficient. On average, pathologists reported they have gained back two hours per batch since the application has been incorporated into the workflow. MLS staff reported they gained back four hours per batch since the application has been incorporated into the workflow.\nIn the 24 months prior to implementation of the application, there were 24,256 total opiate mass spectrometry test orders, and 449 orders had at least one data entry modification (defined as at least one test component having a result entered more than once), for a modification rate of 1.9%. After implementation, over 21 months there were 18,112 total opiate orders and only 10 orders had at least one data entry modification, yielding a modification rate of 0.06%.\n\nDiscussion \nWe have developed an application that centralizes data for UDT interpretation. Moreover, the automatic aggregation of this data provides an efficient and convenient method for pathologists to review the data to generate an interpretation of the results for clinicians and providers to review with their patient's undergoing opioid therapy. There are various groups that have developed novel web-based applications to help increase efficiency.[13][14] Groups have reported that their applications have reduced errors, expenses, staff burnout, and increased the level of quality patient care. Laboratory staff were able to gain back several hours from what they would have spent manually entering results and managing spreadsheets. After the application was introduced, the staff scheduled to the opiate bench was reduced from two full-time employees, six days a week, to one full-time employee, six days a week. Often the reviewing laboratory staff has worked in the department for some time before training for this work. Freeing an experienced laboratory member's time allowed them to help in other critical areas in the laboratory. Additionally, since the transfer process was fully automated, it reduced the number of errors transmitted into the LIS and time spent contacting the provider who may have already reviewed the results. Mays and Mathias[15] found a 3.7% error rate of manual transcription versus auto-entry. In our analysis, 1.8% of orders required a modification prior to implementation, and post-implementation this rate was reduced significantly to 0.05%. While it is likely that some modifications were made within the application as multiple team members composed and reviewed results prior to release, misinterpretation or confusion could be avoided by preventing information that needed modification from being entered into the LIS and seen by the provider or patient.\nThe informatics team dedicated substantial effort through a year and a half timeframe to develop and implement the application. The team included one member responsible for interviewing the stakeholders and documenting the requirements; a faculty member who designed the software architecture and database schema, and also developed the application prototype and deployment infrastructure; two software developers who worked together to complete the application; one software developer who helped interface the LIS to gather the necessary patient and order information; and one LIS specialist to help configure the LIS for auto-filing the results. Team members had specific tasks, which could have been worked upon in parallel, although many tasks relied on the completion of a task by a separate team member. Additionally, no member dedicated their time solely to this project during the project span. Each member balanced their time across several projects. The overall estimated time spent by the team was roughly 350\u202fhours over 18 months. The application was developed by departmental programming staff, one of whom was hired specifically to support the chemistry division, while the others were members of a departmental pool of informatics resources. Having a departmental informatics team that can develop novel solutions to complex, clinical problems require considerable resources, but there are also a variety of opportunities across units of the laboratory that benefit from high efficiency.\nAfter its initial launch, updates and enhancements were made to the application. The enhancements included bug fixes and additional features that were requested from users after working with the application for some time. The amount of time was roughly 130\u202fhours over 18 months. The amount of time spent in developing and enhancing the application was significant. However, introducing the application to the workflow freed up a full-time laboratory employee\u2019s time. Moreover, the laboratory has been able to increase the sample volume with no increased need in MLS staffing.\nThe return on investment (ROI) for the laboratory was significant. We demonstrated that the primary benefit to implementing a custom application has been a profound improvement in workflow and staff efficiency. The application removed mundane and repetitive tasks such as transferring data from one spreadsheet to the another. Additionally, it also removed the need for laboratory staff to fully complete the batch review before a reviewing pathologist could begin their review. With the application, the pathologist could begin as soon as the first patient in the batch was reviewed by the laboratory staff member. Additionally, the process of auto-filing results could be extended to other tests within the department, freeing some additional time for staff. This provides evidence to the value of developing and implementing custom applications.\nA consideration that groups should make when developing custom applications is the time required to understand the problem. For this project, we spent several hours interviewing stakeholders to understand the workflow and the different needs of each personnel. Once requirements were gathered, a plan needed to be discussed and agreed upon by members of both the informatics and chemistry departments. \nAnother challenge groups should consider is the potential requirement to rewrite software programs and repeat validation with upgrades to the software on top of any tests needed to be conducted to ensure the integrity and accuracy of a custom application. For the initial validation of the application, the informatics and chemistry teams discussed and agreed upon a validation plan. The plan involved laboratory members testing the application\u2019s functionality by working through the application workflow with test cases and matching the tested output with expected output. Additionally, groups should consider the availability of the development team to support their custom application. The need for support could stem from latency of servers due to traffic, system downtime, or undiscovered bugs that affect the behavior of the application. For these reasons, a support plan is recommended that is distributed to both users and supporting staff so that users know who to contact and IT staff understand how to effectively address the problem. The support plan may also require that laboratory staff keep up to date with manual result entry processes in case electronic systems undergo an unexpected downtime for a prolonged period. Lastly, the need to be cognizant of protected health information (PHI)-related IT security increases the need of IT involvement for the implementation and ongoing support of custom applications.\nIn our laboratory, the opiate LCMS confirmation assay is one of the most labor-intensive assays because of the sample preparation method and data review steps. This work laid the foundation to move other LCMS assays to automation. As this trend continues, our staff will be able to spend their time reviewing critical issues and tasks, further increasing the efficiency in the laboratory.\n\n Abbreviations, acronyms, and initialisms \nAWS: Amazon Web Services\nCSV: comma-separated values\nEHR: electronic health record\nGC\u2013MS: gas chromatography\u2013mass spectrometry\nLC-MS\/MS: liquid chromatography\u2013tandem mass spectrometry\nLCMS: liquid chromatography\u2013mass spectrometry\nLIS: laboratory information system\nMLS: medical laboratory scientist\nQC: quality control\nROI: return on investment\nTSV: tab-separated values\nUDT: urine drug test\nUI: user interface\nXML: extensible markup language\nAppendix A. Supplementary data \nSupplementary material (.docx)\nAcknowledgements \nThe authors would like to acknowledge the time and effort spent by the medical laboratory scientists of the chemistry laboratories at the University of Washington Medical Center and Harborview Medical Center in performing high quality mass spectrometry testing to serve their patients.\n\nConflict of interest \nThe authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.\n\nReferences \n\n\n\u2191 1.0 1.1 Owen, Graves T.; Burton, Allen W.; Schade, Cristy M.; Passik, Steve (1 July 2012). \"Urine drug testing: current recommendations and best practices\". Pain Physician 15 (3 Suppl): ES119\u2013133. ISSN 2150-1149. PMID 22786451. https:\/\/pubmed.ncbi.nlm.nih.gov\/22786451 .   \n \n\n\u2191 Pesce, Amadeo; West, Cameron; Egan City, Kathy; Strickland, Jennifer (1 July 2012). \"Interpretation of Urine Drug Testing in Pain Patients\" (in en). Pain Medicine 13 (7): 868\u2013885. doi:10.1111\/j.1526-4637.2012.01350.x. ISSN 1526-2375. https:\/\/academic.oup.com\/painmedicine\/article-lookup\/doi\/10.1111\/j.1526-4637.2012.01350.x .   \n \n\n\u2191 3.0 3.1 Dams, Riet; Murphy, Constance M.; Lambert, Willy E.; Huestis, Marilyn A. (30 July 2003). \"Urine drug testing for opioids, cocaine, and metabolites by direct injection liquid chromatography\/tandem mass spectrometry\" (in en). Rapid Communications in Mass Spectrometry 17 (14): 1665\u20131670. doi:10.1002\/rcm.1098. ISSN 0951-4198. https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/rcm.1098 .   \n \n\n\u2191 4.0 4.1 4.2 Coles, Rebecka; Kushnir, Mark M.; Nelson, Gordon J.; McMillin, Gwendolyn A.; Urry, Francis M. (1 January 2007). \"Simultaneous Determination of Codeine, Morphine, Hydrocodone, Hydromorphone, Oxycodone, and 6-Acetylmorphine in Urine, Serum, Plasma, Whole Blood, and Meconium by LC-MS-MS\" (in en). Journal of Analytical Toxicology 31 (1): 1\u201314. doi:10.1093\/jat\/31.1.1. ISSN 1945-2403. http:\/\/academic.oup.com\/jat\/article\/31\/1\/1\/780634\/Simultaneous-Determination-of-Codeine-Morphine .   \n \n\n\u2191 Dickerson, J. A.; Laha, T. J.; Pagano, M. B.; O'Donnell, B. R.; Hoofnagle, A. N. (1 October 2012). \"Improved Detection of Opioid Use in Chronic Pain Patients through Monitoring of Opioid Glucuronides in Urine\" (in en). Journal of Analytical Toxicology 36 (8): 541\u2013547. doi:10.1093\/jat\/bks063. ISSN 0146-4760. https:\/\/academic.oup.com\/jat\/article-lookup\/doi\/10.1093\/jat\/bks063 .   \n \n\n\u2191 Yang, Yifei K; Johnson-Davis, Kamisha L; Kelly, Brian N; McMillin, Gwendolyn A (1 September 2020). \"Demand for Interpretation of a Urine Drug Testing Panel Reflects the Changing Landscape of Clinical Needs; Opportunities for the Laboratory to Provide Added Clinical Value\" (in en). The Journal of Applied Laboratory Medicine 5 (5): 858\u2013868. doi:10.1093\/jalm\/jfaa119. ISSN 2576-9456. https:\/\/academic.oup.com\/jalm\/article\/5\/5\/858\/5900440 .   \n \n\n\u2191 7.0 7.1 Reisfield, MD, Gary M.; Webb, PhD, Fern J.; Bertholf, PhD, Roger L.; Sloan, MD, Paul A.; Wilson, MD, George R. (1 November 2007). \"Family physicians\u2019 proficiency in urine drug test interpretation\". Journal of Opioid Management 3 (6): 333\u2013337. doi:10.5055\/jom.2007.0022. ISSN 1551-7489. https:\/\/wmpllc.org\/ojs\/index.php\/jom\/article\/view\/1034 .   \n \n\n\u2191 West, MS, Robert; Pesce, PhD, DABCC, Amadeo; West, PhD, Cameron; Crews, PhD, Bridgit; Mikel, PhD, Charles; Rosenthal, DO, FAPA, Murray; Almazan, CLS, MT (ASCP), Perla; Latyshev, MS, Sergey (29 January 2018). \"Observations of medication compliance by measurement of urinary drug concentrations in a pain management population\". Journal of Opioid Management 6 (4): 253\u2013257. doi:10.5055\/jom.2010.0023. ISSN 1551-7489. https:\/\/wmpllc.org\/ojs\/index.php\/jom\/article\/view\/894 .   \n \n\n\u2191 Levy, Sharon; Sherritt, Lon; Vaughan, Brigid L.; Germak, Matthew; Knight, John R. (1 April 2007). \"Results of Random Drug Testing in an Adolescent Substance Abuse Program\" (in en). Pediatrics 119 (4): e843\u2013e848. doi:10.1542\/peds.2006-2278. ISSN 0031-4005. https:\/\/publications.aap.org\/pediatrics\/article\/119\/4\/e843\/70141\/Results-of-Random-Drug-Testing-in-an-Adolescent .   \n \n\n\u2191 Yang, Yifei K; Johnson-Davis, Kamisha L; Kelly, Brian N; McMillin, Gwendolyn A (1 September 2020). \"Demand for Interpretation of a Urine Drug Testing Panel Reflects the Changing Landscape of Clinical Needs; Opportunities for the Laboratory to Provide Added Clinical Value\" (in en). The Journal of Applied Laboratory Medicine 5 (5): 858\u2013868. doi:10.1093\/jalm\/jfaa119. ISSN 2576-9456. https:\/\/academic.oup.com\/jalm\/article\/5\/5\/858\/5900440 .   \n \n\n\u2191 11.0 11.1 Dickerson, Jane A.; Schmeling, Michael; Hoofnagle, Andrew N.; Hoffman, Noah G. (1 January 2013). \"Design and implementation of software for automated quality control and data analysis for a complex LC\/MS\/MS assay for urine opiates and metabolites\" (in en). Clinica Chimica Acta 415: 290\u2013294. doi:10.1016\/j.cca.2012.10.055. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S000989811200527X .   \n \n\n\u2191 \"Flask\". Pallets. 2010. https:\/\/flask.palletsprojects.com\/en\/2.3.x\/ .   \n \n\n\u2191 Woo, Jennifer S.; Suslow, Peter; Thorsen, Russell; Ma, Rosaline; Bakhtary, Sara; Moayeri, Morvarid; Nambiar, Ashok (1 January 2019). \"Development and Implementation of Real-Time Web-Based Dashboards in a Multisite Transfusion Service\" (in en). Journal of Pathology Informatics 10 (1): 3. doi:10.4103\/jpi.jpi_36_18. PMC PMC6396429. PMID 30915257. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2153353922003662 .   \n \n\n\u2191 Smith, Matthew A.; Roy, Somak; Nestler, Rick; Augustine, Beth; Miller, David; Parwani, Anil; Nichols, Lawrence (1 January 2011). \"University of Pittsburgh Medical Center Remains Tracker: A novel application for tracking decedents and improving the autopsy workflow\" (in en). Journal of Pathology Informatics 2 (1): 30. doi:10.4103\/2153-3539.82055. PMC PMC3132995. PMID 21773061. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2153353922002061 .   \n \n\n\u2191 Mays, James A; Mathias, Patrick C (1 March 2019). \"Measuring the rate of manual transcription error in outpatient point-of-care testing\" (in en). Journal of the American Medical Informatics Association 26 (3): 269\u2013272. doi:10.1093\/jamia\/ocy170. ISSN 1067-5027. PMC PMC6351970. PMID 30649499. https:\/\/academic.oup.com\/jamia\/article\/26\/3\/269\/5287977 .   \n \n\n\nNotes \nThis presentation is faithful to the original, with only a few minor changes to presentation, spelling, and grammar. In some cases important information was missing from the references, and that information was added.\n\n\n\n\n\nSource: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.limswiki.org\/index.php\/Journal:A_web_application_to_support_the_coordination_of_reflexive,_interpretative_toxicology_testing\">https:\/\/www.limswiki.org\/index.php\/Journal:A_web_application_to_support_the_coordination_of_reflexive,_interpretative_toxicology_testing<\/a>\nCategories: LIMSwiki journal articles (added in 2023)LIMSwiki journal articles (all)LIMSwiki journal articles on clinical informaticsLIMSwiki journal articles on pathology informaticsLIMSwiki journal articles on laboratory managementNavigation menuPage actionsJournalDiscussionView sourceHistoryPage actionsJournalDiscussionMoreToolsIn other languagesPersonal toolsLog inNavigationMain pageEncyclopedic articlesRecent changesRandom pageHelp about MediaWikiSearch\u00a0 ToolsWhat links hereRelated changesSpecial pagesPermanent linkPage informationPopular publications\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\nPrint\/exportCreate a bookDownload as PDFDownload as PDFDownload as Plain textPrintable version This page was last edited on 13 June 2023, at 23:28.Content is available under a Creative Commons Attribution-ShareAlike 4.0 International License unless otherwise noted.This page has been accessed 653 times.Privacy policyAbout LIMSWikiDisclaimers\n\n\n\n","b283f4f5c78061a91491bc16c8576d36_html":"<body class=\"mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-206 ns-subject page-Journal_A_web_application_to_support_the_coordination_of_reflexive_interpretative_toxicology_testing rootpage-Journal_A_web_application_to_support_the_coordination_of_reflexive_interpretative_toxicology_testing skin-monobook action-view skin--responsive\"><div id=\"rdp-ebb-globalWrapper\"><div id=\"rdp-ebb-column-content\"><div id=\"rdp-ebb-content\" class=\"mw-body\" role=\"main\"><a id=\"rdp-ebb-top\"><\/a>\n<h1 id=\"rdp-ebb-firstHeading\" class=\"firstHeading\" lang=\"en\">Journal:A web application to support the coordination of reflexive, interpretative toxicology testing<\/h1><div id=\"rdp-ebb-bodyContent\" class=\"mw-body-content\"><!-- start content --><div id=\"rdp-ebb-mw-content-text\" lang=\"en\" dir=\"ltr\" class=\"mw-content-ltr\"><div class=\"mw-parser-output\">\n\n\n<h2><span class=\"mw-headline\" id=\"Abstract\">Abstract<\/span><\/h2>\n<p><b>Background<\/b>: <a href=\"https:\/\/www.limswiki.org\/index.php\/Reflex_test\" title=\"Reflex test\" class=\"wiki-link\" data-key=\"24baf904d449f87cf2229f90b5e94f6f\">Reflexive laboratory testing<\/a> <a href=\"https:\/\/www.limswiki.org\/index.php\/Workflow\" title=\"Workflow\" class=\"wiki-link\" data-key=\"92bd8748272e20d891008dcb8243e8a8\">workflows<\/a> can improve the assessment of patients receiving pain medications chronically, but complex workflows requiring <a href=\"https:\/\/www.limswiki.org\/index.php\/Pathology\" title=\"Pathology\" class=\"wiki-link\" data-key=\"5d8e2230b55d8760dcb729fc7dcd3dc1\">pathologist<\/a> input and interpretation may not be well-supported by traditional <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory_information_system\" title=\"Laboratory information system\" class=\"wiki-link\" data-key=\"37add65b4d1c678b382a7d4817a9cf64\">laboratory information systems<\/a> (LISs). In this work, we describe the development of a web application that improves the efficiency of pathologists and <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory\" title=\"Laboratory\" class=\"wiki-link\" data-key=\"c57fc5aac9e4abf31dccae81df664c33\">laboratory<\/a> staff in delivering actionable <a href=\"https:\/\/www.limswiki.org\/index.php\/Toxicology\" title=\"Toxicology\" class=\"wiki-link\" data-key=\"0c1c9e1a7e33df53c6ca721eee8b5381\">toxicology<\/a> results.\n<\/p><p><b>Method<\/b>: Before designing the application, we set out to understand the entire workflow, including the laboratory workflow and pathologist review. Additionally, we gathered requirements and specifications from stakeholders. Finally, to assess the performance of the implementation of the application, we surveyed stakeholders and documented the approximate amount of time that is required in each step of the workflow.\n<\/p><p><b>Results<\/b>: A web-based application was chosen for the ease of access for users. Relevant clinical data was routinely received and displayed in the application. The workflows in the laboratory and during the interpretation process served as the basis of the user interface (UI). With the addition of auto-filing software, the return on investment (ROI) was significant. The laboratory saved the equivalent of one full-time employee in time by automating file management and result entry.\n<\/p><p><b>Discussion<\/b>: Implementation of a purpose-built application to support reflex and interpretation workflows in a clinical pathology practice has led to a significant improvement in laboratory efficiency. Custom- and purpose-built applications can help reduce staff burnout, reduce transcription errors, and allow staff to focus on more critical issues around <a href=\"https:\/\/www.limswiki.org\/index.php\/Quality_(business)\" title=\"Quality (business)\" class=\"wiki-link\" data-key=\"c4ac43430d1c3a3a15d1255257aaea37\">quality<\/a>.\n<\/p><p><b>Keywords<\/b>: Python, laboratory workflows, custom web application, quality control, mass spectrometry\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Background\">Background<\/span><\/h2>\n<p>Urine drug monitoring is an increasingly important tool for primary care physicians and pain specialists to monitor patients on chronic opioid therapy.<sup id=\"rdp-ebb-cite_ref-:0_1-0\" class=\"reference\"><a href=\"#cite_note-:0-1\">[1]<\/a><\/sup> Routine and random monitoring for all patients on long-term opioid therapy is recommended prior to starting and throughout drug therapy.<sup id=\"rdp-ebb-cite_ref-2\" class=\"reference\"><a href=\"#cite_note-2\">[2]<\/a><\/sup> There are various analytical methods and matrices available for patient monitoring. Urine <a href=\"https:\/\/www.limswiki.org\/index.php\/Drug_test\" title=\"Drug test\" class=\"wiki-link\" data-key=\"a77e286e4af95b13abc7acf73113e96b\">drug testing<\/a> (UDT) has become the \u201cgold-standard\u201d for detecting illicit drug use and monitoring ongoing therapy. Urine is non-invasive, readily available, and contains higher concentrations of drugs and metabolites compared with other matrices. <a href=\"https:\/\/www.limswiki.org\/index.php\/Immunoassay\" title=\"Immunoassay\" class=\"wiki-link\" data-key=\"ce415f577070e6fa6afa0a305f5f0247\">Immunoassay<\/a> testing, commonly referred to as urine drug screening, detects the presence of selected drugs and\/or metabolites based on a detection threshold. Typically, the immunoassay functions as the initial evaluation for the potential detection of drugs, but because it lacks specificity for analytes of the same drug class, it can lead to false-positive and -negative results. Therefore, drug screens are often coupled with a confirmatory detection method for more specific testing. <a href=\"https:\/\/www.limswiki.org\/index.php\/Chromatography\" title=\"Chromatography\" class=\"wiki-link\" data-key=\"2615535d1f14c6cffdfad7285999ad9d\">Chromatography<\/a> is generally reserved for confirmatory or definitive testing following immunoassay screening. Historically, <a href=\"https:\/\/www.limswiki.org\/index.php\/Gas_chromatography%E2%80%93mass_spectrometry\" title=\"Gas chromatography\u2013mass spectrometry\" class=\"wiki-link\" data-key=\"d7fe02050f81fca3ad7a5845b1879ae2\">gas chromatography\u2013mass spectrometry<\/a> (GC\u2013MS) was the standard for confirmatory testing. However, <a href=\"https:\/\/www.limswiki.org\/index.php\/Liquid_chromatography%E2%80%93mass_spectrometry\" title=\"Liquid chromatography\u2013mass spectrometry\" class=\"wiki-link\" data-key=\"d171745b38c8d2ed7d274d2cc13fa1f3\">liquid chromatography\u2013mass spectrometry<\/a>\u2013<a href=\"https:\/\/www.limswiki.org\/index.php\/Tandem_mass_spectrometry\" title=\"Tandem mass spectrometry\" class=\"wiki-link\" data-key=\"55f167a11d8b5037392ba845986bf6bf\">tandem mass spectrometry<\/a> (LC-MS\/MS) has gained favor over GC-MS due to reduced complexity of <a href=\"https:\/\/www.limswiki.org\/index.php\/Sample_(material)\" title=\"Sample (material)\" class=\"wiki-link\" data-key=\"7f8cd41a077a88d02370c02a3ba3d9d6\">sample<\/a> preparation methods and the broader applicability of LC-MS\/MS to different drug classes.<sup id=\"rdp-ebb-cite_ref-:1_3-0\" class=\"reference\"><a href=\"#cite_note-:1-3\">[3]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:2_4-0\" class=\"reference\"><a href=\"#cite_note-:2-4\">[4]<\/a><\/sup>\n<\/p><p>Many <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory\" title=\"Laboratory\" class=\"wiki-link\" data-key=\"c57fc5aac9e4abf31dccae81df664c33\">laboratories<\/a> are equipped to support UDT.<sup id=\"rdp-ebb-cite_ref-:0_1-1\" class=\"reference\"><a href=\"#cite_note-:0-1\">[1]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:1_3-1\" class=\"reference\"><a href=\"#cite_note-:1-3\">[3]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:2_4-1\" class=\"reference\"><a href=\"#cite_note-:2-4\">[4]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-5\" class=\"reference\"><a href=\"#cite_note-5\">[5]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-6\" class=\"reference\"><a href=\"#cite_note-6\">[6]<\/a><\/sup> A major challenge of staged drug testing (i.e., screening and confirmatory assays) is the review and interpretation of the large number of quantitative and qualitative results that is generated by screening and confirmatory methods.<sup id=\"rdp-ebb-cite_ref-:3_7-0\" class=\"reference\"><a href=\"#cite_note-:3-7\">[7]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-8\" class=\"reference\"><a href=\"#cite_note-8\">[8]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-9\" class=\"reference\"><a href=\"#cite_note-9\">[9]<\/a><\/sup> Pharmacokinetic mechanisms dictate how quickly and how much of a drug and its metabolites appear in an individual's urine, which can be challenging for physicians who have limited understanding due to reviewing a limited number of these results.<sup id=\"rdp-ebb-cite_ref-:3_7-1\" class=\"reference\"><a href=\"#cite_note-:3-7\">[7]<\/a><\/sup> An incorrect interpretation of the LC-MS\/MS results could mistakenly suggest that a patient used a non-prescribed medication. Additionally, the potential for <a href=\"https:\/\/www.limswiki.org\/index.php\/Analyte\" title=\"Analyte\" class=\"wiki-link\" data-key=\"350c296d21767567726aa2675021de30\">analytes<\/a> to co-elute could affect the signal of the analytes, influencing the overall accuracy of the LC-MS\/MS test.<sup id=\"rdp-ebb-cite_ref-:2_4-2\" class=\"reference\"><a href=\"#cite_note-:2-4\">[4]<\/a><\/sup> \n<\/p><p>Yang <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-10\" class=\"reference\"><a href=\"#cite_note-10\">[10]<\/a><\/sup> described that adding interpretative comments from a trained laboratory director to urine drug screening results can greatly impact both the clinician and patient. Clinicians may not understand the analytical method limitations, and any resulting misinterpretation of the results may lead to a patient's stigmatization or inappropriate termination from care. The authors further reported an increase in urine drug screen orders with interpretive comments, indicating that clinicians value the interpretation made by a trained director. Because accurate interpretation of results often requires the integration of documented clinical information, medication data, test results, pharmacokinetics, and test limitations and interferences, assigning a trained <a href=\"https:\/\/www.limswiki.org\/index.php\/Pathology\" title=\"Pathology\" class=\"wiki-link\" data-key=\"5d8e2230b55d8760dcb729fc7dcd3dc1\">pathologist<\/a> or <a href=\"https:\/\/www.limswiki.org\/index.php\/Clinical_chemistry\" title=\"Clinical chemistry\" class=\"wiki-link\" data-key=\"184d3433dd1f9dba149f42bc82234b8d\">clinical chemist<\/a> who is familiar with interpreting clinical <a href=\"https:\/\/www.limswiki.org\/index.php\/Information\" title=\"Information\" class=\"wiki-link\" data-key=\"6300a14d9c2776dcca0999b5ed940e7d\">information<\/a>, as well as the biochemistry and testing methods to provide an interpretation of the results, is a valuable tool in ensuring the correct diagnosis for patients.\n<\/p><p>One challenge laboratories and laboratory directors face with UDT is that there may be results from several tests that inform whether additional testing is required and need to be integrated to provide an interpretation given the specific clinical context. Within the Department of Laboratory Medicine and Pathology at University of Washington Medicine (UW Medicine), we have formulated a urine drug testing ordering protocol and interpretation <a href=\"https:\/\/www.limswiki.org\/index.php\/Workflow\" title=\"Workflow\" class=\"wiki-link\" data-key=\"92bd8748272e20d891008dcb8243e8a8\">workflow<\/a>. Clinicians may select from a menu of panels that are based on the assessed patient's risk of treatment non-compliance. Panels include an <a href=\"https:\/\/www.limswiki.org\/index.php\/Immunoassay\" title=\"Immunoassay\" class=\"wiki-link\" data-key=\"ce415f577070e6fa6afa0a305f5f0247\">immunoassay<\/a> drug screen and confirmation LC-MS\/MS assays for opioids, amphetamines, benzodiazepines, and alcohol. For example, a low-risk patient panel consists of an immunoassay drug screen, and aberrant or unexpected results can lead the reviewing pathologist to order one or more of the confirmation tests. On top of the interpretation challenges, turnaround times (TAT) for the different tests are unevenly distributed. Most <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory_information_system\" title=\"Laboratory information system\" class=\"wiki-link\" data-key=\"37add65b4d1c678b382a7d4817a9cf64\">laboratory information systems<\/a> (LISs) are not equipped to track, collect, and display the different results in an effective and informative manner to support this workflow.\n<\/p><p>The aim of this project was to design and implement a web-based software application that centralizes test results for pathologist review, performs the <a href=\"https:\/\/www.limswiki.org\/index.php\/Quality_control\" title=\"Quality control\" class=\"wiki-link\" data-key=\"1e0e0c2eb3e45aff02f5d61799821f0f\">quality control<\/a> (QC) calculations for the complex LC-MS\/MS analysis of opioids and their metabolites, functions as a user input form for pathologist interpretation entry, and auto-files results into the LIS.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Methods\">Methods<\/span><\/h2>\n<p>The initial step before designing the application was to understand the entire workflow, then gather requirements and specifications from stakeholders.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Pre-application_testing_and_workflows\">Pre-application testing and workflows<\/span><\/h3>\n<p>At UW Medicine, for each patient specimen undergoing the opiate confirmation assay, two extractions are prepared, an undiluted and a diluted 10-fold urine sample. To help improve the complex LC-MS\/MS <a href=\"https:\/\/www.limswiki.org\/index.php\/Data_analysis\" title=\"Data analysis\" class=\"wiki-link\" data-key=\"545c95e40ca67c9e63cd0a16042a5bd1\">data analysis<\/a> of opioids and opioid metabolites, a command-line software application was developed.<sup id=\"rdp-ebb-cite_ref-:4_11-0\" class=\"reference\"><a href=\"#cite_note-:4-11\">[11]<\/a><\/sup> The software application known as \u201cSMACK\u201d was shown to improve the data analysis process by automating a QC algorithm and calculations to improve consistency of analysis as well as reduce the amount of time medical laboratory scientists (MLS) spent reviewing the data.\n<\/p><p>The algorithm was part of an intricate workflow that involved several steps, spreadsheets, and individuals. The goal of the spreadsheets was to collect and consolidate the relevant patient information so that the reviewing pathologist had the necessary information to generate an interpretation for the overall case. The information with the relevant patient data was spread across several spreadsheets, the creation of which relied on laboratory staff and pathologists copying and pasting various fields from one spreadsheet to another. Pathologists and laboratory staff communicated the completion of steps via email while using a naming convention for the relevant files.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Requirements\">Requirements<\/span><\/h3>\n<p>We set to gather information on how clinicians ordered the different drug monitoring tests and how the information arrived at the laboratory. We investigated the LIS data workflow that was in place and how this information could feed the application, including assay results and patient demographics. Additionally, we worked with the laboratory to understand how test results were produced and how this information should be included in the application. Lastly, we investigated what the requirements were for progressing from one step of the workflow to the next along with the data associated with the progression. In addition to the workflow in the laboratory, we needed to understand what data elements are needed for each step of the workflow. With this information, we would create a map of the workflow to visualize and better understand the workflow. We gathered necessary components from stakeholders, pathologists, directors, pathologists in training, and laboratory staff by interviewing stakeholders and observing workflows. During this process, we gathered user interface (UI) requirements from stakeholders to shape functionality of the optimized web application.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Technical_requirements\">Technical requirements<\/span><\/h3>\n<p>The previously described QC software SMACK had been implemented as a command-line application in <a href=\"https:\/\/www.limswiki.org\/index.php\/Python_(programming_language)\" title=\"Python (programming language)\" class=\"wiki-link\" data-key=\"ef6905a29cbb75d3c71e6bdf6e2915dd\">Python<\/a>, with no dependencies outside of the standard library.<sup id=\"rdp-ebb-cite_ref-:4_11-1\" class=\"reference\"><a href=\"#cite_note-:4-11\">[11]<\/a><\/sup> Input data is provided as an <a href=\"https:\/\/www.limswiki.org\/index.php\/Extensible_Markup_Language\" title=\"Extensible Markup Language\" class=\"wiki-link\" data-key=\"f7c17028e7fb39d8b39c6d31504411a8\">XML<\/a> format file exported from <a href=\"https:\/\/www.limswiki.org\/index.php\/Waters_Corporation\" title=\"Waters Corporation\" class=\"wiki-link\" data-key=\"d17153eab93be07066660e3cff4b84c5\">Waters<\/a>' TargetLynx software. The outputs of the software are two CSV-formatted reports, one containing assay results and the other a detailed report of the QC calculations. Appendix A, Supplemental Fig. 1 displays the flow chart defining the QC algorithm, which has been simplified since its initial implementation. For this project, we needed to incorporate the software into the application to perform the same QC calculations on the opiate assay raw data and then display the output to the user in a meaningful manner. Moreover, our group has a defined technical software stack for developing and deploying applications in Python. For these reasons, writing the application in Python was the most efficient choice. An additional requirement was to automatically file results into the LIS (from <a href=\"https:\/\/www.limswiki.org\/index.php\/CliniSys,_Inc.\" title=\"CliniSys, Inc.\" class=\"wiki-link\" data-key=\"20889b2bc3d70218588a6c3be442c648\">CliniSys, Inc.<\/a>). To achieve this requirement, we had to work with Data Innovations' Instrument Manager software.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Performance_measurement_and_user_feedback\">Performance measurement and user feedback<\/span><\/h3>\n<p>Finally, to assess the performance, we surveyed stakeholders and documented the rough amount of time that is required in each step of the workflow. Input was solicited from several laboratory staff members rotating through the area regarding how much time was spent on each step before and after the implementation of the application. We also surveyed the pathologists in training (chemistry fellows and rotating residents), known as trainees. Finally, we extracted data from the LIS to capture the number of occurrences in which a testing order had a correction or modification after the initial data entry so we could compare correction rates prior to and after implementation.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Results\">Results<\/span><\/h2>\n<h3><span class=\"mw-headline\" id=\"Laboratory_workflows_prior_to_implementation\">Laboratory workflows prior to implementation<\/span><\/h3>\n<p>Providers order UDTs for the patient based on an assessment of the patient's risk for compliance. Table 1 displays the patient risk levels, the intended population, and the tests associated with each risk panel.\n<\/p>\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"3\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 1.<\/b> Urine drug testing ordering strategy based on risk. Providers assess risk, review clinical patient care data, and order tests as described.\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Risk\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Intended population\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Tests\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Low\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Morphine equivalent dose <90\u202fmg\/day; Stable on chronic opioid therapy or buprenorphine therapy\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Drug screen immunoassay\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Medium to high\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Morphine equivalent dose >=90\u202fmg\/day; Aberrant behavior (including early prescriptions, lost prescriptions, outbursts in office)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Drug screen immunoassay; Enzymatic assay for alcohol; Opiate confirmation LCMS\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">High\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Same as medium to high; added concern for clonazepam or lorazepam\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Drug screen immunoassay; Enzymatic assay for alcohol; Opiate confirmation LCMS; Benzodiazepine confirmation LCMS\n<\/td><\/tr>\n<\/tbody><\/table>\n<p>For each panel, providers have the option to enter any drug the patient is expected to be taking which may influence the test results. Each test in the panel is conducted separately and the results are entered into the LIS. Once all the tests have been completed and results entered, the pathologist may begin generating an interpretation.\n<\/p><p>When pathologists are reviewing the results, they consider the patient's history in the <a href=\"https:\/\/www.limswiki.org\/index.php\/Electronic_health_record\" title=\"Electronic health record\" class=\"wiki-link\" data-key=\"f2e31a73217185bb01389404c1fd5255\">electronic health records<\/a> (EHRs) for information that may influence the test results such as medications not specified by the ordering provider. These notes would be entered in dedicated fields in the spreadsheet. For training purposes, a pathologist trainee will often review the results first, then review their work with a director who may discuss the case, edit the preliminary interpretation, then make the final approval. Upon review of cases that initially only received the immunoassay drug screen, a probable outcome is for the reviewing pathologist to deem additional confirmation testing necessary. Therefore, the workflow is the summation of two separate parts, cases which receive opiate confirmation testing versus those that do not. The consideration to separate the workflow into two parts was partly because many orders require additional reflexive orders which influences how quickly the final interpretations will be released. For this reason, it was best to group cases by risk level to provide an interpretation as quickly as possible. Laboratory staff worked with departmental software engineers to create and deliver spreadsheets for each workflow which contain the patient's demographics (medical record number, age, sex, name, and order number) along with the results of the tests.\n<\/p><p>The opiate assay is performed on a Waters Xevo TQ-MS tandem mass spectrometer with an Aquity UPLC system. Data acquisition is controlled by Waters MassLynx and chromatography review is completed using Waters TargetLynx software. When the data acquisition is complete, the technologist completes a cursory review of the auto-integration using TargetLynx. Once the preliminary review is complete, an XML file with the raw data is created. The XML file is the input of the application that performs QC calculations from the LC-MS\/MS raw data. The output was two detailed tabular reports; the first was the calculated results after the algorithm was applied and another with a detailed report of the QC calculations. These reports would be reviewed in Microsoft Excel by two laboratory staff members to address any values that did not meet QC standards before entering the results into the LIS. Once the results have been reviewed, staff would send an email to the reviewing pathologists that reports were ready for their review. The pathologist would then combine the calculated results with the spreadsheet containing other test results and patient demographics via a copy-paste method. During the pathologist review, they may identify aberrant results stemming from interfering compounds or possible adulterations based on the data. They would then create an email thread with the laboratory to communicate and resolve the issue.\n<\/p><p>One possible outcome for those cases where patient risk levels are associated with low to medium risk, a reviewing pathologist orders a confirmatory test. Pathologists would type the test codes for the orders they wished to place in the spreadsheet and the reviewing MLS would place the orders in the LIS. Therefore, a third spreadsheet was created to capture the results for the cases that received additional tests. To transfer the notes captured during the initial review, a Microsoft Excel macro was created which would parse the spreadsheets and bring the pathologist's saved work from the initial spreadsheet into the final spreadsheet.\n<\/p><p>These spreadsheets were created and delivered via a dedicated website where staff and faculty were able to download the spreadsheets from any computer in the laboratory. After the spreadsheet was initially downloaded and edited, the file was stored on a shared drive accessible through a protected network that was accessible to the laboratory and pathologists.\n<\/p><p>A naming convention for these files was created to help indicate the stage in the workflow.\n<\/p><p>Once the reviewing pathologist had completed their review, they would send an email to staff so that they could enter the interpretation result into the LIS manually. If the technologist identified any issues with the interpretation, they would email the pathologist to edit\/review the interpretation.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Technical_requirements_2\">Technical requirements<\/span><\/h3>\n<p>The application was implemented in Python using the Flask web framework<sup id=\"rdp-ebb-cite_ref-FlaskProj_12-0\" class=\"reference\"><a href=\"#cite_note-FlaskProj-12\">[12]<\/a><\/sup> and other open-source libraries. Appendix A, Supplemental Table 1 lists the required libraries used for the application.\n<\/p><p>The software was developed on an Apple Macintosh running Mac OS 10.15 and is used in production on Amazon Web Services (AWS) server running Ubuntu (18.04 LTS). The data from the LIS is gathered using Cache Object Script that collects information every 30\u202fminutes from 6 a.m. to 6 p.m. and is delivered to a secure cloud storage resource (S3 bucket) on AWS. Once the results are finalized, the application sends a tab-delimited file to a transfer Linux (4.15.0) server which is picked up by Data Innovations instrument manager that sends the data to the LIS.\n<\/p><p>An important component of the software development and deployment process was the use of the Git version control software. Git is a free, open-source, distributed version control software that can be used to store \u201csnapshots\u201d of source code files in which modifications are captured to the code repository. Each change is attributed to a specific author and can be associated with a comment describing the intent of the change. A unique tag identifies each change, and the application captures the tag in the software version number and records this identifier to every data output. In this manner, results of each analysis can be traced to the exact version of the software used to collect and generate those results. Additionally, <a href=\"https:\/\/www.limswiki.org\/index.php\/GitLab\" title=\"GitLab\" class=\"wiki-link\" data-key=\"0c7fd1ce83146376fffb81cc79a92738\">GitLab<\/a> is a Git-based fully integrated platform which allows users to create \u201cissues\u201d where enhancements, development, and other topics may be discussed collectively. This feature of GitLab can also attribute labels to each issue so that larger topics or concepts can be collectively viewed and labeled.\n<\/p><p>The application is a web-based application that can be launched from any web browser. Access control is managed using our institutional identity provider with single sign-on with two-factor authentication. For auto-resulting, once the results reach their terminal status (pathologists and laboratory staff have completed their collective review), a tab-separated-values (TSV) is sent to Data Innovations' Instrument Manager where it files the results to the LIS.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Requirements_2\">Requirements<\/span><\/h3>\n<p>The data collected and workflows described serviced as the foundation of the application's design. The application needed to gather the necessary data from the LIS and display the information in a manner that is useful to the user. A system needed to be designed to progress a case through the necessary steps for the appropriate reviewer. Entry fields needed to be available for all members to be able to communicate problems, capture notes by the pathologists during their review, and for the final interpretation result. The application needed to ingest the XML file from TargetLynx of the opiate test, apply the SMACK algorithm, display both the QC calculations and results, and save all edits. Lastly, the application needed a system to communicate any additional confirmation tests the pathologist wanted to order.\n<\/p><p>A new component within the laboratory workflow was the introduction of an <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory_automation\" title=\"Laboratory automation\" class=\"wiki-link\" data-key=\"0061880849aeaca05f8aa27ae171f331\">automated liquid handler<\/a> to prepare the urine samples for the opiate assay. The laboratory had been manually preparing the samples and entering the patient identifiers into the acquisition system, MassLynx. When creating the extraction method of the liquid handler, the decision was made to use the <a href=\"https:\/\/www.limswiki.org\/index.php\/Barcode\" title=\"Barcode\" class=\"wiki-link\" data-key=\"e0952b5b262392be0995237aec36d355\">barcode<\/a> on the specimen label as the sample identifier for the assay. To associate the result back to the patient, all specimen IDs also needed to be gathered.\n<\/p><p>The decision to create a web-based application was to provide an easy-access entry point for the laboratory and pathologists. Also, a web-based application provided the flexibility needed to create a custom site that fit the needs of the workflow. Fig. 1 shows the home page of the application.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig1_Pablo_JofPathInfo2023_14.jpg\" class=\"image wiki-link\" data-key=\"68290b4066676f3c32bcd15c04e297c6\"><img alt=\"Fig1 Pablo JofPathInfo2023 14.jpg\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/f\/fd\/Fig1_Pablo_JofPathInfo2023_14.jpg\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 1.<\/b> Home page of the opiate sign-out application. The landing page lists all pending samples in the application along with sample information such as case number, container ID (LCMS), medical record number, patient name, and status of each test.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>We provided a tabular pending list that indicates the case status, order status, order number, LCMS results page, medical record number, patient name, and the status of each test. Cases were provided status IDs to progress each case through the different steps of the workflow. While investigating the overall workflows we determined that workflows could be categorized by the different stakeholders, the pathologists and the MLSs, and the different tasks associated with each personnel. From this, we created filters for the pending lists so that each user could easily jump into their associated workflow and collectively work on cases based on the case status (Fig. 2). We added a search bar so that a user could search any pending or historical case by name, order number, hospital ID, or specimen ID.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig2_Pablo_JofPathInfo2023_14.jpg\" class=\"image wiki-link\" data-key=\"450d8051a182a79ff2849004e40e2004\"><img alt=\"Fig2 Pablo JofPathInfo2023 14.jpg\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/4\/4f\/Fig2_Pablo_JofPathInfo2023_14.jpg\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 2.<\/b> General workflow outline. The status of each case drives the case from one step in the workflow to the next until the result is filed into the LIS.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>For each case, a new page is created where the patient demographics, test results, and entry fields are displayed. Fig. 3 shows what the case page looks like in the application for a given patient, and Table 2 lists the separate features displayed on the page.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig3_Pablo_JofPathInfo2023_14.jpg\" class=\"image wiki-link\" data-key=\"21e5bdc286cb67d8e7420773d84282eb\"><img alt=\"Fig3 Pablo JofPathInfo2023 14.jpg\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/1\/18\/Fig3_Pablo_JofPathInfo2023_14.jpg\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 3.<\/b> Case page example. The page displays the fields and layout of the case page of a given patient.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"3\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 2.<\/b> Listed case page features. Each listed feature is an object that supports the process of generating the test interpretation by a pathologist.\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Number\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Feature\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Description\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Pending case list\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">A list of all the pending cases for review\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Icon definitions\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">The key maps out the status symbols and their definitions\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Order information\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Patient and order information of the overall case\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">4\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Screening results\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">This card displays the results of the tests by test component. If a test is still pending or incomplete in the LIS, a gray \u201cpending\u201d box will be displayed in the \u201cTests\u201d column.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">5\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Medication list\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Entry field where medications found in the patient\u2019s chart are entered\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">6\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Chart reviewed\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">For billing purposes, pathologists indicate whether the patient\u2019s chart was reviewed during review, a value that is entered, as a result, into the LIS.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">7\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Reflex to\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">The pathologist will mark these boxes to indicate which additional tests will be ordered by the MLS in the LIS.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">8\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Request urine specific gravity\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">The pathologist will check this box for MLSs to perform a specific gravity test and they will populate the entry box with the result.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">9\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Case interpretation (UINTDS)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Entry field used to capture the pathologist generated interpretation\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">10\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Case status\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">This is a dropdown box used to communicate and move the case through the sign-out process. Statuses are changed after the person has completed their duties for the case.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">11\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Signoff check boxes\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">When a director checks this box, it will gather their name and associate it with the additional order or result for billing purposes.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">12\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Comment\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Entry field used to gather comments, questions, concerns regarding the overall case\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">13\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Save\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">This button will save all input and changes made to the page\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">14\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">History\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Historical changes and comments are displayed with the employee ID of the person who made the change and timestamp of the change.\n<\/td><\/tr>\n<\/tbody><\/table>\n<p>For orders where the opiate LCMS confirmation test was placed, a separate page is created known as the LCMS page. Fig. 4 displays the LCMS page for a given patient. This page holds much of the same information as the case page but with dedicated space for the result of the QC calculations. In this manner, we avoid cluttering a single page, and if a case did not have the opiate confirmation test ordered, the page only shows relevant information. When the LCMS page exists, the specimen ID is the basis of the URL of the LCMS page. This page duplicates many of the features displayed on the case page to help highlight key information and provide ease of access to resources for the reviewing pathologists.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig4_Pablo_JofPathInfo2023_14.jpg\" class=\"image wiki-link\" data-key=\"e5d7bc67f164548948754903fcfbb317\"><img alt=\"Fig4 Pablo JofPathInfo2023 14.jpg\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/b\/be\/Fig4_Pablo_JofPathInfo2023_14.jpg\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 4.<\/b> LCMS page example. The figure displays the several features of the LCMS page for a given patient including the output table of the SMACK QC calculations.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>When an XML file is uploaded to the application, the application applies the algorithm and automatically associates the result with a patient using the unique specimen ID. The result table can be viewed in one of three configurations:\n<\/p>\n<ul><li>1. Metabolite \u2013 Compounds are listed together by metabolic groups.<\/li>\n<li>2. Worksheet \u2013 Compounds are listed in the order seen in the LIS.<\/li>\n<li>3. Sequential \u2013 Compounds are listed in the order seen in TargetLynx.<\/li><\/ul>\n<p>The opiate confirmation assay is quantitative\/qualitative. Values that require review by the laboratory staff for QC purposes are highlighted in red. Positive values are highlighted in green. Either a positive numerical value is displayed for quantitative results or \u201cPOS\u201d is displayed for positive qualitative results. Negatives are displayed in black and denoted by \u201cNRN.\u201d\n<\/p><p>Once the interpretations have been created, the laboratory staff will review it for accuracy (ensuring that each drug\/analyte is mentioned) before submitting it to the LIS. If they identify a problem, they will change the status of the case to \u201cProblem for Director.\u201d This status captures all cases that require special attention by the attending pathologist, also known as the Director. Once the problem has been addressed, the pathologist may change the status to the appropriate selection.\n<\/p><p>The status has been the main factor for progressing a case through the workflow. We recognized that the overall workflow could be split by personnel, the MLS staff, and pathologists, and could be further broken up by the steps in those workflows. Therefore, we developed separate tabular views (Table 3) that collectively group pending cases based on the step in the workflow the case currently resides.\n<\/p>\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"3\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 3.<\/b> Tabular views of pending cases. Each tab holds a list of cases based on the state the case resides in the workflow.\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Number\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Table filter\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Description\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">UPDRS results entry\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">For these cases, the immunoassay results have been reviewed and an interpretation has been finalized by an attending pathologist. The interpretation of these cases requires a review by an MLS to ensure that there are no expected compounds or medications that were not mentioned in the interpretation.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Reflexes\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">This is a list of cases that has been reviewed by an attending pathologist; the cases have been requested\/approved for a reflex test ordered by an MLS in the LIS.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Opiate interpretation entry\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">For these cases, the opiate assay results have been reviewed and an interpretation has been finalized by an attending pathologist. The interpretation of these cases requires a review by an MLS to ensure that there are no expected compounds or medications that were not mentioned in the interpretation.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">4\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">SG requests\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">This tab holds a list of cases which has been marked by an attending pathologist for a request to MLSs to conduct a specific gravity (SG) test. This test will not be billed or appear in the LIS.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">5\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">MLS problems\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">These cases have been marked as \u201cProblem for MLS\u201d by a trainee or attending. The comment section will indicate more about the problem. These cases require a review by MLSs.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">6\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">UPDRS review\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">These cases require a review of the immunoassay results and medications for the patient to be collected. Once complete, a decision to either order reflex tests or make an interpretation needs to be made.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">7\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">UOPIAC review\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">These cases require a review of the final opiate LCMS confirmation assay results and medications for the patient to be collected, potentially, based on the initial order. Once complete, a decision to either order reflex tests or make an interpretation needs to be made.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">8\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Director problem\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">These cases have been marked as \u201cProblem for Director\u201d by a trainee or MLS. The comment section will indicate more about the problem. These cases require an additional review or resolution by the attending pathologist(s).\n<\/td><\/tr>\n<\/tbody><\/table>\n<h3><span class=\"mw-headline\" id=\"Performance_measurement_and_user_feedback_2\">Performance measurement and user feedback<\/span><\/h3>\n<p>Several new cases are ordered every day. In 2020, there were roughly 150 interpretations generated per week. Having dedicated queues for each step in the workflow helped staff and pathologist coordinate the work. Also, the queues helped pathologists prioritize and schedule the work based on the amount of time each pathologist had at a given time.\n<\/p><p>To investigate the return on investment, we surveyed the users on the amount of time each step in the workflow took to complete both before and after the application. The workflow no longer involved worksheets. Before the application, users reported they often spent one to two hours per day managing and combining files. This number came down to zero as they no longer had to use worksheets. Additionally, the introduction of the liquid handler to automatically prepare samples for the assay helped gain back an hour of time the MLS spent preparing the samples manually. The greatest return for staff was moving from manual result-entry to automated result-entry. The test is conducted five times per week and requires six days of staffing to process and manage data. Each batch requires the review of two laboratory staff members, and that practice continues but they would manually enter the results into the LIS. Staff reported they would spend around one hour per batch entering the results. This number came down to virtually zero as the result entry was automated via the flat file transfer to Data Innovations' Instrument Manager. For pathologists, they reported the interface was easier to navigate the multitude of values in a spreadsheet. The organization helped keep better track of cases and organize the workflow to be more efficient. On average, pathologists reported they have gained back two hours per batch since the application has been incorporated into the workflow. MLS staff reported they gained back four hours per batch since the application has been incorporated into the workflow.\n<\/p><p>In the 24 months prior to implementation of the application, there were 24,256 total opiate mass spectrometry test orders, and 449 orders had at least one data entry modification (defined as at least one test component having a result entered more than once), for a modification rate of 1.9%. After implementation, over 21 months there were 18,112 total opiate orders and only 10 orders had at least one data entry modification, yielding a modification rate of 0.06%.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Discussion\">Discussion<\/span><\/h2>\n<p>We have developed an application that centralizes data for UDT interpretation. Moreover, the automatic aggregation of this data provides an efficient and convenient method for pathologists to review the data to generate an interpretation of the results for clinicians and providers to review with their patient's undergoing opioid therapy. There are various groups that have developed novel web-based applications to help increase efficiency.<sup id=\"rdp-ebb-cite_ref-13\" class=\"reference\"><a href=\"#cite_note-13\">[13]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-14\" class=\"reference\"><a href=\"#cite_note-14\">[14]<\/a><\/sup> Groups have reported that their applications have reduced errors, expenses, staff burnout, and increased the level of quality patient care. Laboratory staff were able to gain back several hours from what they would have spent manually entering results and managing spreadsheets. After the application was introduced, the staff scheduled to the opiate bench was reduced from two full-time employees, six days a week, to one full-time employee, six days a week. Often the reviewing laboratory staff has worked in the department for some time before training for this work. Freeing an experienced laboratory member's time allowed them to help in other critical areas in the laboratory. Additionally, since the transfer process was fully automated, it reduced the number of errors transmitted into the LIS and time spent contacting the provider who may have already reviewed the results. Mays and Mathias<sup id=\"rdp-ebb-cite_ref-15\" class=\"reference\"><a href=\"#cite_note-15\">[15]<\/a><\/sup> found a 3.7% error rate of manual transcription versus auto-entry. In our analysis, 1.8% of orders required a modification prior to implementation, and post-implementation this rate was reduced significantly to 0.05%. While it is likely that some modifications were made within the application as multiple team members composed and reviewed results prior to release, misinterpretation or confusion could be avoided by preventing information that needed modification from being entered into the LIS and seen by the provider or patient.\n<\/p><p>The informatics team dedicated substantial effort through a year and a half timeframe to develop and implement the application. The team included one member responsible for interviewing the stakeholders and documenting the requirements; a faculty member who designed the software architecture and database schema, and also developed the application prototype and deployment infrastructure; two software developers who worked together to complete the application; one software developer who helped interface the LIS to gather the necessary patient and order information; and one LIS specialist to help configure the LIS for auto-filing the results. Team members had specific tasks, which could have been worked upon in parallel, although many tasks relied on the completion of a task by a separate team member. Additionally, no member dedicated their time solely to this project during the project span. Each member balanced their time across several projects. The overall estimated time spent by the team was roughly 350\u202fhours over 18 months. The application was developed by departmental programming staff, one of whom was hired specifically to support the chemistry division, while the others were members of a departmental pool of informatics resources. Having a departmental informatics team that can develop novel solutions to complex, clinical problems require considerable resources, but there are also a variety of opportunities across units of the laboratory that benefit from high efficiency.\n<\/p><p>After its initial launch, updates and enhancements were made to the application. The enhancements included bug fixes and additional features that were requested from users after working with the application for some time. The amount of time was roughly 130\u202fhours over 18 months. The amount of time spent in developing and enhancing the application was significant. However, introducing the application to the workflow freed up a full-time laboratory employee\u2019s time. Moreover, the laboratory has been able to increase the sample volume with no increased need in MLS staffing.\n<\/p><p>The return on investment (ROI) for the laboratory was significant. We demonstrated that the primary benefit to implementing a custom application has been a profound improvement in workflow and staff efficiency. The application removed mundane and repetitive tasks such as transferring data from one spreadsheet to the another. Additionally, it also removed the need for laboratory staff to fully complete the batch review before a reviewing pathologist could begin their review. With the application, the pathologist could begin as soon as the first patient in the batch was reviewed by the laboratory staff member. Additionally, the process of auto-filing results could be extended to other tests within the department, freeing some additional time for staff. This provides evidence to the value of developing and implementing custom applications.\n<\/p><p>A consideration that groups should make when developing custom applications is the time required to understand the problem. For this project, we spent several hours interviewing stakeholders to understand the workflow and the different needs of each personnel. Once requirements were gathered, a plan needed to be discussed and agreed upon by members of both the <a href=\"https:\/\/www.limswiki.org\/index.php\/Informatics_(academic_field)\" title=\"Informatics (academic field)\" class=\"wiki-link\" data-key=\"0391318826a5d9f9a1a1bcc88394739f\">informatics<\/a> and <a href=\"https:\/\/www.limswiki.org\/index.php\/Chemistry\" title=\"Chemistry\" class=\"wiki-link\" data-key=\"792a1b261a4ecc7803e3f7f1de2dbda1\">chemistry<\/a> departments. \n<\/p><p>Another challenge groups should consider is the potential requirement to rewrite software programs and repeat validation with upgrades to the software on top of any tests needed to be conducted to ensure the integrity and accuracy of a custom application. For the initial validation of the application, the informatics and chemistry teams discussed and agreed upon a validation plan. The plan involved laboratory members testing the application\u2019s functionality by working through the application workflow with test cases and matching the tested output with expected output. Additionally, groups should consider the availability of the development team to support their custom application. The need for support could stem from latency of servers due to traffic, system downtime, or undiscovered bugs that affect the behavior of the application. For these reasons, a support plan is recommended that is distributed to both users and supporting staff so that users know who to contact and IT staff understand how to effectively address the problem. The support plan may also require that laboratory staff keep up to date with manual result entry processes in case electronic systems undergo an unexpected downtime for a prolonged period. Lastly, the need to be cognizant of <a href=\"https:\/\/www.limswiki.org\/index.php\/Protected_health_information\" title=\"Protected health information\" class=\"wiki-link\" data-key=\"eca2f6661b6896668bd523e640e12499\">protected health information<\/a> (PHI)-related IT security increases the need of IT involvement for the implementation and ongoing support of custom applications.\n<\/p><p>In our laboratory, the opiate LCMS confirmation assay is one of the most labor-intensive assays because of the sample preparation method and data review steps. This work laid the foundation to move other LCMS assays to automation. As this trend continues, our staff will be able to spend their time reviewing critical issues and tasks, further increasing the efficiency in the laboratory.\n<\/p>\n<h2><span id=\"rdp-ebb-Abbreviations,_acronyms,_and_initialisms\"><\/span><span class=\"mw-headline\" id=\"Abbreviations.2C_acronyms.2C_and_initialisms\">Abbreviations, acronyms, and initialisms<\/span><\/h2>\n<ul><li><b>AWS<\/b>: Amazon Web Services<\/li>\n<li><b>CSV<\/b>: comma-separated values<\/li>\n<li><b>EHR<\/b>: electronic health record<\/li>\n<li><b>GC\u2013MS<\/b>: gas chromatography\u2013mass spectrometry<\/li>\n<li><b>LC-MS\/MS<\/b>: liquid chromatography\u2013tandem mass spectrometry<\/li>\n<li><b>LCMS<\/b>: liquid chromatography\u2013mass spectrometry<\/li>\n<li><b>LIS<\/b>: laboratory information system<\/li>\n<li><b>MLS<\/b>: medical laboratory scientist<\/li>\n<li><b>QC<\/b>: quality control<\/li>\n<li><b>ROI<\/b>: return on investment<\/li>\n<li><b>TSV<\/b>: tab-separated values<\/li>\n<li><b>UDT<\/b>: urine drug test<\/li>\n<li><b>UI<\/b>: user interface<\/li>\n<li><b>XML<\/b>: extensible markup language<\/li><\/ul>\n<h2><span class=\"mw-headline\" id=\"Appendix_A._Supplementary_data\">Appendix A. Supplementary data<\/span><\/h2>\n<ul><li><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ars.els-cdn.com\/content\/image\/1-s2.0-S2153353923001177-mmc1.docx\" target=\"_blank\">Supplementary material<\/a> (.docx)<\/li><\/ul>\n<h2><span class=\"mw-headline\" id=\"Acknowledgements\">Acknowledgements<\/span><\/h2>\n<p>The authors would like to acknowledge the time and effort spent by the medical laboratory scientists of the chemistry laboratories at the University of Washington Medical Center and Harborview Medical Center in performing high quality mass spectrometry testing to serve their patients.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Conflict_of_interest\">Conflict of interest<\/span><\/h3>\n<p>The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"References\">References<\/span><\/h2>\n<div class=\"reflist references-column-width\" style=\"-moz-column-width: 30em; -webkit-column-width: 30em; column-width: 30em; list-style-type: decimal;\">\n<div class=\"mw-references-wrap mw-references-columns\"><ol class=\"references\">\n<li id=\"cite_note-:0-1\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:0_1-0\">1.0<\/a><\/sup> <sup><a href=\"#cite_ref-:0_1-1\">1.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Owen, Graves T.; Burton, Allen W.; Schade, Cristy M.; Passik, Steve (1 July 2012). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/22786451\" target=\"_blank\">\"Urine drug testing: current recommendations and best practices\"<\/a>. <i>Pain Physician<\/i> <b>15<\/b> (3 Suppl): ES119\u2013133. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2150-1149\" target=\"_blank\">2150-1149<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/22786451\" target=\"_blank\">22786451<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/22786451\" target=\"_blank\">https:\/\/pubmed.ncbi.nlm.nih.gov\/22786451<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Urine+drug+testing%3A+current+recommendations+and+best+practices&rft.jtitle=Pain+Physician&rft.aulast=Owen&rft.aufirst=Graves+T.&rft.au=Owen%2C%26%2332%3BGraves+T.&rft.au=Burton%2C%26%2332%3BAllen+W.&rft.au=Schade%2C%26%2332%3BCristy+M.&rft.au=Passik%2C%26%2332%3BSteve&rft.date=1+July+2012&rft.volume=15&rft.issue=3+Suppl&rft.pages=ES119%E2%80%93133&rft.issn=2150-1149&rft_id=info:pmid\/22786451&rft_id=https%3A%2F%2Fpubmed.ncbi.nlm.nih.gov%2F22786451&rfr_id=info:sid\/en.wikipedia.org:Journal:A_web_application_to_support_the_coordination_of_reflexive,_interpretative_toxicology_testing\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-2\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-2\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Pesce, Amadeo; West, Cameron; Egan City, Kathy; Strickland, Jennifer (1 July 2012). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/academic.oup.com\/painmedicine\/article-lookup\/doi\/10.1111\/j.1526-4637.2012.01350.x\" target=\"_blank\">\"Interpretation of Urine Drug Testing in Pain Patients\"<\/a> (in en). <i>Pain Medicine<\/i> <b>13<\/b> (7): 868\u2013885. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1111%2Fj.1526-4637.2012.01350.x\" target=\"_blank\">10.1111\/j.1526-4637.2012.01350.x<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1526-2375\" target=\"_blank\">1526-2375<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/academic.oup.com\/painmedicine\/article-lookup\/doi\/10.1111\/j.1526-4637.2012.01350.x\" target=\"_blank\">https:\/\/academic.oup.com\/painmedicine\/article-lookup\/doi\/10.1111\/j.1526-4637.2012.01350.x<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Interpretation+of+Urine+Drug+Testing+in+Pain+Patients&rft.jtitle=Pain+Medicine&rft.aulast=Pesce&rft.aufirst=Amadeo&rft.au=Pesce%2C%26%2332%3BAmadeo&rft.au=West%2C%26%2332%3BCameron&rft.au=Egan+City%2C%26%2332%3BKathy&rft.au=Strickland%2C%26%2332%3BJennifer&rft.date=1+July+2012&rft.volume=13&rft.issue=7&rft.pages=868%E2%80%93885&rft_id=info:doi\/10.1111%2Fj.1526-4637.2012.01350.x&rft.issn=1526-2375&rft_id=https%3A%2F%2Facademic.oup.com%2Fpainmedicine%2Farticle-lookup%2Fdoi%2F10.1111%2Fj.1526-4637.2012.01350.x&rfr_id=info:sid\/en.wikipedia.org:Journal:A_web_application_to_support_the_coordination_of_reflexive,_interpretative_toxicology_testing\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:1-3\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:1_3-0\">3.0<\/a><\/sup> <sup><a href=\"#cite_ref-:1_3-1\">3.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Dams, Riet; Murphy, Constance M.; Lambert, Willy E.; Huestis, Marilyn A. (30 July 2003). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/rcm.1098\" target=\"_blank\">\"Urine drug testing for opioids, cocaine, and metabolites by direct injection liquid chromatography\/tandem mass spectrometry\"<\/a> (in en). <i>Rapid Communications in Mass Spectrometry<\/i> <b>17<\/b> (14): 1665\u20131670. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1002%2Frcm.1098\" target=\"_blank\">10.1002\/rcm.1098<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0951-4198\" target=\"_blank\">0951-4198<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/rcm.1098\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/rcm.1098<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Urine+drug+testing+for+opioids%2C+cocaine%2C+and+metabolites+by+direct+injection+liquid+chromatography%2Ftandem+mass+spectrometry&rft.jtitle=Rapid+Communications+in+Mass+Spectrometry&rft.aulast=Dams&rft.aufirst=Riet&rft.au=Dams%2C%26%2332%3BRiet&rft.au=Murphy%2C%26%2332%3BConstance+M.&rft.au=Lambert%2C%26%2332%3BWilly+E.&rft.au=Huestis%2C%26%2332%3BMarilyn+A.&rft.date=30+July+2003&rft.volume=17&rft.issue=14&rft.pages=1665%E2%80%931670&rft_id=info:doi\/10.1002%2Frcm.1098&rft.issn=0951-4198&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1002%2Frcm.1098&rfr_id=info:sid\/en.wikipedia.org:Journal:A_web_application_to_support_the_coordination_of_reflexive,_interpretative_toxicology_testing\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:2-4\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:2_4-0\">4.0<\/a><\/sup> <sup><a href=\"#cite_ref-:2_4-1\">4.1<\/a><\/sup> <sup><a href=\"#cite_ref-:2_4-2\">4.2<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Coles, Rebecka; Kushnir, Mark M.; Nelson, Gordon J.; McMillin, Gwendolyn A.; Urry, Francis M. (1 January 2007). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/academic.oup.com\/jat\/article\/31\/1\/1\/780634\/Simultaneous-Determination-of-Codeine-Morphine\" target=\"_blank\">\"Simultaneous Determination of Codeine, Morphine, Hydrocodone, Hydromorphone, Oxycodone, and 6-Acetylmorphine in Urine, Serum, Plasma, Whole Blood, and Meconium by LC-MS-MS\"<\/a> (in en). <i>Journal of Analytical Toxicology<\/i> <b>31<\/b> (1): 1\u201314. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1093%2Fjat%2F31.1.1\" target=\"_blank\">10.1093\/jat\/31.1.1<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1945-2403\" target=\"_blank\">1945-2403<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/academic.oup.com\/jat\/article\/31\/1\/1\/780634\/Simultaneous-Determination-of-Codeine-Morphine\" target=\"_blank\">http:\/\/academic.oup.com\/jat\/article\/31\/1\/1\/780634\/Simultaneous-Determination-of-Codeine-Morphine<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Simultaneous+Determination+of+Codeine%2C+Morphine%2C+Hydrocodone%2C+Hydromorphone%2C+Oxycodone%2C+and+6-Acetylmorphine+in+Urine%2C+Serum%2C+Plasma%2C+Whole+Blood%2C+and+Meconium+by+LC-MS-MS&rft.jtitle=Journal+of+Analytical+Toxicology&rft.aulast=Coles&rft.aufirst=Rebecka&rft.au=Coles%2C%26%2332%3BRebecka&rft.au=Kushnir%2C%26%2332%3BMark+M.&rft.au=Nelson%2C%26%2332%3BGordon+J.&rft.au=McMillin%2C%26%2332%3BGwendolyn+A.&rft.au=Urry%2C%26%2332%3BFrancis+M.&rft.date=1+January+2007&rft.volume=31&rft.issue=1&rft.pages=1%E2%80%9314&rft_id=info:doi\/10.1093%2Fjat%2F31.1.1&rft.issn=1945-2403&rft_id=http%3A%2F%2Facademic.oup.com%2Fjat%2Farticle%2F31%2F1%2F1%2F780634%2FSimultaneous-Determination-of-Codeine-Morphine&rfr_id=info:sid\/en.wikipedia.org:Journal:A_web_application_to_support_the_coordination_of_reflexive,_interpretative_toxicology_testing\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-5\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-5\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Dickerson, J. A.; Laha, T. J.; Pagano, M. B.; O'Donnell, B. R.; Hoofnagle, A. N. (1 October 2012). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/academic.oup.com\/jat\/article-lookup\/doi\/10.1093\/jat\/bks063\" target=\"_blank\">\"Improved Detection of Opioid Use in Chronic Pain Patients through Monitoring of Opioid Glucuronides in Urine\"<\/a> (in en). <i>Journal of Analytical Toxicology<\/i> <b>36<\/b> (8): 541\u2013547. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1093%2Fjat%2Fbks063\" target=\"_blank\">10.1093\/jat\/bks063<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0146-4760\" target=\"_blank\">0146-4760<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/academic.oup.com\/jat\/article-lookup\/doi\/10.1093\/jat\/bks063\" target=\"_blank\">https:\/\/academic.oup.com\/jat\/article-lookup\/doi\/10.1093\/jat\/bks063<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Improved+Detection+of+Opioid+Use+in+Chronic+Pain+Patients+through+Monitoring+of+Opioid+Glucuronides+in+Urine&rft.jtitle=Journal+of+Analytical+Toxicology&rft.aulast=Dickerson&rft.aufirst=J.+A.&rft.au=Dickerson%2C%26%2332%3BJ.+A.&rft.au=Laha%2C%26%2332%3BT.+J.&rft.au=Pagano%2C%26%2332%3BM.+B.&rft.au=O%27Donnell%2C%26%2332%3BB.+R.&rft.au=Hoofnagle%2C%26%2332%3BA.+N.&rft.date=1+October+2012&rft.volume=36&rft.issue=8&rft.pages=541%E2%80%93547&rft_id=info:doi\/10.1093%2Fjat%2Fbks063&rft.issn=0146-4760&rft_id=https%3A%2F%2Facademic.oup.com%2Fjat%2Farticle-lookup%2Fdoi%2F10.1093%2Fjat%2Fbks063&rfr_id=info:sid\/en.wikipedia.org:Journal:A_web_application_to_support_the_coordination_of_reflexive,_interpretative_toxicology_testing\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-6\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-6\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Yang, Yifei K; Johnson-Davis, Kamisha L; Kelly, Brian N; McMillin, Gwendolyn A (1 September 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/academic.oup.com\/jalm\/article\/5\/5\/858\/5900440\" target=\"_blank\">\"Demand for Interpretation of a Urine Drug Testing Panel Reflects the Changing Landscape of Clinical Needs; Opportunities for the Laboratory to Provide Added Clinical Value\"<\/a> (in en). <i>The Journal of Applied Laboratory Medicine<\/i> <b>5<\/b> (5): 858\u2013868. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1093%2Fjalm%2Fjfaa119\" target=\"_blank\">10.1093\/jalm\/jfaa119<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2576-9456\" target=\"_blank\">2576-9456<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/academic.oup.com\/jalm\/article\/5\/5\/858\/5900440\" target=\"_blank\">https:\/\/academic.oup.com\/jalm\/article\/5\/5\/858\/5900440<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Demand+for+Interpretation+of+a+Urine+Drug+Testing+Panel+Reflects+the+Changing+Landscape+of+Clinical+Needs%3B+Opportunities+for+the+Laboratory+to+Provide+Added+Clinical+Value&rft.jtitle=The+Journal+of+Applied+Laboratory+Medicine&rft.aulast=Yang&rft.aufirst=Yifei+K&rft.au=Yang%2C%26%2332%3BYifei+K&rft.au=Johnson-Davis%2C%26%2332%3BKamisha+L&rft.au=Kelly%2C%26%2332%3BBrian+N&rft.au=McMillin%2C%26%2332%3BGwendolyn+A&rft.date=1+September+2020&rft.volume=5&rft.issue=5&rft.pages=858%E2%80%93868&rft_id=info:doi\/10.1093%2Fjalm%2Fjfaa119&rft.issn=2576-9456&rft_id=https%3A%2F%2Facademic.oup.com%2Fjalm%2Farticle%2F5%2F5%2F858%2F5900440&rfr_id=info:sid\/en.wikipedia.org:Journal:A_web_application_to_support_the_coordination_of_reflexive,_interpretative_toxicology_testing\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:3-7\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:3_7-0\">7.0<\/a><\/sup> <sup><a href=\"#cite_ref-:3_7-1\">7.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Reisfield, MD, Gary M.; Webb, PhD, Fern J.; Bertholf, PhD, Roger L.; Sloan, MD, Paul A.; Wilson, MD, George R. (1 November 2007). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/wmpllc.org\/ojs\/index.php\/jom\/article\/view\/1034\" target=\"_blank\">\"Family physicians\u2019 proficiency in urine drug test interpretation\"<\/a>. <i>Journal of Opioid Management<\/i> <b>3<\/b> (6): 333\u2013337. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.5055%2Fjom.2007.0022\" target=\"_blank\">10.5055\/jom.2007.0022<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1551-7489\" target=\"_blank\">1551-7489<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/wmpllc.org\/ojs\/index.php\/jom\/article\/view\/1034\" target=\"_blank\">https:\/\/wmpllc.org\/ojs\/index.php\/jom\/article\/view\/1034<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Family+physicians%E2%80%99+proficiency+in+urine+drug+test+interpretation&rft.jtitle=Journal+of+Opioid+Management&rft.aulast=Reisfield%2C+MD&rft.aufirst=Gary+M.&rft.au=Reisfield%2C+MD%2C%26%2332%3BGary+M.&rft.au=Webb%2C+PhD%2C%26%2332%3BFern+J.&rft.au=Bertholf%2C+PhD%2C%26%2332%3BRoger+L.&rft.au=Sloan%2C+MD%2C%26%2332%3BPaul+A.&rft.au=Wilson%2C+MD%2C%26%2332%3BGeorge+R.&rft.date=1+November+2007&rft.volume=3&rft.issue=6&rft.pages=333%E2%80%93337&rft_id=info:doi\/10.5055%2Fjom.2007.0022&rft.issn=1551-7489&rft_id=https%3A%2F%2Fwmpllc.org%2Fojs%2Findex.php%2Fjom%2Farticle%2Fview%2F1034&rfr_id=info:sid\/en.wikipedia.org:Journal:A_web_application_to_support_the_coordination_of_reflexive,_interpretative_toxicology_testing\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-8\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-8\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">West, MS, Robert; Pesce, PhD, DABCC, Amadeo; West, PhD, Cameron; Crews, PhD, Bridgit; Mikel, PhD, Charles; Rosenthal, DO, FAPA, Murray; Almazan, CLS, MT (ASCP), Perla; Latyshev, MS, Sergey (29 January 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/wmpllc.org\/ojs\/index.php\/jom\/article\/view\/894\" target=\"_blank\">\"Observations of medication compliance by measurement of urinary drug concentrations in a pain management population\"<\/a>. <i>Journal of Opioid Management<\/i> <b>6<\/b> (4): 253\u2013257. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.5055%2Fjom.2010.0023\" target=\"_blank\">10.5055\/jom.2010.0023<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1551-7489\" target=\"_blank\">1551-7489<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/wmpllc.org\/ojs\/index.php\/jom\/article\/view\/894\" target=\"_blank\">https:\/\/wmpllc.org\/ojs\/index.php\/jom\/article\/view\/894<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Observations+of+medication+compliance+by+measurement+of+urinary+drug+concentrations+in+a+pain+management+population&rft.jtitle=Journal+of+Opioid+Management&rft.aulast=West%2C+MS&rft.aufirst=Robert&rft.au=West%2C+MS%2C%26%2332%3BRobert&rft.au=Pesce%2C+PhD%2C+DABCC%2C%26%2332%3BAmadeo&rft.au=West%2C+PhD%2C%26%2332%3BCameron&rft.au=Crews%2C+PhD%2C%26%2332%3BBridgit&rft.au=Mikel%2C+PhD%2C%26%2332%3BCharles&rft.au=Rosenthal%2C+DO%2C+FAPA%2C%26%2332%3BMurray&rft.au=Almazan%2C+CLS%2C+MT+%28ASCP%29%2C%26%2332%3BPerla&rft.au=Latyshev%2C+MS%2C%26%2332%3BSergey&rft.date=29+January+2018&rft.volume=6&rft.issue=4&rft.pages=253%E2%80%93257&rft_id=info:doi\/10.5055%2Fjom.2010.0023&rft.issn=1551-7489&rft_id=https%3A%2F%2Fwmpllc.org%2Fojs%2Findex.php%2Fjom%2Farticle%2Fview%2F894&rfr_id=info:sid\/en.wikipedia.org:Journal:A_web_application_to_support_the_coordination_of_reflexive,_interpretative_toxicology_testing\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-9\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-9\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Levy, Sharon; Sherritt, Lon; Vaughan, Brigid L.; Germak, Matthew; Knight, John R. (1 April 2007). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/publications.aap.org\/pediatrics\/article\/119\/4\/e843\/70141\/Results-of-Random-Drug-Testing-in-an-Adolescent\" target=\"_blank\">\"Results of Random Drug Testing in an Adolescent Substance Abuse Program\"<\/a> (in en). <i>Pediatrics<\/i> <b>119<\/b> (4): e843\u2013e848. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1542%2Fpeds.2006-2278\" target=\"_blank\">10.1542\/peds.2006-2278<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0031-4005\" target=\"_blank\">0031-4005<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/publications.aap.org\/pediatrics\/article\/119\/4\/e843\/70141\/Results-of-Random-Drug-Testing-in-an-Adolescent\" target=\"_blank\">https:\/\/publications.aap.org\/pediatrics\/article\/119\/4\/e843\/70141\/Results-of-Random-Drug-Testing-in-an-Adolescent<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Results+of+Random+Drug+Testing+in+an+Adolescent+Substance+Abuse+Program&rft.jtitle=Pediatrics&rft.aulast=Levy&rft.aufirst=Sharon&rft.au=Levy%2C%26%2332%3BSharon&rft.au=Sherritt%2C%26%2332%3BLon&rft.au=Vaughan%2C%26%2332%3BBrigid+L.&rft.au=Germak%2C%26%2332%3BMatthew&rft.au=Knight%2C%26%2332%3BJohn+R.&rft.date=1+April+2007&rft.volume=119&rft.issue=4&rft.pages=e843%E2%80%93e848&rft_id=info:doi\/10.1542%2Fpeds.2006-2278&rft.issn=0031-4005&rft_id=https%3A%2F%2Fpublications.aap.org%2Fpediatrics%2Farticle%2F119%2F4%2Fe843%2F70141%2FResults-of-Random-Drug-Testing-in-an-Adolescent&rfr_id=info:sid\/en.wikipedia.org:Journal:A_web_application_to_support_the_coordination_of_reflexive,_interpretative_toxicology_testing\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-10\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-10\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Yang, Yifei K; Johnson-Davis, Kamisha L; Kelly, Brian N; McMillin, Gwendolyn A (1 September 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/academic.oup.com\/jalm\/article\/5\/5\/858\/5900440\" target=\"_blank\">\"Demand for Interpretation of a Urine Drug Testing Panel Reflects the Changing Landscape of Clinical Needs; Opportunities for the Laboratory to Provide Added Clinical Value\"<\/a> (in en). <i>The Journal of Applied Laboratory Medicine<\/i> <b>5<\/b> (5): 858\u2013868. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1093%2Fjalm%2Fjfaa119\" target=\"_blank\">10.1093\/jalm\/jfaa119<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2576-9456\" target=\"_blank\">2576-9456<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/academic.oup.com\/jalm\/article\/5\/5\/858\/5900440\" target=\"_blank\">https:\/\/academic.oup.com\/jalm\/article\/5\/5\/858\/5900440<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Demand+for+Interpretation+of+a+Urine+Drug+Testing+Panel+Reflects+the+Changing+Landscape+of+Clinical+Needs%3B+Opportunities+for+the+Laboratory+to+Provide+Added+Clinical+Value&rft.jtitle=The+Journal+of+Applied+Laboratory+Medicine&rft.aulast=Yang&rft.aufirst=Yifei+K&rft.au=Yang%2C%26%2332%3BYifei+K&rft.au=Johnson-Davis%2C%26%2332%3BKamisha+L&rft.au=Kelly%2C%26%2332%3BBrian+N&rft.au=McMillin%2C%26%2332%3BGwendolyn+A&rft.date=1+September+2020&rft.volume=5&rft.issue=5&rft.pages=858%E2%80%93868&rft_id=info:doi\/10.1093%2Fjalm%2Fjfaa119&rft.issn=2576-9456&rft_id=https%3A%2F%2Facademic.oup.com%2Fjalm%2Farticle%2F5%2F5%2F858%2F5900440&rfr_id=info:sid\/en.wikipedia.org:Journal:A_web_application_to_support_the_coordination_of_reflexive,_interpretative_toxicology_testing\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:4-11\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:4_11-0\">11.0<\/a><\/sup> <sup><a href=\"#cite_ref-:4_11-1\">11.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Dickerson, Jane A.; Schmeling, Michael; Hoofnagle, Andrew N.; Hoffman, Noah G. (1 January 2013). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S000989811200527X\" target=\"_blank\">\"Design and implementation of software for automated quality control and data analysis for a complex LC\/MS\/MS assay for urine opiates and metabolites\"<\/a> (in en). <i>Clinica Chimica Acta<\/i> <b>415<\/b>: 290\u2013294. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.cca.2012.10.055\" target=\"_blank\">10.1016\/j.cca.2012.10.055<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S000989811200527X\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S000989811200527X<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Design+and+implementation+of+software+for+automated+quality+control+and+data+analysis+for+a+complex+LC%2FMS%2FMS+assay+for+urine+opiates+and+metabolites&rft.jtitle=Clinica+Chimica+Acta&rft.aulast=Dickerson&rft.aufirst=Jane+A.&rft.au=Dickerson%2C%26%2332%3BJane+A.&rft.au=Schmeling%2C%26%2332%3BMichael&rft.au=Hoofnagle%2C%26%2332%3BAndrew+N.&rft.au=Hoffman%2C%26%2332%3BNoah+G.&rft.date=1+January+2013&rft.volume=415&rft.pages=290%E2%80%93294&rft_id=info:doi\/10.1016%2Fj.cca.2012.10.055&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS000989811200527X&rfr_id=info:sid\/en.wikipedia.org:Journal:A_web_application_to_support_the_coordination_of_reflexive,_interpretative_toxicology_testing\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-FlaskProj-12\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-FlaskProj_12-0\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/flask.palletsprojects.com\/en\/2.3.x\/\" target=\"_blank\">\"Flask\"<\/a>. Pallets. 2010<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/flask.palletsprojects.com\/en\/2.3.x\/\" target=\"_blank\">https:\/\/flask.palletsprojects.com\/en\/2.3.x\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Flask&rft.atitle=&rft.date=2010&rft.pub=Pallets&rft_id=https%3A%2F%2Fflask.palletsprojects.com%2Fen%2F2.3.x%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:A_web_application_to_support_the_coordination_of_reflexive,_interpretative_toxicology_testing\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-13\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-13\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Woo, Jennifer S.; Suslow, Peter; Thorsen, Russell; Ma, Rosaline; Bakhtary, Sara; Moayeri, Morvarid; Nambiar, Ashok (1 January 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2153353922003662\" target=\"_blank\">\"Development and Implementation of Real-Time Web-Based Dashboards in a Multisite Transfusion Service\"<\/a> (in en). <i>Journal of Pathology Informatics<\/i> <b>10<\/b> (1): 3. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.4103%2Fjpi.jpi_36_18\" target=\"_blank\">10.4103\/jpi.jpi_36_18<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC6396429\/\" target=\"_blank\">PMC6396429<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/30915257\" target=\"_blank\">30915257<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2153353922003662\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2153353922003662<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Development+and+Implementation+of+Real-Time+Web-Based+Dashboards+in+a+Multisite+Transfusion+Service&rft.jtitle=Journal+of+Pathology+Informatics&rft.aulast=Woo&rft.aufirst=Jennifer+S.&rft.au=Woo%2C%26%2332%3BJennifer+S.&rft.au=Suslow%2C%26%2332%3BPeter&rft.au=Thorsen%2C%26%2332%3BRussell&rft.au=Ma%2C%26%2332%3BRosaline&rft.au=Bakhtary%2C%26%2332%3BSara&rft.au=Moayeri%2C%26%2332%3BMorvarid&rft.au=Nambiar%2C%26%2332%3BAshok&rft.date=1+January+2019&rft.volume=10&rft.issue=1&rft.pages=3&rft_id=info:doi\/10.4103%2Fjpi.jpi_36_18&rft_id=info:pmc\/PMC6396429&rft_id=info:pmid\/30915257&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2153353922003662&rfr_id=info:sid\/en.wikipedia.org:Journal:A_web_application_to_support_the_coordination_of_reflexive,_interpretative_toxicology_testing\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-14\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-14\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Smith, Matthew A.; Roy, Somak; Nestler, Rick; Augustine, Beth; Miller, David; Parwani, Anil; Nichols, Lawrence (1 January 2011). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2153353922002061\" target=\"_blank\">\"University of Pittsburgh Medical Center Remains Tracker: A novel application for tracking decedents and improving the autopsy workflow\"<\/a> (in en). <i>Journal of Pathology Informatics<\/i> <b>2<\/b> (1): 30. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.4103%2F2153-3539.82055\" target=\"_blank\">10.4103\/2153-3539.82055<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC3132995\/\" target=\"_blank\">PMC3132995<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/21773061\" target=\"_blank\">21773061<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2153353922002061\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2153353922002061<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=University+of+Pittsburgh+Medical+Center+Remains+Tracker%3A+A+novel+application+for+tracking+decedents+and+improving+the+autopsy+workflow&rft.jtitle=Journal+of+Pathology+Informatics&rft.aulast=Smith&rft.aufirst=Matthew+A.&rft.au=Smith%2C%26%2332%3BMatthew+A.&rft.au=Roy%2C%26%2332%3BSomak&rft.au=Nestler%2C%26%2332%3BRick&rft.au=Augustine%2C%26%2332%3BBeth&rft.au=Miller%2C%26%2332%3BDavid&rft.au=Parwani%2C%26%2332%3BAnil&rft.au=Nichols%2C%26%2332%3BLawrence&rft.date=1+January+2011&rft.volume=2&rft.issue=1&rft.pages=30&rft_id=info:doi\/10.4103%2F2153-3539.82055&rft_id=info:pmc\/PMC3132995&rft_id=info:pmid\/21773061&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2153353922002061&rfr_id=info:sid\/en.wikipedia.org:Journal:A_web_application_to_support_the_coordination_of_reflexive,_interpretative_toxicology_testing\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-15\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-15\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Mays, James A; Mathias, Patrick C (1 March 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/academic.oup.com\/jamia\/article\/26\/3\/269\/5287977\" target=\"_blank\">\"Measuring the rate of manual transcription error in outpatient point-of-care testing\"<\/a> (in en). <i>Journal of the American Medical Informatics Association<\/i> <b>26<\/b> (3): 269\u2013272. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1093%2Fjamia%2Focy170\" target=\"_blank\">10.1093\/jamia\/ocy170<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1067-5027\" target=\"_blank\">1067-5027<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC6351970\/\" target=\"_blank\">PMC6351970<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/30649499\" target=\"_blank\">30649499<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/academic.oup.com\/jamia\/article\/26\/3\/269\/5287977\" target=\"_blank\">https:\/\/academic.oup.com\/jamia\/article\/26\/3\/269\/5287977<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Measuring+the+rate+of+manual+transcription+error+in+outpatient+point-of-care+testing&rft.jtitle=Journal+of+the+American+Medical+Informatics+Association&rft.aulast=Mays&rft.aufirst=James+A&rft.au=Mays%2C%26%2332%3BJames+A&rft.au=Mathias%2C%26%2332%3BPatrick+C&rft.date=1+March+2019&rft.volume=26&rft.issue=3&rft.pages=269%E2%80%93272&rft_id=info:doi\/10.1093%2Fjamia%2Focy170&rft.issn=1067-5027&rft_id=info:pmc\/PMC6351970&rft_id=info:pmid\/30649499&rft_id=https%3A%2F%2Facademic.oup.com%2Fjamia%2Farticle%2F26%2F3%2F269%2F5287977&rfr_id=info:sid\/en.wikipedia.org:Journal:A_web_application_to_support_the_coordination_of_reflexive,_interpretative_toxicology_testing\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<\/ol><\/div><\/div>\n<h2><span class=\"mw-headline\" id=\"Notes\">Notes<\/span><\/h2>\n<p>This presentation is faithful to the original, with only a few minor changes to presentation, spelling, and grammar. In some cases important information was missing from the references, and that information was added.\n<\/p>\n<!-- \nNewPP limit report\nCached time: 20231215231613\nCache expiry: 86400\nDynamic content: false\nComplications: []\nCPU time usage: 0.415 seconds\nReal time usage: 0.595 seconds\nPreprocessor visited node count: 17519\/1000000\nPost\u2010expand include size: 155122\/2097152 bytes\nTemplate argument size: 48627\/2097152 bytes\nHighest expansion depth: 25\/40\nExpensive parser function count: 0\/100\nUnstrip recursion depth: 0\/20\nUnstrip post\u2010expand size: 40056\/5000000 bytes\n-->\n<!--\nTransclusion expansion time report (%,ms,calls,template)\n100.00% 260.154 1 -total\n 81.71% 212.568 1 Template:Reflist\n 65.68% 170.858 14 Template:Cite_journal\n 63.31% 164.711 15 Template:Citation\/core\n 12.26% 31.898 1 Template:Infobox_journal_article\n 11.48% 29.856 15 Template:Date\n 10.78% 28.036 1 Template:Infobox\n 9.54% 24.817 31 Template:Citation\/identifier\n 5.40% 14.045 80 Template:Infobox\/row\n 4.56% 11.853 1 Template:Cite_web\n-->\n\n<!-- Saved in parser cache with key limswiki:pcache:idhash:14244-0!canonical and timestamp 20231215231612 and revision id 52295. Serialized with JSON.\n -->\n<\/div><\/div><div class=\"printfooter\">Source: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.limswiki.org\/index.php\/Journal:A_web_application_to_support_the_coordination_of_reflexive,_interpretative_toxicology_testing\">https:\/\/www.limswiki.org\/index.php\/Journal:A_web_application_to_support_the_coordination_of_reflexive,_interpretative_toxicology_testing<\/a><\/div>\n<!-- end content --><div class=\"visualClear\"><\/div><\/div><\/div><div class=\"visualClear\"><\/div><\/div><!-- end of the left (by default at least) column --><div class=\"visualClear\"><\/div><\/div>\n\n\n\n<\/body>","b283f4f5c78061a91491bc16c8576d36_images":["https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/f\/fd\/Fig1_Pablo_JofPathInfo2023_14.jpg","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/4\/4f\/Fig2_Pablo_JofPathInfo2023_14.jpg","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/1\/18\/Fig3_Pablo_JofPathInfo2023_14.jpg","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/b\/be\/Fig4_Pablo_JofPathInfo2023_14.jpg"],"b283f4f5c78061a91491bc16c8576d36_timestamp":1702682172,"49083cfbd81897f5701da971794583e5_type":"article","49083cfbd81897f5701da971794583e5_title":"Laboratory automation, informatics, and artificial intelligence: Current and future perspectives in clinical microbiology (Mencacci et al. 2023)","49083cfbd81897f5701da971794583e5_url":"https:\/\/www.limswiki.org\/index.php\/Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology","49083cfbd81897f5701da971794583e5_plaintext":"\n\nJournal:Laboratory automation, informatics, and artificial intelligence: Current and future perspectives in clinical microbiologyFrom LIMSWikiJump to navigationJump to searchFull article title\n \nLaboratory automation, informatics, and artificial intelligence: Current and future perspectives in clinical microbiologyJournal\n \nFrontiers in Cellular and Infection MicrobiologyAuthor(s)\n \nMencacci, Antonella; De Socio, Guiseppe V.; Pirelli, Eleonora; Bondi, Paola; Cenci, ElioAuthor affiliation(s)\n \nUniversity of Perugia, Perugia General HospitalPrimary contact\n \nEmail: antonella at mencacci at unipg dot itEditors\n \nBlasi, ElisabettaYear published\n \n2023Volume and issue\n \n13Article #\n \n1188684DOI\n \n10.3389\/fcimb.2023.1188684ISSN\n \n2235-2988Distribution license\n \nCreative Commons Attribution 4.0 InternationalWebsite\n \nhttps:\/\/www.frontiersin.org\/articles\/10.3389\/fcimb.2023.1188684\/fullDownload\n \nhttps:\/\/www.frontiersin.org\/articles\/10.3389\/fcimb.2023.1188684\/pdf (PDF)\n\nContents \n\n1 Abstract \n2 Introduction \n3 Impact on laboratory management \n\n3.1 Inoculation \n3.2 Incubation \n3.3 Reading \n3.4 ID and AST \n3.5 Artificial intelligence \n3.6 Other functions of laboratory automation \n\n\n4 Impact and possible improvements of laboratory automation \n\n4.1 Impact on patient management \n4.2 Impact on hospital management \n4.3 Possible improvements to laboratory automation systems \n\n\n5 Discussion \n6 Conclusion \n7 Abbreviations, acronyms, and initialisms \n8 Acknowledgements \n\n8.1 Author contributions \n8.2 Data availability statement \n8.3 Conflict of interest \n\n\n9 References \n10 Notes \n\n\n\nAbstract \nClinical diagnostic laboratories produce one product\u2014information\u2014and for this to be valuable, the information must be clinically relevant, accurate, and timely. Although diagnostic information can clearly improve patient outcomes and decrease healthcare costs, technological challenges and laboratory workflow practices affect the timeliness and clinical value of diagnostics. This article will examine how prioritizing laboratory practices in a patient-oriented approach can be used to optimize technology advances for improved patient care.\nKeywords: laboratory automation, artificial intelligence, informatics, laboratory workflow, Kiestra, WASPLab\n\nIntroduction \nPatterns of infectious diseases have changed dramatically: patients are frequently immunocompromised and often have complicating comorbidities; infections with multi-drug-resistant organisms (MDRO) are a global problem; and new antibiotics are available, but it is mandatory to preserve their efficacy. It is estimated that at least 700,000 people die worldwide every year with infections caused by MDRO, and it is predicted that by 2050, 10 million deaths might occur due to these organisms.[1] Administration of rapid, broad-spectrum empiric therapy is essential to improve patient outcome[2], but this is often inappropriate.[3][4] For example, meta-analysis assessing the impact of antibiotic therapy on Gram-negative sepsis showed that inappropriate therapy was associated with 3.3-fold increased risk of mortality, longer hospitalization, and higher costs.[5] Thus, rapid, accurate diagnostics are critical for the selection of the most appropriate therapy.\nAdvanced, sophisticated technologies such as mass spectrometry and molecular diagnostics are rapidly changing our ability to diagnose infections[6], although they should be viewed as complementary to traditional growth-based diagnostics. Laboratory automation and intelligent applications of of informatics also have a transformative impact of microbiology diagnostics. These tools have the potential to accelerate clinical decision-making and positively impact the management of infections, improve patient outcome, and facilitate diagnostic and antimicrobial stewardship (AS) programs.[7] However, it is a challenge for clinical microbiologists to implement these technologies because it requires changing well-established workflow practices. This paper will focus on the impact of automation and informatics combined with workflow changes on laboratory, patient, and hospital management (Figure 1).\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 1. Impact of automation on laboratory, patient, and hospital management.\n\n\n\nImpact on laboratory management \nIn clinical microbiology, the term \u201ctotal laboratory automation\u201d (TLA) is used to describe the automation of the entire diagnostic workflow, from inoculation of the agar plates to incubation, reading of culture results, identification (ID), and antimicrobial susceptibility testing (AST). All these steps in a conventional laboratory are performed manually, usually according to a sample-centered approach. At present, two laboratory automation systems are available: the BD Kiestra system (Becton Dickinson, Sparks, MD) and the WASPLab system (Copan Diagnostics, Murrieta, CA).[8] We discuss these systems within the context of this workflow and then address other laboratory automation tools, as well artificial intelligence (AI).\n\nInoculation \nQuality and precision of inoculation are improved by automation. Instruments work in a standardized and consistent mode, not achievable with a manual procedure, and independent of operator variability. Indeed, laboratory automation allows better isolation of colonies compared to manual inoculation, with decreased need of subcultures for follow-up work, mainly AST, resulting in a more rapid report.[9][10] It was found that WASP automated streaking of urines using a sterile loop was superior to manual streaking, yielding a higher number of single colonies and of detected morphologies, species, and pathogens.[11] The BD Kiestra system, based on a rolling magnetic bead streaking technology, has been shown to improve the accuracy of quantitative culture results and the recovery of discrete colonies from polymicrobial samples, compared to manual and automated WASP streaking.[9][12] This implies a reduction of bacterial subcultures to perform ID and AST, thus shortening time to results, as evidenced for urines[13] and both methicillin-resistant Staphylococcus aureus (MRSA) and carbapenem-resistant Enterobacterales screening samples.[14]\n\nIncubation \nClosed incubators with digital imaging of cultures allow more rapid growth than conventional incubators that are opened frequently throughout the day. Moreover, in TLA, plates are fully tracked as long as they stay within the system, so that it is possible to define by hours and minutes incubation times and plate examination, in contrast with the traditional system in which incubation times are defined in days. Burckhardt et al. showed that first growth of MRSA, multi-drug-resistant (MDR) Gram-negative bacteria, and vancomycin-resistant enterococci (VRE) on selective chromogenic plates was visible as early as after four hours of inoculation, although the bacterial mass was not sufficient for follow-up work.[15] Also, growth of Escherichia coli, Pseudomonas aeruginosa, Enterococcus faecalis, and S. aureus on chromogenic plates was three to four hours faster in the automated system than in the classic system.[16] Implementation of BD Kiestra TLA significantly improved turnaround times (TAT) for positive and negative urine cultures.[17] Similarly, WASPLab automation enabled a reduction of the culture reading time for different specimens without affecting performances.[18] However, minimum incubation times for each type of specimen, for a timely and accurate positive or negative report, are not yet defined, and additional studies are needed.\n\nReading \nThe Kiestra laboratory automation system, through a real-time dashboard, times tasks as they are scheduled. Thus, each technician perfectly knows when the culture plates will be ready for reading and when follow-up work can be performed. This strongly facilitates laboratory workflow management, avoiding wasted time and allowing results to be delivered to the clinician as soon as possible. In addition, while in the classical system plates are read one by one, digital reading allows simultaneous viewing of all the plate images from the same sample, and even of different samples from the same patient. This greatly facilitates and speeds up the interpretation of culture results, either for monomicrobial or polymicrobial infections.\n\nID and AST \nThe implementation of TLA in clinical microbiology has leveraged the advancement brought by matrix-assisted laser desorption\/ionization time-of-flight mass spectrometry (MALDI-TOF\/MS).[19][20] Furthermore, Copan\u2019s TLA has recently integrated an automated device (Colibri) that can reproducibly prepare the MALDI target for microbial identification. A recent study conducted by Cherkaoui et al. established that the WASPLab coupled to MALDI-TOF\/MS significantly reduces the TAT for positive blood cultures.[21] Similarly, the BD Kiestra IdentifA\/SusceptA, a prototype for automatic colony picking, bacterial suspension preparation, MALDI-TOF target plates spotting, and Phoenix M50 AST panel preparation, exhibited high ID and AST performances.[22] In particular, the IdentifA showed excellent identification rates for Gram-negative bacteria, outperforming manual processing for Enterobacterales identification[22], but not for streptococci, coagulase-negative staphylococci (CoNS), and yeasts.[22]\nFinally, an automated solution for disk diffusion AST was developed and integrated with the Copan WASPLab system. It prepares inoculum suspensions, inoculates culture media plates, dispenses appropriate antibiotic disks according to predefined panels, transports the plates to the incubators, takes digitalized images of the media plates, and measures and interprets the inhibition zones\u2019 diameters. Cherkaoui et al.\u2014evaluating 718 bacterial strains, including S. aureus, CoNS, E. faecalis, Enterococcus faecium, P. aeruginosa, and different species of Enterobacterales\u2014found 99.1% overall categorical agreement between this automated AST and Vitek2.[23]\n\nArtificial intelligence \nThe development of intelligent image analysis based on tailored algorithms designed on type of specimens and patient characteristics allows automated detection of microbial growth, release of negative samples, presumptive ID, and quantification of bacterial colonies. This represents a major innovation that has the potential to increase laboratory quality and productivity while reducing TAT.[13] Promising results have been obtained on urine samples, with a 97%\u201399% sensitivity and 85%\u201394% specificity by the BD Kiestra system.[15] By a different approach, the WASPLab Chromogenic Detection Module has developed automated categorization of agar plates as \u201cnegative\u201d (i.e., sterile) or \u201cnon-negative,\u201d comparing the same plate at time point zero to the plate after the established incubation time. With this system, an optimal diagnostic accuracy in MRSA[24], VRE[25], and carbapenemase-producing Enterobacteriaceae[26] detection has been observed.\n\nOther functions of laboratory automation \nLaboratory automation can greatly facilitate the implementation of an effective quality management system (QMS), which is required to ensure that reliable results are reported for patients. Laboratory automation systems automatically track and record all the useful information for quality control (QC): user credentials, media (e.g., lot number and expiration date), inoculation (e.g., volumes of samples, patterns, and times of streaking), incubation (e.g., atmosphere, temperature, and times) and imaging (e.g., digital images of plates and times) data.[27] Thus, the proper integration of laboratory automation with a laboratory information system (LIS) allows for complete traceability of the analytical process, from sample receipt to the final report.\nMoreover, the possibility to access and review any taken image represents an invaluable tool from a diagnostic point of view (e.g., comparing morphology of colonies in recent and old samples from the same patient) and also for other activities such as monitoring laboratory quality, teaching, training, and discussing culture results with colleagues and clinicians.\n\nImpact and possible improvements of laboratory automation \nImpact on patient management \nClinical impact of an assay, a technology, or a modified workflow can be defined based on its added value for patient management. In the case of sepsis, this can be measured as time to targeted therapy and, hopefully, a decrease in the mortality rate. In manual processing laboratories, the activities are performed in batches, usually based on the type of sample and type of activity (e.g., inoculation, reading, ID, AST, technical validation, and clinical validation), and the results are usually delivered mostly during the morning hours. Indeed, a study evaluating the TAT for positive blood cultures (BC) in 13 US acute care hospitals demonstrated a significant discrepancy between times of BC collection and reporting laboratory test results. While only 25% of specimens were collected between 6:00 a.m. and 11:59 a.m., approximately 80% of laboratory ID and AST results were reported in this time interval.[28] This can have a negative impact on septic patient management, delaying clinical decision-making for optimal targeted therapy.\nIn contrast, in automated laboratories, the activity can be organized according to lean principles, creating a continuous \u201cflow\u201d and producing \u201cjust-in-time\u201d results. De Socio et al. evaluated the impact of laboratory automation on septic patient management. Positive BC were processed by fully automatic inoculation on solid media and digital reading after eight hours of incubation, followed by ID and AST. The authors found that a reduction of time to report (TTR) of about one day led to a significant reduction of the duration of empirical therapy (from approximately 87 hours\u2009to approximately 55 hours) and of 30-day crude mortality rate (from 29.0% to 16.7%).[29]\nTherefore, provided that the laboratory is open 24 hours a day, or taking advantage of telemedicine systems for clinical validation, laboratory automation has a potentially great impact on patient management. However, the success of such organization lies in the responsiveness of the medical teams, who should act upon the results soon after delivery by the laboratory.[30]\n\nImpact on hospital management \nLaboratory automation can improve the laboratory's ability to characterize MDRO and produce quality results, permitting a more standardized workflow, while leaving more time for laboratory staff to focus on second-level phenotypic and\/or genotypic tests. Indeed, the large diffusion of MDRO and the expanding spectrum of resistance mechanisms among pathogens pointed out the limitations of commercial routine methods for susceptibility testing of selected antibiotics, increasing the demand for cumbersome and time-consuming reference methods. For example, in the case of MDRO Gram-negative isolates, colistin MIC should be determined by the broth microdilution method[31]; fosfomycin MIC, by the agar-dilution method[32]; and cefiderocol, a novel siderophore-conjugated cephalosporin, by the broth microdilution method using an iron-depleted cation-adjusted Mueller-Hinton broth.[33] Moreover, in the case of detection of uncommon resistance phenotypes, molecular methods, gene sequencing, or other next-generation sequencing (NGS) methods are often required.[34]\nAccuracy is not sufficient per se for a result to be useful. Information must be given to clinicians or other healthcare providers (e.g., pharmacists and the patient\u2019s primary care nurse) as quickly as possible. Timely reporting can affect hospital conditions in at least two ways: permitting the rapid control of the spread of MDRO (i.e., contact precautions, investigation of clusters of colonized\/infected patients) and reducing the duration of broad-spectrum antibiotic therapy (i.e., positive results) or unnecessary empiric antibiotic therapy (i.e., negative results).\nIn a study proposing a cumulative antimicrobial resistance index as a tool to predict antimicrobial resistance (AR) trend in a hospital, a reversion of AR trend was observed in 2018, in comparison with the 2014\u20132017 period.[35] The authors speculate that this could have been a consequence of some changes in the management of infections in their hospital: (i) incubation of all BC within one hour from collection using satellite incubators, (ii) a significant reduction in TTR after the introduction of molecular technologies and laboratory automation, and (iii) an established close collaboration between infectious disease clinicians and clinical microbiologists.[35]\nFinally, Culbreath et al. demonstrated that the implementation of TLA increased laboratory productivity by up to 90%, while reducing the cost per specimen by up to 47%, providing an excellent elaboration of the efficiencies and cost-savings that are achievable by implementation of full laboratory automation in the bacteriology laboratory.[36]\n\nPossible improvements to laboratory automation systems \nA detailed wish list of technical issues to be evaluated in order to improve the performance and workflow of laboratory automation systems has been recently published.[10] Here, we will focus on facts that, in our opinion, could affect laboratory, patient, and hospital management.\nTo facilitate the reading of the plates according to a patient-centered approach, it would be useful to view specimens\u2019 Gram stains in the same screen of cultured plates. The images could also be shared with clinicians, improving clinician\u2013microbiologist interplay. Further improvement can be made by automated microscopy systems, which can significantly reduce the workload of the technical staff.[37]\nThe availability of digital images lays the foundation for telebacteriology, intended as the use of digital imaging and file storage for on-screen reading and decision-making.[8] It makes it possible to geographically dissociate plate manipulation from reading and validation of the results. This could promote the microbiologist counseling activity and interaction with clinicians, as the images could be shared between consultants located at different sites. Also, it could support 24\/7 laboratory activity, allowing the plates to be read outside the laboratory in a hub laboratory or even at home, with follow-up work performed in real time where the plates are incubated.\nTo make these technological innovations fully operational, a middleware information technology (IT) solution is needed to connect all the laboratory's instruments.[37]\n\nDiscussion \nThe main reason to introduce automation in a laboratory is to increase productivity in the face of limited budgets and personnel shortages. However, implementation of laboratory automation can represent an exceptional opportunity to change laboratory organization, improve quality, and reduce TTR, with a potential positive impact on laboratory, patient, and hospital management.\nOne of the most relevant innovations of laboratory automation regards the reading phase, with the possibility to read simultaneously all the plates inoculated from one of even more samples from the same patient. Moreover, taking advantage of informatics, it is also possible to view patient microbiological, hematological, and even clinical and therapeutic data while reading the plates. This patient-oriented approach provides meaningful clinical interpretation of results and decision-making.\nBy continuously tracing all the analytical steps, laboratory automation ensures that the microbiologist knows in real time the work to be carried out. This concept fully adheres to the so-called \u201clean\u201d organization that, initially envisaged for industry[38]], is increasingly applied to healthcare processes. \u201cLean\u201d means to do only valuable activities, without any delay, avoiding \u201cwaste\u201d or unnecessary work. This implies a dramatic revolution in the mentality of microbiologists, transitioning from exclusively sample-centered laboratory work towards a more clinically oriented activity, shortening TTR and prioritizing diagnosis of time-dependent infections. Taking advantage of workflow optimization, a nearly 24 hour reduction in TTR has been observed for positive BC processed by laboratory automation, with a significant decrease of duration of empirical therapy and mortality.[29] Similar results were observed for urines[39] and nasal MRSA surveillance[40], as well as other specimen types.[13][17]\nAn AI algorithm to interpret culture results is another important tool applicable to laboratory automation: automated reporting of negative samples can be done without delay and further human assistance, such that clinicians can receive earlier results to rule out MDRO colonization or a urinary tract infection and reduce the need for patient isolation or antibiotic treatment.[20][24][25]\nOutside laboratory automation, a variety of technologies are revolutionizing clinical microbiology. These include MALDI-TOF\/MS[41], time-lapse microscopy for ID and phenotypic AST[42], molecular diagnostic tests and syndromic panels[6][43], and NGS.[44][45] All of them can significantly improve the diagnosis and therapy of infections, but as stated above, they are primarily complementary to culture-based methods.[6] Thus, in an advanced laboratory, the goal will be to implement the use of all these technologies in a coordinated and timely program of diagnostic stewardship (DS). For example, for active surveillance of MDRO, both molecular- and culture-based methods should be available in the laboratory.[46] Indeed, active surveillance of carbapenem-resistant Enterobacteriaceae can limit and prevent their spread and infections, which is crucially relevant to AS.[47] In high-risk patients, rapid molecular methods are more appropriate but cannot replace culture-based methods, as the latter can detect all types of carbapenem-resistant organisms, perform phenotypic susceptibility testing, and collect and store the isolates.[47]] An interesting algorithm\u2014based on a multi-parametric score that takes into account clinical, microbiological, and biochemical parameters\u2014has been recently proposed to establish patient priority, including information on infection or colonization by MDRO.[48] It is reasonable to think that by combining DS and AS programs with a strict collaboration between laboratory and clinicians, the impact of modern microbiology on the management of infection can progressively increase.\nIn this vein, rapid and effective communication from laboratory to wards and back is essential for optimal patient care. A recent study showed that many barriers exist, like verbal reporting of results, poorly integrated information systems, mutual lack of insight into each other\u2019s area of expertise, and limited laboratory services.[49] Electronic reporting improves communication between microbiologists and clinical staff, but a type of alert system for the right physician (i.e., the treating clinician, an infectious diseases specialist, or a sepsis team member) to look up the data immediately should be integrated. Nevertheless, we believe that direct microbiologist\/clinician interplay remains crucial for an optimal patient management: positive BC, detection of MDRO, isolation of alert organisms from sterile fluids, and acid-fast bacilli in respiratory samples must be immediately reported to someone who will act on the results.\nMoreover, as microbiological methods become increasingly sophisticated, good clinical practice should be for the microbiologist to report the results with comments to facilitate the clinician\u2019s interpretation of the significance of the data.[43] In our experience, after effective laboratory automation implementation, a closer relationship with clinicians is largely established, providing an opportunity to convey insight into microbiology and microbiological work processes to clinical staff. On the other hand, patients are increasingly complex and heterogeneous, and management of severe and MDRO infections is challenging, often requiring a multidisciplinary approach for optimal personalized diagnostics and therapy.[50] Therefore, to integrate DS with AS, microbiologists should broaden their knowledge of patient care by working closely with physicians.\nInformation from the microbiology laboratory is essential for the control and management of infections in a hospital. In particular, timely and accurate data on the antibiotic susceptibility profiles for pathogens isolated from different wards and on MDRO colonization\/infection are the basis for setting up hospital infection control and AS programs, which can ultimately affect patient outcomes. Unfortunately, laboratories are not always able to provide timely information due to lack of specific expertise, personnel, user-friendly software, and optimized workflow practices. The implementation of laboratory automation and laboratory informatics can support integration into routine practice monitoring specimens\u2019 quality, isolation of specific pathogens, alert reports for infection control practitioners, and real-time collection of lab trend data, all essential for the prevention and control of infections and epidemiological studies.\n\nConclusion \nIn conclusion, timely, accurate, and clinically relevant information is the basis for prevention and treatment of infections. Laboratory automation and laboratory informatics can greatly improve the accuracy of diagnostic procedures, TTR, and laboratory workflow. However, to exploit these technologies for the benefit of the patients, clinical microbiologists need to change their way of working\u2014according to a lean workflow and a patient-centered approach\u2014and their way of thinking, working more closely with clinical staff.\n\n Abbreviations, acronyms, and initialisms \nAI: artificial intelligence\nAR: antimicrobial resistance\nAS: antimicrobial stewardship\nAST: antimicrobial susceptibility testing\nBC: blood culture\nID: identification\nIT: information technology\nLIS: laboratory information system\nMALDI-TOF\/MS: matrix-assisted laser desorption\/ionization time-of-flight mass spectrometry\nMDR: multi-drug-resistant\nMDRO: multi-drug-resistant organism\nMRSA: methicillin-resistant Staphylococcus aureus\nNGS: next-generation sequencing\nQC: quality control\nQMS: quality management system\nTAT: turnaround time\nTLA: total laboratory automation\nTTR: time to report\nVRE: vancomycin-resistant enterococci\nAcknowledgements \nAuthor contributions \nStudy concept: AM, GD, EC. Critical revision of manuscript: PB, EP. Approval of manuscript: AM, GD, EC, PB, EP. All authors contributed to the article and approved the submitted version.\n\nData availability statement \nThe original contributions presented in the study are included in the article\/supplementary material. Further inquiries can be directed to the corresponding author.\n\nConflict of interest \nAM has received funds for speaking at a symposium organized on behalf of Becton\u2010Dickinson. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.\n\nReferences \n\n\n\u2191 O'Neill, J. (May 2016). \"Tackling drug-resistant infections globally: Final report and recommendations\" (PDF). Review on Antimicrobial Resistance. pp. 80. https:\/\/amr-review.org\/sites\/default\/files\/160525_Final%20paper_with%20cover.pdf .   \n \n\n\u2191 Levy, Mitchell M.; Evans, Laura E.; Rhodes, Andrew (1 June 2018). \"The Surviving Sepsis Campaign Bundle: 2018 update\" (in en). Intensive Care Medicine 44 (6): 925\u2013928. doi:10.1007\/s00134-018-5085-0. ISSN 0342-4642. http:\/\/link.springer.com\/10.1007\/s00134-018-5085-0 .   \n \n\n\u2191 Kumar, Anand; Ellis, Paul; Arabi, Yaseen; Roberts, Dan; Light, Bruce; Parrillo, Joseph E.; Dodek, Peter; Wood, Gordon et al. (1 November 2009). \"Initiation of Inappropriate Antimicrobial Therapy Results in a Fivefold Reduction of Survival in Human Septic Shock\" (in en). Chest 136 (5): 1237\u20131248. doi:10.1378\/chest.09-0087. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0012369209606796 .   \n \n\n\u2191 Zilberberg, Marya D.; Nathanson, Brian H.; Sulham, Kate; Fan, Weihong; Shorr, Andrew F. (1 December 2017). \"Carbapenem resistance, inappropriate empiric treatment and outcomes among patients hospitalized with Enterobacteriaceae urinary tract infection, pneumonia and sepsis\" (in en). BMC Infectious Diseases 17 (1): 279. doi:10.1186\/s12879-017-2383-z. ISSN 1471-2334. PMC PMC5393012. PMID 28415969. http:\/\/bmcinfectdis.biomedcentral.com\/articles\/10.1186\/s12879-017-2383-z .   \n \n\n\u2191 Raman, Gowri; Avendano, Esther; Berger, Samantha; Menon, Vandana (1 December 2015). \"Appropriate initial antibiotic therapy in hospitalized patients with gram-negative infections: systematic review and meta-analysis\" (in en). BMC Infectious Diseases 15 (1): 395. doi:10.1186\/s12879-015-1123-5. ISSN 1471-2334. PMC PMC4589179. PMID 26423743. http:\/\/bmcinfectdis.biomedcentral.com\/articles\/10.1186\/s12879-015-1123-5 .   \n \n\n\u2191 6.0 6.1 6.2 Trotter, Alexander J.; Aydin, Alp; Strinden, Michael J.; O\u2019Grady, Justin (1 October 2019). \"Recent and emerging technologies for the rapid diagnosis of infection and antimicrobial resistance\" (in en). Current Opinion in Microbiology 51: 39\u201345. doi:10.1016\/j.mib.2019.03.001. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1369527418301279 .   \n \n\n\u2191 Messacar, Kevin; Parker, Sarah K.; Todd, James K.; Dominguez, Samuel R. (1 March 2017). Kraft, Colleen Suzanne. ed. \"Implementation of Rapid Molecular Infectious Disease Diagnostics: the Role of Diagnostic and Antimicrobial Stewardship\" (in en). Journal of Clinical Microbiology 55 (3): 715\u2013723. doi:10.1128\/JCM.02264-16. ISSN 0095-1137. PMC PMC5328439. PMID 28031432. https:\/\/journals.asm.org\/doi\/10.1128\/JCM.02264-16 .   \n \n\n\u2191 8.0 8.1 Croxatto, A.; Prod'hom, G.; Faverjon, F.; Rochais, Y.; Greub, G. (1 March 2016). \"Laboratory automation in clinical bacteriology: what system to choose?\" (in en). Clinical Microbiology and Infection 22 (3): 217\u2013235. doi:10.1016\/j.cmi.2015.09.030. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X16000069 .   \n \n\n\u2191 9.0 9.1 Croxatto, Antony; Dijkstra, Klaas; Prod'hom, Guy; Greub, Gilbert (1 July 2015). Burnham, C. -A. D.. ed. \"Comparison of Inoculation with the InoqulA and WASP Automated Systems with Manual Inoculation\" (in en). Journal of Clinical Microbiology 53 (7): 2298\u20132307. doi:10.1128\/JCM.03076-14. ISSN 0095-1137. PMC PMC4473203. PMID 25972424. https:\/\/journals.asm.org\/doi\/10.1128\/JCM.03076-14 .   \n \n\n\u2191 10.0 10.1 Burckhardt, Irene (22 November 2018). \"Laboratory Automation in Clinical Microbiology\" (in en). Bioengineering 5 (4): 102. doi:10.3390\/bioengineering5040102. ISSN 2306-5354. PMC PMC6315553. PMID 30467275. http:\/\/www.mdpi.com\/2306-5354\/5\/4\/102 .   \n \n\n\u2191 Quiblier, Chantal; Jetter, Marion; Rominski, Mark; Mouttet, Forouhar; B\u00f6ttger, Erik C.; Keller, Peter M.; Hombach, Michael (1 March 2016). Onderdonk, A. B.. ed. \"Performance of Copan WASP for Routine Urine Microbiology\" (in en). Journal of Clinical Microbiology 54 (3): 585\u2013592. doi:10.1128\/JCM.02577-15. ISSN 0095-1137. PMC PMC4767997. PMID 26677255. https:\/\/journals.asm.org\/doi\/10.1128\/JCM.02577-15 .   \n \n\n\u2191 Iversen, Jesper; Stendal, Gitta; Gerdes, Cecilie M.; Meyer, Christian H.; Andersen, Christian \u00d8stergaard; Frimodt-M\u00f8ller, Niels (1 February 2016). Ledeboer, N. A.. ed. \"Comparative Evaluation of Inoculation of Urine Samples with the Copan WASP and BD Kiestra InoqulA Instruments\" (in en). Journal of Clinical Microbiology 54 (2): 328\u2013332. doi:10.1128\/JCM.01718-15. ISSN 0095-1137. PMC PMC4733172. PMID 26607980. https:\/\/journals.asm.org\/doi\/10.1128\/JCM.01718-15 .   \n \n\n\u2191 13.0 13.1 13.2 Croxatto, Antony; Marcelpoil, Rapha\u00ebl; Orny, C\u00e9drick; Morel, Didier; Prod'hom, Guy; Greub, Gilbert (1 December 2017). \"Towards automated detection, semi-quantification and identification of microbial growth in clinical bacteriology: A proof of concept\" (in en). Biomedical Journal 40 (6): 317\u2013328. doi:10.1016\/j.bj.2017.09.001. PMC PMC6138813. PMID 29433835. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2319417017302834 .   \n \n\n\u2191 Cheng, C.W.R.; Ong, C.H.; Chan, D.S.G. (1 September 2020). \"Impact of BD Kiestra InoqulA streaking patterns on colony isolation and turnaround time of methicillin-resistant Staphylococcus aureus and carbapenem-resistant Enterobacterale surveillance samples\" (in en). Clinical Microbiology and Infection 26 (9): 1201\u20131206. doi:10.1016\/j.cmi.2020.01.006. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X20300288 .   \n \n\n\u2191 15.0 15.1 Burckhardt, Irene; Last, Katharina; Zimmermann, Stefan (28 January 2019). \"Shorter Incubation Times for Detecting Multi-drug Resistant Bacteria in Patient Samples: Defining Early Imaging Time Points Using Growth Kinetics and Total Laboratory Automation\" (in en). Annals of Laboratory Medicine 39 (1): 43\u201349. doi:10.3343\/alm.2019.39.1.43. ISSN 2234-3806. PMC PMC6143461. PMID 30215229. http:\/\/annlabmed.org\/journal\/view.html?doi=10.3343\/alm.2019.39.1.43 .   \n \n\n\u2191 Moreno-Camacho, Jos\u00e9 L; Calva-Espinosa, Diana Y; Leal-Leyva, Yoseli Y; Elizalde-Olivas, Dolores C; Campos-Romero, Abraham; Alc\u00e1ntar-Fern\u00e1ndez, Jonathan (1 January 2018). \"Transformation From a Conventional Clinical Microbiology Laboratory to Full Automation\" (in en). Laboratory Medicine 49 (1): e1\u2013e8. doi:10.1093\/labmed\/lmx079. ISSN 0007-5027. https:\/\/academic.oup.com\/labmed\/article\/49\/1\/e1\/4743273 .   \n \n\n\u2191 17.0 17.1 Theparee, Talent; Das, Sanchita; Thomson, Richard B. (1 January 2018). Patel, Robin. ed. \"Total Laboratory Automation and Matrix-Assisted Laser Desorption Ionization\u2013Time of Flight Mass Spectrometry Improve Turnaround Times in the Clinical Microbiology Laboratory: a Retrospective Analysis\" (in en). Journal of Clinical Microbiology 56 (1): e01242\u201317. doi:10.1128\/JCM.01242-17. ISSN 0095-1137. PMC PMC5744220. PMID 29118171. https:\/\/journals.asm.org\/doi\/10.1128\/JCM.01242-17 .   \n \n\n\u2191 Cherkaoui, A.; Renzi, G.; Vuilleumier, N.; Schrenzel, J. (1 November 2019). \"Copan WASPLab automation significantly reduces incubation times and allows earlier culture readings\" (in en). Clinical Microbiology and Infection 25 (11): 1430.e5\u20131430.e12. doi:10.1016\/j.cmi.2019.04.001. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X19301521 .   \n \n\n\u2191 Thomson, Richard B.; McElvania, Erin (1 September 2019). \"Total Laboratory Automation\" (in en). Clinics in Laboratory Medicine 39 (3): 371\u2013389. doi:10.1016\/j.cll.2019.05.002. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0272271219300290 .   \n \n\n\u2191 20.0 20.1 Cherkaoui, Abdessalam; Schrenzel, Jacques (3 February 2022). \"Total Laboratory Automation for Rapid Detection and Identification of Microorganisms and Their Antimicrobial Resistance Profiles\". Frontiers in Cellular and Infection Microbiology 12: 807668. doi:10.3389\/fcimb.2022.807668. ISSN 2235-2988. PMC PMC8851030. PMID 35186794. https:\/\/www.frontiersin.org\/articles\/10.3389\/fcimb.2022.807668\/full .   \n \n\n\u2191 Cherkaoui, Abdessalam; Riat, Arnaud; Renzi, Gesuele; Fischer, Adrien; Schrenzel, Jacques (1 February 2023). \"Diagnostic test accuracy of an automated device for the MALDI target preparation for microbial identification\" (in en). European Journal of Clinical Microbiology & Infectious Diseases 42 (2): 153\u2013159. doi:10.1007\/s10096-022-04531-3. ISSN 0934-9723. PMC PMC9836989. PMID 36469165. https:\/\/link.springer.com\/10.1007\/s10096-022-04531-3 .   \n \n\n\u2191 22.0 22.1 22.2 Jacot, Damien; Sarton-Loh\u00e9ac, Garance; Coste, Alix T.; Bertelli, Claire; Greub, Gilbert; Prod'hom, Guy; Croxatto, Antony (1 August 2021). \"Performance evaluation of the Becton Dickinson Kiestra\u2122 IdentifA\/SusceptA\" (in en). Clinical Microbiology and Infection 27 (8): 1167.e9\u20131167.e17. doi:10.1016\/j.cmi.2020.09.050. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X20306042 .   \n \n\n\u2191 Cherkaoui, Abdessalam; Renzi, Gesuele; Vuilleumier, Nicolas; Schrenzel, Jacques (18 August 2021). McElvania, Erin. ed. \"Performance of Fully Automated Antimicrobial Disk Diffusion Susceptibility Testing Using Copan WASP Colibri Coupled to the Radian In-Line Carousel and Expert System\" (in en). Journal of Clinical Microbiology 59 (9): e00777\u201321. doi:10.1128\/JCM.00777-21. ISSN 0095-1137. PMC PMC8373016. PMID 34160274. https:\/\/journals.asm.org\/doi\/10.1128\/JCM.00777-21 .   \n \n\n\u2191 24.0 24.1 Faron, Matthew L.; Buchan, Blake W.; Vismara, Chiara; Lacchini, Carla; Bielli, Alessandra; Gesu, Giovanni; Liebregts, Theo; van Bree, Anita et al. (1 March 2016). Richter, S. S.. ed. \"Automated Scoring of Chromogenic Media for Detection of Methicillin-Resistant Staphylococcus aureus by Use of WASPLab Image Analysis Software\" (in en). Journal of Clinical Microbiology 54 (3): 620\u2013624. doi:10.1128\/JCM.02778-15. ISSN 0095-1137. PMC PMC4767952. PMID 26719443. https:\/\/journals.asm.org\/doi\/10.1128\/JCM.02778-15 .   \n \n\n\u2191 25.0 25.1 Faron, Matthew L.; Buchan, Blake W.; Coon, Christopher; Liebregts, Theo; van Bree, Anita; Jansz, Arjan R.; Soucy, Genevieve; Korver, John et al. (1 October 2016). Burnham, C.-A. D.. ed. \"Automatic Digital Analysis of Chromogenic Media for Vancomycin-Resistant-Enterococcus Screens Using Copan WASPLab\" (in en). Journal of Clinical Microbiology 54 (10): 2464\u20132469. doi:10.1128\/JCM.01040-16. ISSN 0095-1137. PMC PMC5035414. PMID 27413193. https:\/\/journals.asm.org\/doi\/10.1128\/JCM.01040-16 .   \n \n\n\u2191 Foschi, Claudio; Gaibani, Paolo; Lombardo, Donatella; Re, Maria Carla; Ambretti, Simone (1 June 2020). \"Rectal screening for carbapenemase-producing Enterobacteriaceae: a proposed workflow\" (in en). Journal of Global Antimicrobial Resistance 21: 86\u201390. doi:10.1016\/j.jgar.2019.10.012. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2213716519302668 .   \n \n\n\u2191 Dauwalder, O.; Landrieve, L.; Laurent, F.; de Montclos, M.; Vandenesch, F.; Lina, G. (1 March 2016). \"Does bacteriology laboratory automation reduce time to results and increase quality management?\" (in en). Clinical Microbiology and Infection 22 (3): 236\u2013243. doi:10.1016\/j.cmi.2015.10.037. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X15009787 .   \n \n\n\u2191 Tabak, Ying P.; Vankeepuram, Latha; Ye, Gang; Jeffers, Kay; Gupta, Vikas; Murray, Patrick R. (1 December 2018). Carroll, Karen C.. ed. \"Blood Culture Turnaround Time in U.S. Acute Care Hospitals and Implications for Laboratory Process Optimization\" (in en). Journal of Clinical Microbiology 56 (12): e00500\u201318. doi:10.1128\/JCM.00500-18. ISSN 0095-1137. PMC PMC6258864. PMID 30135230. https:\/\/journals.asm.org\/doi\/10.1128\/JCM.00500-18 .   \n \n\n\u2191 29.0 29.1 De Socio, Giuseppe Vittorio; Di Donato, Francesco; Paggi, Riccardo; Gabrielli, Chiara; Belati, Alessandra; Rizza, Giuseppe; Savoia, Martina; Repetto, Antonella et al. (1 December 2018). \"Laboratory automation reduces time to report of positive blood cultures and improves management of patients with bloodstream infection\" (in en). European Journal of Clinical Microbiology & Infectious Diseases 37 (12): 2313\u20132322. doi:10.1007\/s10096-018-3377-5. ISSN 0934-9723. http:\/\/link.springer.com\/10.1007\/s10096-018-3377-5 .   \n \n\n\u2191 Vandenberg, Olivier; Durand, G\u00e9raldine; Hallin, Marie; Diefenbach, Andreas; Gant, Vanya; Murray, Patrick; Kozlakidis, Zisis; van Belkum, Alex (18 March 2020). \"Consolidation of Clinical Microbiology Laboratories and Introduction of Transformative Technologies\" (in en). Clinical Microbiology Reviews 33 (2): e00057\u201319. doi:10.1128\/CMR.00057-19. ISSN 0893-8512. PMC PMC7048017. PMID 32102900. https:\/\/journals.asm.org\/doi\/10.1128\/CMR.00057-19 .   \n \n\n\u2191 Kulengowski, B.; Ribes, J.A.; Burgess, D.S. (1 January 2019). \"Polymyxin B Etest\u00ae compared with gold-standard broth microdilution in carbapenem-resistant Enterobacteriaceae exhibiting a wide range of polymyxin B MICs\" (in en). Clinical Microbiology and Infection 25 (1): 92\u201395. doi:10.1016\/j.cmi.2018.04.008. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X18303434 .   \n \n\n\u2191 Camarlinghi, Giulio; Parisio, Eva Maria; Antonelli, Alberto; Nardone, Maria; Coppi, Marco; Giani, Tommaso; Mattei, Romano; Rossolini, Gian Maria (1 January 2019). \"Discrepancies in fosfomycin susceptibility testing of KPC-producing Klebsiella pneumoniae with various commercial methods\" (in en). Diagnostic Microbiology and Infectious Disease 93 (1): 74\u201376. doi:10.1016\/j.diagmicrobio.2018.07.014. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0732889318302505 .   \n \n\n\u2191 Simner, Patricia J.; Patel, Robin (17 December 2020). Burnham, Carey-Ann D.. ed. \"Cefiderocol Antimicrobial Susceptibility Testing Considerations: the Achilles' Heel of the Trojan Horse?\" (in en). Journal of Clinical Microbiology 59 (1): e00951\u201320. doi:10.1128\/JCM.00951-20. ISSN 0095-1137. PMC PMC7771437. PMID 32727829. https:\/\/journals.asm.org\/doi\/10.1128\/JCM.00951-20 .   \n \n\n\u2191 Antonelli, Alberto; Giani, Tommaso; Di Pilato, Vincenzo; Riccobono, Eleonora; Perriello, Gabriele; Mencacci, Antonella; Rossolini, Gian Maria (1 August 2019). \"KPC-31 expressed in a ceftazidime\/avibactam-resistant Klebsiella pneumoniae is associated with relevant detection issues\" (in en). Journal of Antimicrobial Chemotherapy 74 (8): 2464\u20132466. doi:10.1093\/jac\/dkz156. ISSN 0305-7453. https:\/\/academic.oup.com\/jac\/article\/74\/8\/2464\/5477394 .   \n \n\n\u2191 35.0 35.1 De Socio, Giuseppe Vittorio; Rubbioni, Paola; Botta, Daniele; Cenci, Elio; Belati, Alessandra; Paggi, Riccardo; Pasticci, Maria Bruna; Mencacci, Antonella (1 December 2019). \"Measurement and prediction of antimicrobial resistance in bloodstream infections by ESKAPE pathogens and Escherichia coli\" (in en). Journal of Global Antimicrobial Resistance 19: 154\u2013160. doi:10.1016\/j.jgar.2019.05.013. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2213716519301237 .   \n \n\n\u2191 Culbreath, Karissa; Piwonka, Heather; Korver, John; Noorbakhsh, Mir (18 February 2021). McElvania, Erin. ed. \"Benefits Derived from Full Laboratory Automation in Microbiology: a Tale of Four Laboratories\" (in en). Journal of Clinical Microbiology 59 (3): e01969\u201320. doi:10.1128\/JCM.01969-20. ISSN 0095-1137. PMC PMC8106725. PMID 33239383. https:\/\/journals.asm.org\/doi\/10.1128\/JCM.01969-20 .   \n \n\n\u2191 37.0 37.1 Zimmermann, Stefan (18 February 2021). McElvania, Erin. ed. \"Laboratory Automation in the Microbiology Laboratory: an Ongoing Journey, Not a Tale?\" (in en). Journal of Clinical Microbiology 59 (3): e02592\u201320. doi:10.1128\/JCM.02592-20. ISSN 0095-1137. PMC PMC8106703. PMID 33361341. https:\/\/journals.asm.org\/doi\/10.1128\/JCM.02592-20 .   \n \n\n\u2191 Womack, James P.; Jones, Daniel T.; Roos, Daniel (1990). The machine that changed the world: the story of lean production - Toyota\u00b4s secret weapon in the global car wars that is revolutionizing world industry (1. paperback ed ed.). London: Free Press. ISBN 978-0-7432-9979-4.   \n \n\n\u2191 Yarbrough, Melanie L.; Lainhart, William; McMullen, Allison R.; Anderson, Neil W.; Burnham, Carey-Ann D. (1 December 2018). \"Impact of total laboratory automation on workflow and specimen processing time for culture of urine specimens\" (in en). European Journal of Clinical Microbiology & Infectious Diseases 37 (12): 2405\u20132411. doi:10.1007\/s10096-018-3391-7. ISSN 0934-9723. http:\/\/link.springer.com\/10.1007\/s10096-018-3391-7 .   \n \n\n\u2191 Burckhardt, Irene; Horner, Susanne; Burckhardt, Florian; Zimmermann, Stefan (1 September 2018). \"Detection of MRSA in nasal swabs\u2014marked reduction of time to report for negative reports by substituting classical manual workflow with total lab automation\" (in en). European Journal of Clinical Microbiology & Infectious Diseases 37 (9): 1745\u20131751. doi:10.1007\/s10096-018-3308-5. ISSN 0934-9723. PMC PMC6133036. PMID 29943308. http:\/\/link.springer.com\/10.1007\/s10096-018-3308-5 .   \n \n\n\u2191 Seng, Piseth; Drancourt, Michel; Gouriet, Fr\u00e9d\u00e9rique; La Scola, Bernard; Fournier, Pierre\u2010Edouard; Rolain, Jean Marc; Raoult, Didier (15 August 2009). \"Ongoing Revolution in Bacteriology: Routine Identification of Bacteria by Matrix\u2010Assisted Laser Desorption Ionization Time\u2010of\u2010Flight Mass Spectrometry\" (in en). Clinical Infectious Diseases 49 (4): 543\u2013551. doi:10.1086\/600885. ISSN 1058-4838. https:\/\/academic.oup.com\/cid\/article-lookup\/doi\/10.1086\/600885 .   \n \n\n\u2191 Charnot-Katsikas, Angella; Tesic, Vera; Love, Nedra; Hill, Brandy; Bethel, Cindy; Boonlayangoor, Sue; Beavis, Kathleen G. (1 January 2018). Bourbeau, Paul. ed. \"Use of the Accelerate Pheno System for Identification and Antimicrobial Susceptibility Testing of Pathogens in Positive Blood Cultures and Impact on Time to Results and Workflow\" (in en). Journal of Clinical Microbiology 56 (1): e01166\u201317. doi:10.1128\/JCM.01166-17. ISSN 0095-1137. PMC PMC5744213. PMID 29118168. https:\/\/journals.asm.org\/doi\/10.1128\/JCM.01166-17 .   \n \n\n\u2191 43.0 43.1 Arena, Fabio; Giani, Tommaso; Pollini, Simona; Viaggi, Bruno; Pecile, Patrizia; Rossolini, Gian Maria (1 April 2017). \"Molecular antibiogram in diagnostic clinical microbiology: advantages and challenges\" (in en). Future Microbiology 12 (5): 361\u2013364. doi:10.2217\/fmb-2017-0019. ISSN 1746-0913. https:\/\/www.futuremedicine.com\/doi\/10.2217\/fmb-2017-0019 .   \n \n\n\u2191 Mitchell, Stephanie L.; Simner, Patricia J. (1 September 2019). \"Next-Generation Sequencing in Clinical Microbiology\" (in en). Clinics in Laboratory Medicine 39 (3): 405\u2013418. doi:10.1016\/j.cll.2019.05.003. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0272271219300307 .   \n \n\n\u2191 Pitashny, Milena; Kadry, Balqees; Shalaginov, Raya; Gazit, Liat; Zohar, Yaniv; Szwarcwort, Moran; Stabholz, Yoav; Paul, Mical (19 October 2022). \"NGS in the clinical microbiology settings\". Frontiers in Cellular and Infection Microbiology 12: 955481. doi:10.3389\/fcimb.2022.955481. ISSN 2235-2988. PMC PMC9627026. PMID 36339334. https:\/\/www.frontiersin.org\/articles\/10.3389\/fcimb.2022.955481\/full .   \n \n\n\u2191 Anandan, Shalini (2015). \"Rapid Screening for Carbapenem Resistant Organisms: Current Results and Future Approaches\". JOURNAL OF CLINICAL AND DIAGNOSTIC RESEARCH. doi:10.7860\/JCDR\/2015\/14246.6530. PMC PMC4606238. PMID 26500909. http:\/\/jcdr.net\/article_fulltext.asp?issn=0973-709x&year=2015&volume=9&issue=9&page=DM01&issn=0973-709x&id=6530 .   \n \n\n\u2191 47.0 47.1 Ambretti, Simone; Bassetti, Matteo; Clerici, Pierangelo; Petrosillo, Nicola; Tumietto, Fabio; Viale, Pierluigi; Rossolini, Gian Maria (1 December 2019). \"Screening for carriage of carbapenem-resistant Enterobacteriaceae in settings of high endemicity: a position paper from an Italian working group on CRE infections\" (in en). Antimicrobial Resistance & Infection Control 8 (1): 136. doi:10.1186\/s13756-019-0591-6. ISSN 2047-2994. PMC PMC6693230. PMID 31423299. https:\/\/aricjournal.biomedcentral.com\/articles\/10.1186\/s13756-019-0591-6 .   \n \n\n\u2191 Mangioni, Davide; Viaggi, Bruno; Giani, Tommaso; Arena, Fabio; D'Arienzo, Sara; Forni, Silvia; Tulli, Giorgio; Rossolini, Gian M (1 February 2019). \"Diagnostic stewardship for sepsis: the need for risk stratification to triage patients for fast microbiology workflows\" (in en). Future Microbiology 14 (3): 169\u2013174. doi:10.2217\/fmb-2018-0329. ISSN 1746-0913. https:\/\/www.futuremedicine.com\/doi\/10.2217\/fmb-2018-0329 .   \n \n\n\u2191 Skodvin, Brita; Aase, Karina; Brekken, Anita L\u00f8v\u00e5s; Charani, Esmita; Lindemann, Paul Christoffer; Smith, Ingrid (1 September 2017). \"Addressing the key communication barriers between microbiology laboratories and clinical units: a qualitative study\" (in en). Journal of Antimicrobial Chemotherapy 72 (9): 2666\u20132672. doi:10.1093\/jac\/dkx163. ISSN 0305-7453. PMC PMC5890706. PMID 28633405. https:\/\/academic.oup.com\/jac\/article\/72\/9\/2666\/3867670 .   \n \n\n\u2191 Tiseo, Giusy; Brigante, Gioconda; Giacobbe, Daniele Roberto; Maraolo, Alberto Enrico; Gona, Floriana; Falcone, Marco; Giannella, Maddalena; Grossi, Paolo et al. (1 August 2022). \"Diagnosis and management of infections caused by multidrug-resistant bacteria: guideline endorsed by the Italian Society of Infection and Tropical Diseases (SIMIT), the Italian Society of Anti-Infective Therapy (SITA), the Italian Group for Antimicrobial Stewardship (GISA), the Italian Association of Clinical Microbiologists (AMCLI) and the Italian Society of Microbiology (SIM)\" (in en). International Journal of Antimicrobial Agents 60 (2): 106611. doi:10.1016\/j.ijantimicag.2022.106611. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0924857922001236 .   \n \n\n\nNotes \nThis presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added. The original article lists references alphabetically; they are listed by order of appearance for this version, by design.\n\n\n\n\n\nSource: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.limswiki.org\/index.php\/Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\">https:\/\/www.limswiki.org\/index.php\/Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology<\/a>\nCategories: LIMSwiki journal articles (added in 2023)LIMSwiki journal articles (all)LIMSwiki journal articles on artificial intelligenceLIMSwiki journal articles on laboratory informaticsLIMSwiki journal articles on laboratory managementNavigation menuPage actionsJournalDiscussionView sourceHistoryPage actionsJournalDiscussionMoreToolsIn other languagesPersonal toolsLog inNavigationMain pageEncyclopedic articlesRecent changesRandom pageHelp about MediaWikiSearch\u00a0 ToolsWhat links hereRelated changesSpecial pagesPermanent linkPage informationPopular publications\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\nPrint\/exportCreate a bookDownload as PDFDownload as PDFDownload as Plain textPrintable version This page was last edited on 1 August 2023, at 21:39.Content is available under a Creative Commons Attribution-ShareAlike 4.0 International License unless otherwise noted.This page has been accessed 555 times.Privacy policyAbout LIMSWikiDisclaimers\n\n\n\n","49083cfbd81897f5701da971794583e5_html":"<body class=\"mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-206 ns-subject page-Journal_Laboratory_automation_informatics_and_artificial_intelligence_Current_and_future_perspectives_in_clinical_microbiology rootpage-Journal_Laboratory_automation_informatics_and_artificial_intelligence_Current_and_future_perspectives_in_clinical_microbiology skin-monobook action-view skin--responsive\"><div id=\"rdp-ebb-globalWrapper\"><div id=\"rdp-ebb-column-content\"><div id=\"rdp-ebb-content\" class=\"mw-body\" role=\"main\"><a id=\"rdp-ebb-top\"><\/a>\n<h1 id=\"rdp-ebb-firstHeading\" class=\"firstHeading\" lang=\"en\">Journal:Laboratory automation, informatics, and artificial intelligence: Current and future perspectives in clinical microbiology<\/h1><div id=\"rdp-ebb-bodyContent\" class=\"mw-body-content\"><!-- start content --><div id=\"rdp-ebb-mw-content-text\" lang=\"en\" dir=\"ltr\" class=\"mw-content-ltr\"><div class=\"mw-parser-output\">\n\n\n<h2><span class=\"mw-headline\" id=\"Abstract\">Abstract<\/span><\/h2>\n<p><a href=\"https:\/\/www.limswiki.org\/index.php\/Clinical_laboratory\" title=\"Clinical laboratory\" class=\"wiki-link\" data-key=\"307bcdf1bdbcd1bb167cee435b7a5463\">Clinical diagnostic laboratories<\/a> produce one product\u2014<a href=\"https:\/\/www.limswiki.org\/index.php\/Information\" title=\"Information\" class=\"wiki-link\" data-key=\"6300a14d9c2776dcca0999b5ed940e7d\">information<\/a>\u2014and for this to be valuable, the information must be clinically relevant, accurate, and timely. Although diagnostic information can clearly improve patient outcomes and decrease healthcare costs, technological challenges and <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory\" title=\"Laboratory\" class=\"wiki-link\" data-key=\"c57fc5aac9e4abf31dccae81df664c33\">laboratory<\/a> <a href=\"https:\/\/www.limswiki.org\/index.php\/Workflow\" title=\"Workflow\" class=\"wiki-link\" data-key=\"92bd8748272e20d891008dcb8243e8a8\">workflow<\/a> practices affect the timeliness and clinical value of diagnostics. This article will examine how prioritizing laboratory practices in a patient-oriented approach can be used to optimize technology advances for improved patient care.\n<\/p><p><b>Keywords<\/b>: laboratory automation, artificial intelligence, informatics, laboratory workflow, Kiestra, WASPLab\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Introduction\">Introduction<\/span><\/h2>\n<p>Patterns of <a href=\"https:\/\/www.limswiki.org\/index.php\/Infection\" title=\"Infection\" class=\"wiki-link\" data-key=\"f582685a10c11067cf15ce7799b4541a\">infectious diseases<\/a> have changed dramatically: patients are frequently immunocompromised and often have complicating comorbidities; infections with multi-drug-resistant organisms (MDRO) are a global problem; and new antibiotics are available, but it is mandatory to preserve their efficacy. It is estimated that at least 700,000 people die worldwide every year with infections caused by MDRO, and it is predicted that by 2050, 10 million deaths might occur due to these organisms.<sup id=\"rdp-ebb-cite_ref-1\" class=\"reference\"><a href=\"#cite_note-1\">[1]<\/a><\/sup> Administration of rapid, broad-spectrum empiric therapy is essential to improve patient outcome<sup id=\"rdp-ebb-cite_ref-2\" class=\"reference\"><a href=\"#cite_note-2\">[2]<\/a><\/sup>, but this is often inappropriate.<sup id=\"rdp-ebb-cite_ref-3\" class=\"reference\"><a href=\"#cite_note-3\">[3]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-4\" class=\"reference\"><a href=\"#cite_note-4\">[4]<\/a><\/sup> For example, meta-analysis assessing the impact of antibiotic therapy on Gram-negative sepsis showed that inappropriate therapy was associated with 3.3-fold increased risk of mortality, longer hospitalization, and higher costs.<sup id=\"rdp-ebb-cite_ref-5\" class=\"reference\"><a href=\"#cite_note-5\">[5]<\/a><\/sup> Thus, rapid, accurate diagnostics are critical for the selection of the most appropriate therapy.\n<\/p><p>Advanced, sophisticated technologies such as <a href=\"https:\/\/www.limswiki.org\/index.php\/Mass_spectrometry\" title=\"Mass spectrometry\" class=\"wiki-link\" data-key=\"fb548eafe2596c35d7ea741849aa83d4\">mass spectrometry<\/a> and <a href=\"https:\/\/www.limswiki.org\/index.php\/Molecular_diagnostics\" title=\"Molecular diagnostics\" class=\"wiki-link\" data-key=\"8fc14cae7a6fbac9a53fae1394fae7ee\">molecular diagnostics<\/a> are rapidly changing our ability to diagnose infections<sup id=\"rdp-ebb-cite_ref-:0_6-0\" class=\"reference\"><a href=\"#cite_note-:0-6\">[6]<\/a><\/sup>, although they should be viewed as complementary to traditional growth-based diagnostics. <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory_automation\" title=\"Laboratory automation\" class=\"wiki-link\" data-key=\"0061880849aeaca05f8aa27ae171f331\">Laboratory automation<\/a> and intelligent applications of of <a href=\"https:\/\/www.limswiki.org\/index.php\/Informatics_(academic_field)\" title=\"Informatics (academic field)\" class=\"wiki-link\" data-key=\"0391318826a5d9f9a1a1bcc88394739f\">informatics<\/a> also have a transformative impact of <a href=\"https:\/\/www.limswiki.org\/index.php\/Microbiology\" title=\"Microbiology\" class=\"wiki-link\" data-key=\"920bf32dc9c9cf492c58c4c5484df41f\">microbiology<\/a> diagnostics. These tools have the potential to accelerate clinical decision-making and positively impact the management of infections, improve patient outcome, and facilitate diagnostic and antimicrobial stewardship (AS) programs.<sup id=\"rdp-ebb-cite_ref-7\" class=\"reference\"><a href=\"#cite_note-7\">[7]<\/a><\/sup> However, it is a challenge for clinical microbiologists to implement these technologies because it requires changing well-established <a href=\"https:\/\/www.limswiki.org\/index.php\/Workflow\" title=\"Workflow\" class=\"wiki-link\" data-key=\"92bd8748272e20d891008dcb8243e8a8\">workflow<\/a> practices. This paper will focus on the impact of automation and informatics combined with workflow changes on laboratory, patient, and <a href=\"https:\/\/www.limswiki.org\/index.php\/Hospital\" title=\"Hospital\" class=\"wiki-link\" data-key=\"b8f070c66d8123fe91063594befebdff\">hospital<\/a> management (Figure 1).\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig1_Mencacci_FrontCellInfectMicro2023_13.jpg\" class=\"image wiki-link\" data-key=\"ff63349f8b3b812df8c99efc1ae8cd0d\"><img alt=\"Fig1 Mencacci FrontCellInfectMicro2023 13.jpg\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/6\/66\/Fig1_Mencacci_FrontCellInfectMicro2023_13.jpg\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 1.<\/b> Impact of automation on laboratory, patient, and hospital management.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h2><span class=\"mw-headline\" id=\"Impact_on_laboratory_management\">Impact on laboratory management<\/span><\/h2>\n<p>In clinical microbiology, the term \u201ctotal laboratory automation\u201d (TLA) is used to describe the automation of the entire diagnostic workflow, from inoculation of the agar plates to incubation, reading of culture results, identification (ID), and <a href=\"https:\/\/www.limswiki.org\/index.php\/Antibiotic_sensitivity_testing\" title=\"Antibiotic sensitivity testing\" class=\"wiki-link\" data-key=\"dd08251f4fb1bc9d6e137494c4cf799c\">antimicrobial susceptibility testing<\/a> (AST). All these steps in a conventional laboratory are performed manually, usually according to a <a href=\"https:\/\/www.limswiki.org\/index.php\/Sample_(material)\" title=\"Sample (material)\" class=\"wiki-link\" data-key=\"7f8cd41a077a88d02370c02a3ba3d9d6\">sample<\/a>-centered approach. At present, two laboratory automation systems are available: the BD Kiestra system (Becton Dickinson, Sparks, MD) and the WASPLab system (Copan Diagnostics, Murrieta, CA).<sup id=\"rdp-ebb-cite_ref-:1_8-0\" class=\"reference\"><a href=\"#cite_note-:1-8\">[8]<\/a><\/sup> We discuss these systems within the context of this workflow and then address other laboratory automation tools, as well <a href=\"https:\/\/www.limswiki.org\/index.php\/Artificial_intelligence\" title=\"Artificial intelligence\" class=\"wiki-link\" data-key=\"0c45a597361ca47e1cd8112af676276e\">artificial intelligence<\/a> (AI).\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Inoculation\">Inoculation<\/span><\/h3>\n<p>Quality and precision of inoculation are improved by automation. Instruments work in a standardized and consistent mode, not achievable with a manual procedure, and independent of operator variability. Indeed, laboratory automation allows better isolation of colonies compared to manual inoculation, with decreased need of subcultures for follow-up work, mainly AST, resulting in a more rapid report.<sup id=\"rdp-ebb-cite_ref-:2_9-0\" class=\"reference\"><a href=\"#cite_note-:2-9\">[9]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:3_10-0\" class=\"reference\"><a href=\"#cite_note-:3-10\">[10]<\/a><\/sup> It was found that WASP automated streaking of urines using a sterile loop was superior to manual streaking, yielding a higher number of single colonies and of detected morphologies, species, and pathogens.<sup id=\"rdp-ebb-cite_ref-11\" class=\"reference\"><a href=\"#cite_note-11\">[11]<\/a><\/sup> The BD Kiestra system, based on a rolling magnetic bead streaking technology, has been shown to improve the accuracy of quantitative culture results and the recovery of discrete colonies from polymicrobial samples, compared to manual and automated WASP streaking.<sup id=\"rdp-ebb-cite_ref-:2_9-1\" class=\"reference\"><a href=\"#cite_note-:2-9\">[9]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-12\" class=\"reference\"><a href=\"#cite_note-12\">[12]<\/a><\/sup> This implies a reduction of bacterial subcultures to perform ID and AST, thus shortening time to results, as evidenced for urines<sup id=\"rdp-ebb-cite_ref-:4_13-0\" class=\"reference\"><a href=\"#cite_note-:4-13\">[13]<\/a><\/sup> and both methicillin-resistant <i>Staphylococcus aureus<\/i> (MRSA) and carbapenem-resistant <i>Enterobacterales<\/i> screening samples.<sup id=\"rdp-ebb-cite_ref-14\" class=\"reference\"><a href=\"#cite_note-14\">[14]<\/a><\/sup>\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Incubation\">Incubation<\/span><\/h3>\n<p>Closed <a href=\"https:\/\/www.limswiki.org\/index.php\/Incubator_(culture)\" title=\"Incubator (culture)\" class=\"wiki-link\" data-key=\"a8cab45782e204a6c0b0843d3ae273d9\">incubators<\/a> with digital imaging of cultures allow more rapid growth than conventional incubators that are opened frequently throughout the day. Moreover, in TLA, plates are fully tracked as long as they stay within the system, so that it is possible to define by hours and minutes incubation times and plate examination, in contrast with the traditional system in which incubation times are defined in days. Burckhardt <i>et al.<\/i> showed that first growth of MRSA, multi-drug-resistant (MDR) Gram-negative bacteria, and vancomycin-resistant enterococci (VRE) on selective chromogenic plates was visible as early as after four hours of inoculation, although the bacterial mass was not sufficient for follow-up work.<sup id=\"rdp-ebb-cite_ref-:5_15-0\" class=\"reference\"><a href=\"#cite_note-:5-15\">[15]<\/a><\/sup> Also, growth of <i>Escherichia coli<\/i>, <i>Pseudomonas aeruginosa<\/i>, <i>Enterococcus faecalis<\/i>, and <i>S. aureus<\/i> on chromogenic plates was three to four hours faster in the automated system than in the classic system.<sup id=\"rdp-ebb-cite_ref-16\" class=\"reference\"><a href=\"#cite_note-16\">[16]<\/a><\/sup> Implementation of BD Kiestra TLA significantly improved turnaround times (TAT) for positive and negative urine cultures.<sup id=\"rdp-ebb-cite_ref-:6_17-0\" class=\"reference\"><a href=\"#cite_note-:6-17\">[17]<\/a><\/sup> Similarly, WASPLab automation enabled a reduction of the culture reading time for different specimens without affecting performances.<sup id=\"rdp-ebb-cite_ref-18\" class=\"reference\"><a href=\"#cite_note-18\">[18]<\/a><\/sup> However, minimum incubation times for each type of specimen, for a timely and accurate positive or negative report, are not yet defined, and additional studies are needed.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Reading\">Reading<\/span><\/h3>\n<p>The Kiestra laboratory automation system, through a real-time dashboard, times tasks as they are scheduled. Thus, each technician perfectly knows when the culture plates will be ready for reading and when follow-up work can be performed. This strongly facilitates laboratory workflow management, avoiding wasted time and allowing results to be delivered to the clinician as soon as possible. In addition, while in the classical system plates are read one by one, digital reading allows simultaneous viewing of all the plate images from the same sample, and even of different samples from the same patient. This greatly facilitates and speeds up the interpretation of culture results, either for monomicrobial or polymicrobial infections.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"ID_and_AST\">ID and AST<\/span><\/h3>\n<p>The implementation of TLA in clinical microbiology has leveraged the advancement brought by <a href=\"https:\/\/www.limswiki.org\/index.php\/Matrix-assisted_laser_desorption\/ionization\" title=\"Matrix-assisted laser desorption\/ionization\" class=\"wiki-link\" data-key=\"40fd846ed04445fcd70cf279f2971d29\">matrix-assisted laser desorption\/ionization<\/a> <a href=\"https:\/\/www.limswiki.org\/index.php\/Time-of-flight_mass_spectrometry\" title=\"Time-of-flight mass spectrometry\" class=\"wiki-link\" data-key=\"5bbed401afb889a1300d168c278f56f5\">time-of-flight mass spectrometry<\/a> (MALDI-TOF\/MS).<sup id=\"rdp-ebb-cite_ref-19\" class=\"reference\"><a href=\"#cite_note-19\">[19]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:7_20-0\" class=\"reference\"><a href=\"#cite_note-:7-20\">[20]<\/a><\/sup> Furthermore, Copan\u2019s TLA has recently integrated an automated device (Colibri) that can reproducibly prepare the MALDI target for microbial identification. A recent study conducted by Cherkaoui <i>et al.<\/i> established that the WASPLab coupled to MALDI-TOF\/MS significantly reduces the TAT for positive blood cultures.<sup id=\"rdp-ebb-cite_ref-21\" class=\"reference\"><a href=\"#cite_note-21\">[21]<\/a><\/sup> Similarly, the BD Kiestra IdentifA\/SusceptA, a prototype for automatic colony picking, bacterial suspension preparation, MALDI-TOF target plates spotting, and Phoenix M50 AST panel preparation, exhibited high ID and AST performances.<sup id=\"rdp-ebb-cite_ref-:8_22-0\" class=\"reference\"><a href=\"#cite_note-:8-22\">[22]<\/a><\/sup> In particular, the IdentifA showed excellent identification rates for Gram-negative bacteria, outperforming manual processing for <i>Enterobacterales<\/i> identification<sup id=\"rdp-ebb-cite_ref-:8_22-1\" class=\"reference\"><a href=\"#cite_note-:8-22\">[22]<\/a><\/sup>, but not for streptococci, coagulase-negative staphylococci (CoNS), and yeasts.<sup id=\"rdp-ebb-cite_ref-:8_22-2\" class=\"reference\"><a href=\"#cite_note-:8-22\">[22]<\/a><\/sup>\n<\/p><p>Finally, an automated solution for disk diffusion AST was developed and integrated with the Copan WASPLab system. It prepares inoculum suspensions, inoculates culture media plates, dispenses appropriate antibiotic disks according to predefined panels, transports the plates to the incubators, takes digitalized images of the media plates, and measures and interprets the inhibition zones\u2019 diameters. Cherkaoui <i>et al.<\/i>\u2014evaluating 718 bacterial strains, including <i>S. aureus<\/i>, CoNS, <i>E. faecalis<\/i>, <i>Enterococcus faecium<\/i>, <i>P. aeruginosa<\/i>, and different species of <i>Enterobacterales<\/i>\u2014found 99.1% overall categorical agreement between this automated AST and Vitek2.<sup id=\"rdp-ebb-cite_ref-23\" class=\"reference\"><a href=\"#cite_note-23\">[23]<\/a><\/sup>\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Artificial_intelligence\">Artificial intelligence<\/span><\/h3>\n<p>The development of intelligent image analysis based on tailored algorithms designed on type of specimens and patient characteristics allows automated detection of microbial growth, release of negative samples, presumptive ID, and quantification of bacterial colonies. This represents a major innovation that has the potential to increase laboratory quality and productivity while reducing TAT.<sup id=\"rdp-ebb-cite_ref-:4_13-1\" class=\"reference\"><a href=\"#cite_note-:4-13\">[13]<\/a><\/sup> Promising results have been obtained on urine samples, with a 97%\u201399% sensitivity and 85%\u201394% specificity by the BD Kiestra system.<sup id=\"rdp-ebb-cite_ref-:5_15-1\" class=\"reference\"><a href=\"#cite_note-:5-15\">[15]<\/a><\/sup> By a different approach, the WASPLab Chromogenic Detection Module has developed automated categorization of agar plates as \u201cnegative\u201d (i.e., sterile) or \u201cnon-negative,\u201d comparing the same plate at time point zero to the plate after the established incubation time. With this system, an optimal diagnostic accuracy in MRSA<sup id=\"rdp-ebb-cite_ref-:9_24-0\" class=\"reference\"><a href=\"#cite_note-:9-24\">[24]<\/a><\/sup>, VRE<sup id=\"rdp-ebb-cite_ref-:10_25-0\" class=\"reference\"><a href=\"#cite_note-:10-25\">[25]<\/a><\/sup>, and carbapenemase-producing Enterobacteriaceae<sup id=\"rdp-ebb-cite_ref-26\" class=\"reference\"><a href=\"#cite_note-26\">[26]<\/a><\/sup> detection has been observed.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Other_functions_of_laboratory_automation\">Other functions of laboratory automation<\/span><\/h3>\n<p>Laboratory automation can greatly facilitate the implementation of an effective <a href=\"https:\/\/www.limswiki.org\/index.php\/Quality_management_system\" title=\"Quality management system\" class=\"wiki-link\" data-key=\"dfecf3cd6f18d4a5e9ac49ca360b447d\">quality management system<\/a> (QMS), which is required to ensure that reliable results are reported for patients. Laboratory automation systems automatically track and record all the useful information for <a href=\"https:\/\/www.limswiki.org\/index.php\/Quality_control\" title=\"Quality control\" class=\"wiki-link\" data-key=\"1e0e0c2eb3e45aff02f5d61799821f0f\">quality control<\/a> (QC): user credentials, media (e.g., lot number and expiration date), inoculation (e.g., volumes of samples, patterns, and times of streaking), incubation (e.g., atmosphere, temperature, and times) and imaging (e.g., digital images of plates and times) data.<sup id=\"rdp-ebb-cite_ref-27\" class=\"reference\"><a href=\"#cite_note-27\">[27]<\/a><\/sup> Thus, the proper integration of laboratory automation with a <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory_information_system\" title=\"Laboratory information system\" class=\"wiki-link\" data-key=\"37add65b4d1c678b382a7d4817a9cf64\">laboratory information system<\/a> (LIS) allows for complete <a href=\"https:\/\/www.limswiki.org\/index.php\/Audit_trail\" title=\"Audit trail\" class=\"wiki-link\" data-key=\"96a617b543c5b2f26617288ba923c0f0\">traceability<\/a> of the analytical process, from sample receipt to the final report.\n<\/p><p>Moreover, the possibility to access and review any taken image represents an invaluable tool from a diagnostic point of view (e.g., comparing morphology of colonies in recent and old samples from the same patient) and also for other activities such as monitoring laboratory quality, teaching, training, and discussing culture results with colleagues and clinicians.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Impact_and_possible_improvements_of_laboratory_automation\">Impact and possible improvements of laboratory automation<\/span><\/h2>\n<h3><span class=\"mw-headline\" id=\"Impact_on_patient_management\">Impact on patient management<\/span><\/h3>\n<p>Clinical impact of an <a href=\"https:\/\/www.limswiki.org\/index.php\/Assay\" title=\"Assay\" class=\"wiki-link\" data-key=\"ea17cf4415e898e1838538495235ef71\">assay<\/a>, a technology, or a modified workflow can be defined based on its added value for patient management. In the case of sepsis, this can be measured as time to targeted therapy and, hopefully, a decrease in the mortality rate. In manual processing laboratories, the activities are performed in batches, usually based on the type of sample and type of activity (e.g., inoculation, reading, ID, AST, technical validation, and clinical validation), and the results are usually delivered mostly during the morning hours. Indeed, a study evaluating the TAT for positive blood cultures (BC) in 13 US acute care hospitals demonstrated a significant discrepancy between times of BC collection and reporting laboratory test results. While only 25% of specimens were collected between 6:00 a.m. and 11:59 a.m., approximately 80% of laboratory ID and AST results were reported in this time interval.<sup id=\"rdp-ebb-cite_ref-28\" class=\"reference\"><a href=\"#cite_note-28\">[28]<\/a><\/sup> This can have a negative impact on septic patient management, delaying clinical decision-making for optimal targeted therapy.\n<\/p><p>In contrast, in automated laboratories, the activity can be organized according to lean principles, creating a continuous \u201cflow\u201d and producing \u201cjust-in-time\u201d results. De Socio <i>et al.<\/i> evaluated the impact of laboratory automation on septic patient management. Positive BC were processed by fully automatic inoculation on solid media and digital reading after eight hours of incubation, followed by ID and AST. The authors found that a reduction of time to report (TTR) of about one day led to a significant reduction of the duration of empirical therapy (from approximately 87 hours\u2009to approximately 55 hours) and of 30-day crude mortality rate (from 29.0% to 16.7%).<sup id=\"rdp-ebb-cite_ref-:11_29-0\" class=\"reference\"><a href=\"#cite_note-:11-29\">[29]<\/a><\/sup>\n<\/p><p>Therefore, provided that the laboratory is open 24 hours a day, or taking advantage of telemedicine systems for clinical validation, laboratory automation has a potentially great impact on patient management. However, the success of such organization lies in the responsiveness of the medical teams, who should act upon the results soon after delivery by the laboratory.<sup id=\"rdp-ebb-cite_ref-30\" class=\"reference\"><a href=\"#cite_note-30\">[30]<\/a><\/sup>\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Impact_on_hospital_management\">Impact on hospital management<\/span><\/h3>\n<p>Laboratory automation can improve the laboratory's ability to characterize MDRO and produce quality results, permitting a more standardized workflow, while leaving more time for laboratory staff to focus on second-level phenotypic and\/or genotypic tests. Indeed, the large diffusion of MDRO and the expanding spectrum of resistance mechanisms among pathogens pointed out the limitations of commercial routine methods for susceptibility testing of selected antibiotics, increasing the demand for cumbersome and time-consuming reference methods. For example, in the case of MDRO Gram-negative isolates, colistin MIC should be determined by the broth microdilution method<sup id=\"rdp-ebb-cite_ref-31\" class=\"reference\"><a href=\"#cite_note-31\">[31]<\/a><\/sup>; fosfomycin MIC, by the agar-dilution method<sup id=\"rdp-ebb-cite_ref-32\" class=\"reference\"><a href=\"#cite_note-32\">[32]<\/a><\/sup>; and cefiderocol, a novel siderophore-conjugated cephalosporin, by the broth microdilution method using an iron-depleted cation-adjusted Mueller-Hinton broth.<sup id=\"rdp-ebb-cite_ref-33\" class=\"reference\"><a href=\"#cite_note-33\">[33]<\/a><\/sup> Moreover, in the case of detection of uncommon resistance phenotypes, molecular methods, gene <a href=\"https:\/\/www.limswiki.org\/index.php\/Sequencing\" class=\"mw-disambig wiki-link\" title=\"Sequencing\" data-key=\"e36167a9eb152ca16a0c4c4e6d13f323\">sequencing<\/a>, or other <a href=\"https:\/\/www.limswiki.org\/index.php\/Next-generation_sequencing\" class=\"mw-redirect wiki-link\" title=\"Next-generation sequencing\" data-key=\"c9d965c11eed1543f2a7e5f1abed4bb7\">next-generation sequencing<\/a> (NGS) methods are often required.<sup id=\"rdp-ebb-cite_ref-34\" class=\"reference\"><a href=\"#cite_note-34\">[34]<\/a><\/sup>\n<\/p><p>Accuracy is not sufficient per se for a result to be useful. Information must be given to clinicians or other healthcare providers (e.g., pharmacists and the patient\u2019s primary care nurse) as quickly as possible. Timely reporting can affect hospital conditions in at least two ways: permitting the rapid control of the spread of MDRO (i.e., contact precautions, investigation of clusters of colonized\/infected patients) and reducing the duration of broad-spectrum antibiotic therapy (i.e., positive results) or unnecessary empiric antibiotic therapy (i.e., negative results).\n<\/p><p>In a study proposing a cumulative antimicrobial resistance index as a tool to predict antimicrobial resistance (AR) trend in a hospital, a reversion of AR trend was observed in 2018, in comparison with the 2014\u20132017 period.<sup id=\"rdp-ebb-cite_ref-:12_35-0\" class=\"reference\"><a href=\"#cite_note-:12-35\">[35]<\/a><\/sup> The authors speculate that this could have been a consequence of some changes in the management of infections in their hospital: (i) incubation of all BC within one hour from collection using satellite incubators, (ii) a significant reduction in TTR after the introduction of molecular technologies and laboratory automation, and (iii) an established close collaboration between infectious disease clinicians and clinical microbiologists.<sup id=\"rdp-ebb-cite_ref-:12_35-1\" class=\"reference\"><a href=\"#cite_note-:12-35\">[35]<\/a><\/sup>\n<\/p><p>Finally, Culbreath <i>et al.<\/i> demonstrated that the implementation of TLA increased laboratory productivity by up to 90%, while reducing the cost per specimen by up to 47%, providing an excellent elaboration of the efficiencies and cost-savings that are achievable by implementation of full laboratory automation in the bacteriology laboratory.<sup id=\"rdp-ebb-cite_ref-36\" class=\"reference\"><a href=\"#cite_note-36\">[36]<\/a><\/sup>\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Possible_improvements_to_laboratory_automation_systems\">Possible improvements to laboratory automation systems<\/span><\/h3>\n<p>A detailed wish list of technical issues to be evaluated in order to improve the performance and workflow of laboratory automation systems has been recently published.<sup id=\"rdp-ebb-cite_ref-:3_10-1\" class=\"reference\"><a href=\"#cite_note-:3-10\">[10]<\/a><\/sup> Here, we will focus on facts that, in our opinion, could affect laboratory, patient, and hospital management.\n<\/p><p>To facilitate the reading of the plates according to a patient-centered approach, it would be useful to view specimens\u2019 Gram stains in the same screen of cultured plates. The images could also be shared with clinicians, improving clinician\u2013microbiologist interplay. Further improvement can be made by automated microscopy systems, which can significantly reduce the workload of the technical staff.<sup id=\"rdp-ebb-cite_ref-:13_37-0\" class=\"reference\"><a href=\"#cite_note-:13-37\">[37]<\/a><\/sup>\n<\/p><p>The availability of digital images lays the foundation for telebacteriology, intended as the use of digital imaging and file storage for on-screen reading and decision-making.<sup id=\"rdp-ebb-cite_ref-:1_8-1\" class=\"reference\"><a href=\"#cite_note-:1-8\">[8]<\/a><\/sup> It makes it possible to geographically dissociate plate manipulation from reading and validation of the results. This could promote the microbiologist counseling activity and interaction with clinicians, as the images could be shared between consultants located at different sites. Also, it could support 24\/7 laboratory activity, allowing the plates to be read outside the laboratory in a hub laboratory or even at home, with follow-up work performed in real time where the plates are incubated.\n<\/p><p>To make these technological innovations fully operational, a <a href=\"https:\/\/www.limswiki.org\/index.php\/Middleware\" title=\"Middleware\" class=\"wiki-link\" data-key=\"82ee1d9577571b4f9e4d83d6d6124c81\">middleware<\/a> information technology (IT) solution is needed to connect all the laboratory's instruments.<sup id=\"rdp-ebb-cite_ref-:13_37-1\" class=\"reference\"><a href=\"#cite_note-:13-37\">[37]<\/a><\/sup>\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Discussion\">Discussion<\/span><\/h2>\n<p>The main reason to introduce automation in a laboratory is to increase productivity in the face of limited budgets and personnel shortages. However, implementation of laboratory automation can represent an exceptional opportunity to change laboratory organization, improve quality, and reduce TTR, with a potential positive impact on laboratory, patient, and hospital management.\n<\/p><p>One of the most relevant innovations of laboratory automation regards the reading phase, with the possibility to read simultaneously all the plates inoculated from one of even more samples from the same patient. Moreover, taking advantage of informatics, it is also possible to view patient microbiological, hematological, and even clinical and therapeutic data while reading the plates. This patient-oriented approach provides meaningful clinical interpretation of results and decision-making.\n<\/p><p>By continuously tracing all the analytical steps, laboratory automation ensures that the microbiologist knows in real time the work to be carried out. This concept fully adheres to the so-called \u201c<a href=\"https:\/\/www.limswiki.org\/index.php\/Lean_laboratory\" title=\"Lean laboratory\" class=\"wiki-link\" data-key=\"6adad97006cb4eb4cac7dfcf767e3d5c\">lean<\/a>\u201d organization that, initially envisaged for industry<sup id=\"rdp-ebb-cite_ref-38\" class=\"reference\"><a href=\"#cite_note-38\">[38]<\/a><\/sup>], is increasingly applied to healthcare processes. \u201cLean\u201d means to do only valuable activities, without any delay, avoiding \u201cwaste\u201d or unnecessary work. This implies a dramatic revolution in the mentality of microbiologists, transitioning from exclusively sample-centered laboratory work towards a more clinically oriented activity, shortening TTR and prioritizing diagnosis of time-dependent infections. Taking advantage of workflow optimization, a nearly 24 hour reduction in TTR has been observed for positive BC processed by laboratory automation, with a significant decrease of duration of empirical therapy and mortality.<sup id=\"rdp-ebb-cite_ref-:11_29-1\" class=\"reference\"><a href=\"#cite_note-:11-29\">[29]<\/a><\/sup> Similar results were observed for urines<sup id=\"rdp-ebb-cite_ref-39\" class=\"reference\"><a href=\"#cite_note-39\">[39]<\/a><\/sup> and nasal MRSA surveillance<sup id=\"rdp-ebb-cite_ref-40\" class=\"reference\"><a href=\"#cite_note-40\">[40]<\/a><\/sup>, as well as other specimen types.<sup id=\"rdp-ebb-cite_ref-:4_13-2\" class=\"reference\"><a href=\"#cite_note-:4-13\">[13]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:6_17-1\" class=\"reference\"><a href=\"#cite_note-:6-17\">[17]<\/a><\/sup>\n<\/p><p>An AI algorithm to interpret culture results is another important tool applicable to laboratory automation: automated reporting of negative samples can be done without delay and further human assistance, such that clinicians can receive earlier results to rule out MDRO colonization or a urinary tract infection and reduce the need for patient isolation or antibiotic treatment.<sup id=\"rdp-ebb-cite_ref-:7_20-1\" class=\"reference\"><a href=\"#cite_note-:7-20\">[20]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:9_24-1\" class=\"reference\"><a href=\"#cite_note-:9-24\">[24]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:10_25-1\" class=\"reference\"><a href=\"#cite_note-:10-25\">[25]<\/a><\/sup>\n<\/p><p>Outside laboratory automation, a variety of technologies are revolutionizing clinical microbiology. These include MALDI-TOF\/MS<sup id=\"rdp-ebb-cite_ref-41\" class=\"reference\"><a href=\"#cite_note-41\">[41]<\/a><\/sup>, time-lapse microscopy for ID and phenotypic AST<sup id=\"rdp-ebb-cite_ref-42\" class=\"reference\"><a href=\"#cite_note-42\">[42]<\/a><\/sup>, molecular diagnostic tests and syndromic panels<sup id=\"rdp-ebb-cite_ref-:0_6-1\" class=\"reference\"><a href=\"#cite_note-:0-6\">[6]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:14_43-0\" class=\"reference\"><a href=\"#cite_note-:14-43\">[43]<\/a><\/sup>, and NGS.<sup id=\"rdp-ebb-cite_ref-44\" class=\"reference\"><a href=\"#cite_note-44\">[44]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-45\" class=\"reference\"><a href=\"#cite_note-45\">[45]<\/a><\/sup> All of them can significantly improve the diagnosis and therapy of infections, but as stated above, they are primarily complementary to culture-based methods.<sup id=\"rdp-ebb-cite_ref-:0_6-2\" class=\"reference\"><a href=\"#cite_note-:0-6\">[6]<\/a><\/sup> Thus, in an advanced laboratory, the goal will be to implement the use of all these technologies in a coordinated and timely program of diagnostic stewardship (DS). For example, for active surveillance of MDRO, both molecular- and culture-based methods should be available in the laboratory.<sup id=\"rdp-ebb-cite_ref-46\" class=\"reference\"><a href=\"#cite_note-46\">[46]<\/a><\/sup> Indeed, active surveillance of carbapenem-resistant Enterobacteriaceae can limit and prevent their spread and infections, which is crucially relevant to AS.<sup id=\"rdp-ebb-cite_ref-:15_47-0\" class=\"reference\"><a href=\"#cite_note-:15-47\">[47]<\/a><\/sup> In high-risk patients, rapid molecular methods are more appropriate but cannot replace culture-based methods, as the latter can detect all types of carbapenem-resistant organisms, perform phenotypic susceptibility testing, and collect and store the isolates.<sup id=\"rdp-ebb-cite_ref-:15_47-1\" class=\"reference\"><a href=\"#cite_note-:15-47\">[47]<\/a><\/sup>] An interesting algorithm\u2014based on a multi-parametric score that takes into account clinical, microbiological, and biochemical parameters\u2014has been recently proposed to establish patient priority, including information on infection or colonization by MDRO.<sup id=\"rdp-ebb-cite_ref-48\" class=\"reference\"><a href=\"#cite_note-48\">[48]<\/a><\/sup> It is reasonable to think that by combining DS and AS programs with a strict collaboration between laboratory and clinicians, the impact of modern microbiology on the management of infection can progressively increase.\n<\/p><p>In this vein, rapid and effective communication from laboratory to wards and back is essential for optimal patient care. A recent study showed that many barriers exist, like verbal reporting of results, poorly integrated information systems, mutual lack of insight into each other\u2019s area of expertise, and limited laboratory services.<sup id=\"rdp-ebb-cite_ref-49\" class=\"reference\"><a href=\"#cite_note-49\">[49]<\/a><\/sup> Electronic reporting improves communication between microbiologists and clinical staff, but a type of alert system for the right physician (i.e., the treating clinician, an infectious diseases specialist, or a sepsis team member) to look up the data immediately should be integrated. Nevertheless, we believe that direct microbiologist\/clinician interplay remains crucial for an optimal patient management: positive BC, detection of MDRO, isolation of alert organisms from sterile fluids, and acid-fast bacilli in respiratory samples must be immediately reported to someone who will act on the results.\n<\/p><p>Moreover, as microbiological methods become increasingly sophisticated, good clinical practice should be for the microbiologist to report the results with comments to facilitate the clinician\u2019s interpretation of the significance of the data.<sup id=\"rdp-ebb-cite_ref-:14_43-1\" class=\"reference\"><a href=\"#cite_note-:14-43\">[43]<\/a><\/sup> In our experience, after effective laboratory automation implementation, a closer relationship with clinicians is largely established, providing an opportunity to convey insight into microbiology and microbiological work processes to clinical staff. On the other hand, patients are increasingly complex and heterogeneous, and management of severe and MDRO infections is challenging, often requiring a multidisciplinary approach for optimal personalized diagnostics and therapy.<sup id=\"rdp-ebb-cite_ref-50\" class=\"reference\"><a href=\"#cite_note-50\">[50]<\/a><\/sup> Therefore, to integrate DS with AS, microbiologists should broaden their knowledge of patient care by working closely with physicians.\n<\/p><p>Information from the microbiology laboratory is essential for the control and management of infections in a hospital. In particular, timely and accurate data on the antibiotic susceptibility profiles for pathogens isolated from different wards and on MDRO colonization\/infection are the basis for setting up hospital infection control and AS programs, which can ultimately affect patient outcomes. Unfortunately, laboratories are not always able to provide timely information due to lack of specific expertise, personnel, user-friendly software, and optimized workflow practices. The implementation of laboratory automation and <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory_informatics\" title=\"Laboratory informatics\" class=\"wiki-link\" data-key=\"00edfa43edcde538a695f6d429280301\">laboratory informatics<\/a> can support integration into routine practice monitoring specimens\u2019 quality, isolation of specific pathogens, alert reports for infection control practitioners, and real-time collection of lab trend data, all essential for the prevention and control of infections and <a href=\"https:\/\/www.limswiki.org\/index.php\/Epidemiology\" title=\"Epidemiology\" class=\"wiki-link\" data-key=\"123badb8bf0b37a513182dbcfc3875bc\">epidemiological<\/a> studies.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Conclusion\">Conclusion<\/span><\/h2>\n<p>In conclusion, timely, accurate, and clinically relevant information is the basis for prevention and treatment of infections. Laboratory automation and laboratory informatics can greatly improve the accuracy of diagnostic procedures, TTR, and laboratory workflow. However, to exploit these technologies for the benefit of the patients, clinical microbiologists need to change their way of working\u2014according to a lean workflow and a patient-centered approach\u2014and their way of thinking, working more closely with clinical staff.\n<\/p>\n<h2><span id=\"rdp-ebb-Abbreviations,_acronyms,_and_initialisms\"><\/span><span class=\"mw-headline\" id=\"Abbreviations.2C_acronyms.2C_and_initialisms\">Abbreviations, acronyms, and initialisms<\/span><\/h2>\n<ul><li><b>AI<\/b>: artificial intelligence<\/li>\n<li><b>AR<\/b>: antimicrobial resistance<\/li>\n<li><b>AS<\/b>: antimicrobial stewardship<\/li>\n<li><b>AST<\/b>: antimicrobial susceptibility testing<\/li>\n<li><b>BC<\/b>: blood culture<\/li>\n<li><b>ID<\/b>: identification<\/li>\n<li><b>IT<\/b>: information technology<\/li>\n<li><b>LIS<\/b>: laboratory information system<\/li>\n<li><b>MALDI-TOF\/MS<\/b>: matrix-assisted laser desorption\/ionization time-of-flight mass spectrometry<\/li>\n<li><b>MDR<\/b>: multi-drug-resistant<\/li>\n<li><b>MDRO<\/b>: multi-drug-resistant organism<\/li>\n<li><b>MRSA<\/b>: methicillin-resistant <i>Staphylococcus aureus<\/i><\/li>\n<li><b>NGS<\/b>: next-generation sequencing<\/li>\n<li><b>QC<\/b>: quality control<\/li>\n<li><b>QMS<\/b>: quality management system<\/li>\n<li><b>TAT<\/b>: turnaround time<\/li>\n<li><b>TLA<\/b>: total laboratory automation<\/li>\n<li><b>TTR<\/b>: time to report<\/li>\n<li><b>VRE<\/b>: vancomycin-resistant enterococci<\/li><\/ul>\n<h2><span class=\"mw-headline\" id=\"Acknowledgements\">Acknowledgements<\/span><\/h2>\n<h3><span class=\"mw-headline\" id=\"Author_contributions\">Author contributions<\/span><\/h3>\n<p>Study concept: AM, GD, EC. Critical revision of manuscript: PB, EP. Approval of manuscript: AM, GD, EC, PB, EP. All authors contributed to the article and approved the submitted version.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Data_availability_statement\">Data availability statement<\/span><\/h3>\n<p>The original contributions presented in the study are included in the article\/supplementary material. Further inquiries can be directed to the corresponding author.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Conflict_of_interest\">Conflict of interest<\/span><\/h3>\n<p>AM has received funds for speaking at a symposium organized on behalf of Becton\u2010Dickinson. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"References\">References<\/span><\/h2>\n<div class=\"reflist references-column-width\" style=\"-moz-column-width: 30em; -webkit-column-width: 30em; column-width: 30em; list-style-type: decimal;\">\n<div class=\"mw-references-wrap mw-references-columns\"><ol class=\"references\">\n<li id=\"cite_note-1\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-1\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">O'Neill, J. (May 2016). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/amr-review.org\/sites\/default\/files\/160525_Final%20paper_with%20cover.pdf\" target=\"_blank\">\"Tackling drug-resistant infections globally: Final report and recommendations\"<\/a> (PDF). <i>Review on Antimicrobial Resistance<\/i>. pp. 80<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/amr-review.org\/sites\/default\/files\/160525_Final%20paper_with%20cover.pdf\" target=\"_blank\">https:\/\/amr-review.org\/sites\/default\/files\/160525_Final%20paper_with%20cover.pdf<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Tackling+drug-resistant+infections+globally%3A+Final+report+and+recommendations&rft.atitle=Review+on+Antimicrobial+Resistance&rft.aulast=O%27Neill%2C+J.&rft.au=O%27Neill%2C+J.&rft.date=May+2016&rft.pages=pp.+80&rft_id=https%3A%2F%2Famr-review.org%2Fsites%2Fdefault%2Ffiles%2F160525_Final%2520paper_with%2520cover.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-2\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-2\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Levy, Mitchell M.; Evans, Laura E.; Rhodes, Andrew (1 June 2018). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/s00134-018-5085-0\" target=\"_blank\">\"The Surviving Sepsis Campaign Bundle: 2018 update\"<\/a> (in en). <i>Intensive Care Medicine<\/i> <b>44<\/b> (6): 925\u2013928. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2Fs00134-018-5085-0\" target=\"_blank\">10.1007\/s00134-018-5085-0<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0342-4642\" target=\"_blank\">0342-4642<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/s00134-018-5085-0\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/s00134-018-5085-0<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+Surviving+Sepsis+Campaign+Bundle%3A+2018+update&rft.jtitle=Intensive+Care+Medicine&rft.aulast=Levy&rft.aufirst=Mitchell+M.&rft.au=Levy%2C%26%2332%3BMitchell+M.&rft.au=Evans%2C%26%2332%3BLaura+E.&rft.au=Rhodes%2C%26%2332%3BAndrew&rft.date=1+June+2018&rft.volume=44&rft.issue=6&rft.pages=925%E2%80%93928&rft_id=info:doi\/10.1007%2Fs00134-018-5085-0&rft.issn=0342-4642&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2Fs00134-018-5085-0&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-3\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-3\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Kumar, Anand; Ellis, Paul; Arabi, Yaseen; Roberts, Dan; Light, Bruce; Parrillo, Joseph E.; Dodek, Peter; Wood, Gordon <i>et al.<\/i> (1 November 2009). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0012369209606796\" target=\"_blank\">\"Initiation of Inappropriate Antimicrobial Therapy Results in a Fivefold Reduction of Survival in Human Septic Shock\"<\/a> (in en). <i>Chest<\/i> <b>136<\/b> (5): 1237\u20131248. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1378%2Fchest.09-0087\" target=\"_blank\">10.1378\/chest.09-0087<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0012369209606796\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0012369209606796<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Initiation+of+Inappropriate+Antimicrobial+Therapy+Results+in+a+Fivefold+Reduction+of+Survival+in+Human+Septic+Shock&rft.jtitle=Chest&rft.aulast=Kumar&rft.aufirst=Anand&rft.au=Kumar%2C%26%2332%3BAnand&rft.au=Ellis%2C%26%2332%3BPaul&rft.au=Arabi%2C%26%2332%3BYaseen&rft.au=Roberts%2C%26%2332%3BDan&rft.au=Light%2C%26%2332%3BBruce&rft.au=Parrillo%2C%26%2332%3BJoseph+E.&rft.au=Dodek%2C%26%2332%3BPeter&rft.au=Wood%2C%26%2332%3BGordon&rft.au=Kumar%2C%26%2332%3BAseem&rft.date=1+November+2009&rft.volume=136&rft.issue=5&rft.pages=1237%E2%80%931248&rft_id=info:doi\/10.1378%2Fchest.09-0087&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0012369209606796&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-4\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-4\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Zilberberg, Marya D.; Nathanson, Brian H.; Sulham, Kate; Fan, Weihong; Shorr, Andrew F. (1 December 2017). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/bmcinfectdis.biomedcentral.com\/articles\/10.1186\/s12879-017-2383-z\" target=\"_blank\">\"Carbapenem resistance, inappropriate empiric treatment and outcomes among patients hospitalized with Enterobacteriaceae urinary tract infection, pneumonia and sepsis\"<\/a> (in en). <i>BMC Infectious Diseases<\/i> <b>17<\/b> (1): 279. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1186%2Fs12879-017-2383-z\" target=\"_blank\">10.1186\/s12879-017-2383-z<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1471-2334\" target=\"_blank\">1471-2334<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC5393012\/\" target=\"_blank\">PMC5393012<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/28415969\" target=\"_blank\">28415969<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/bmcinfectdis.biomedcentral.com\/articles\/10.1186\/s12879-017-2383-z\" target=\"_blank\">http:\/\/bmcinfectdis.biomedcentral.com\/articles\/10.1186\/s12879-017-2383-z<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Carbapenem+resistance%2C+inappropriate+empiric+treatment+and+outcomes+among+patients+hospitalized+with+Enterobacteriaceae+urinary+tract+infection%2C+pneumonia+and+sepsis&rft.jtitle=BMC+Infectious+Diseases&rft.aulast=Zilberberg&rft.aufirst=Marya+D.&rft.au=Zilberberg%2C%26%2332%3BMarya+D.&rft.au=Nathanson%2C%26%2332%3BBrian+H.&rft.au=Sulham%2C%26%2332%3BKate&rft.au=Fan%2C%26%2332%3BWeihong&rft.au=Shorr%2C%26%2332%3BAndrew+F.&rft.date=1+December+2017&rft.volume=17&rft.issue=1&rft.pages=279&rft_id=info:doi\/10.1186%2Fs12879-017-2383-z&rft.issn=1471-2334&rft_id=info:pmc\/PMC5393012&rft_id=info:pmid\/28415969&rft_id=http%3A%2F%2Fbmcinfectdis.biomedcentral.com%2Farticles%2F10.1186%2Fs12879-017-2383-z&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-5\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-5\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Raman, Gowri; Avendano, Esther; Berger, Samantha; Menon, Vandana (1 December 2015). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/bmcinfectdis.biomedcentral.com\/articles\/10.1186\/s12879-015-1123-5\" target=\"_blank\">\"Appropriate initial antibiotic therapy in hospitalized patients with gram-negative infections: systematic review and meta-analysis\"<\/a> (in en). <i>BMC Infectious Diseases<\/i> <b>15<\/b> (1): 395. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1186%2Fs12879-015-1123-5\" target=\"_blank\">10.1186\/s12879-015-1123-5<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1471-2334\" target=\"_blank\">1471-2334<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC4589179\/\" target=\"_blank\">PMC4589179<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/26423743\" target=\"_blank\">26423743<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/bmcinfectdis.biomedcentral.com\/articles\/10.1186\/s12879-015-1123-5\" target=\"_blank\">http:\/\/bmcinfectdis.biomedcentral.com\/articles\/10.1186\/s12879-015-1123-5<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Appropriate+initial+antibiotic+therapy+in+hospitalized+patients+with+gram-negative+infections%3A+systematic+review+and+meta-analysis&rft.jtitle=BMC+Infectious+Diseases&rft.aulast=Raman&rft.aufirst=Gowri&rft.au=Raman%2C%26%2332%3BGowri&rft.au=Avendano%2C%26%2332%3BEsther&rft.au=Berger%2C%26%2332%3BSamantha&rft.au=Menon%2C%26%2332%3BVandana&rft.date=1+December+2015&rft.volume=15&rft.issue=1&rft.pages=395&rft_id=info:doi\/10.1186%2Fs12879-015-1123-5&rft.issn=1471-2334&rft_id=info:pmc\/PMC4589179&rft_id=info:pmid\/26423743&rft_id=http%3A%2F%2Fbmcinfectdis.biomedcentral.com%2Farticles%2F10.1186%2Fs12879-015-1123-5&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:0-6\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:0_6-0\">6.0<\/a><\/sup> <sup><a href=\"#cite_ref-:0_6-1\">6.1<\/a><\/sup> <sup><a href=\"#cite_ref-:0_6-2\">6.2<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Trotter, Alexander J.; Aydin, Alp; Strinden, Michael J.; O\u2019Grady, Justin (1 October 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1369527418301279\" target=\"_blank\">\"Recent and emerging technologies for the rapid diagnosis of infection and antimicrobial resistance\"<\/a> (in en). <i>Current Opinion in Microbiology<\/i> <b>51<\/b>: 39\u201345. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.mib.2019.03.001\" target=\"_blank\">10.1016\/j.mib.2019.03.001<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1369527418301279\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1369527418301279<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Recent+and+emerging+technologies+for+the+rapid+diagnosis+of+infection+and+antimicrobial+resistance&rft.jtitle=Current+Opinion+in+Microbiology&rft.aulast=Trotter&rft.aufirst=Alexander+J.&rft.au=Trotter%2C%26%2332%3BAlexander+J.&rft.au=Aydin%2C%26%2332%3BAlp&rft.au=Strinden%2C%26%2332%3BMichael+J.&rft.au=O%E2%80%99Grady%2C%26%2332%3BJustin&rft.date=1+October+2019&rft.volume=51&rft.pages=39%E2%80%9345&rft_id=info:doi\/10.1016%2Fj.mib.2019.03.001&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS1369527418301279&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-7\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-7\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Messacar, Kevin; Parker, Sarah K.; Todd, James K.; Dominguez, Samuel R. (1 March 2017). Kraft, Colleen Suzanne. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.02264-16\" target=\"_blank\">\"Implementation of Rapid Molecular Infectious Disease Diagnostics: the Role of Diagnostic and Antimicrobial Stewardship\"<\/a> (in en). <i>Journal of Clinical Microbiology<\/i> <b>55<\/b> (3): 715\u2013723. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1128%2FJCM.02264-16\" target=\"_blank\">10.1128\/JCM.02264-16<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0095-1137\" target=\"_blank\">0095-1137<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC5328439\/\" target=\"_blank\">PMC5328439<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/28031432\" target=\"_blank\">28031432<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.02264-16\" target=\"_blank\">https:\/\/journals.asm.org\/doi\/10.1128\/JCM.02264-16<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Implementation+of+Rapid+Molecular+Infectious+Disease+Diagnostics%3A+the+Role+of+Diagnostic+and+Antimicrobial+Stewardship&rft.jtitle=Journal+of+Clinical+Microbiology&rft.aulast=Messacar&rft.aufirst=Kevin&rft.au=Messacar%2C%26%2332%3BKevin&rft.au=Parker%2C%26%2332%3BSarah+K.&rft.au=Todd%2C%26%2332%3BJames+K.&rft.au=Dominguez%2C%26%2332%3BSamuel+R.&rft.date=1+March+2017&rft.volume=55&rft.issue=3&rft.pages=715%E2%80%93723&rft_id=info:doi\/10.1128%2FJCM.02264-16&rft.issn=0095-1137&rft_id=info:pmc\/PMC5328439&rft_id=info:pmid\/28031432&rft_id=https%3A%2F%2Fjournals.asm.org%2Fdoi%2F10.1128%2FJCM.02264-16&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:1-8\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:1_8-0\">8.0<\/a><\/sup> <sup><a href=\"#cite_ref-:1_8-1\">8.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Croxatto, A.; Prod'hom, G.; Faverjon, F.; Rochais, Y.; Greub, G. (1 March 2016). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X16000069\" target=\"_blank\">\"Laboratory automation in clinical bacteriology: what system to choose?\"<\/a> (in en). <i>Clinical Microbiology and Infection<\/i> <b>22<\/b> (3): 217\u2013235. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.cmi.2015.09.030\" target=\"_blank\">10.1016\/j.cmi.2015.09.030<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X16000069\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X16000069<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Laboratory+automation+in+clinical+bacteriology%3A+what+system+to+choose%3F&rft.jtitle=Clinical+Microbiology+and+Infection&rft.aulast=Croxatto&rft.aufirst=A.&rft.au=Croxatto%2C%26%2332%3BA.&rft.au=Prod%27hom%2C%26%2332%3BG.&rft.au=Faverjon%2C%26%2332%3BF.&rft.au=Rochais%2C%26%2332%3BY.&rft.au=Greub%2C%26%2332%3BG.&rft.date=1+March+2016&rft.volume=22&rft.issue=3&rft.pages=217%E2%80%93235&rft_id=info:doi\/10.1016%2Fj.cmi.2015.09.030&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS1198743X16000069&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:2-9\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:2_9-0\">9.0<\/a><\/sup> <sup><a href=\"#cite_ref-:2_9-1\">9.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Croxatto, Antony; Dijkstra, Klaas; Prod'hom, Guy; Greub, Gilbert (1 July 2015). Burnham, C. -A. D.. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.03076-14\" target=\"_blank\">\"Comparison of Inoculation with the InoqulA and WASP Automated Systems with Manual Inoculation\"<\/a> (in en). <i>Journal of Clinical Microbiology<\/i> <b>53<\/b> (7): 2298\u20132307. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1128%2FJCM.03076-14\" target=\"_blank\">10.1128\/JCM.03076-14<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0095-1137\" target=\"_blank\">0095-1137<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC4473203\/\" target=\"_blank\">PMC4473203<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/25972424\" target=\"_blank\">25972424<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.03076-14\" target=\"_blank\">https:\/\/journals.asm.org\/doi\/10.1128\/JCM.03076-14<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Comparison+of+Inoculation+with+the+InoqulA+and+WASP+Automated+Systems+with+Manual+Inoculation&rft.jtitle=Journal+of+Clinical+Microbiology&rft.aulast=Croxatto&rft.aufirst=Antony&rft.au=Croxatto%2C%26%2332%3BAntony&rft.au=Dijkstra%2C%26%2332%3BKlaas&rft.au=Prod%27hom%2C%26%2332%3BGuy&rft.au=Greub%2C%26%2332%3BGilbert&rft.date=1+July+2015&rft.volume=53&rft.issue=7&rft.pages=2298%E2%80%932307&rft_id=info:doi\/10.1128%2FJCM.03076-14&rft.issn=0095-1137&rft_id=info:pmc\/PMC4473203&rft_id=info:pmid\/25972424&rft_id=https%3A%2F%2Fjournals.asm.org%2Fdoi%2F10.1128%2FJCM.03076-14&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:3-10\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:3_10-0\">10.0<\/a><\/sup> <sup><a href=\"#cite_ref-:3_10-1\">10.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Burckhardt, Irene (22 November 2018). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.mdpi.com\/2306-5354\/5\/4\/102\" target=\"_blank\">\"Laboratory Automation in Clinical Microbiology\"<\/a> (in en). <i>Bioengineering<\/i> <b>5<\/b> (4): 102. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3390%2Fbioengineering5040102\" target=\"_blank\">10.3390\/bioengineering5040102<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2306-5354\" target=\"_blank\">2306-5354<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC6315553\/\" target=\"_blank\">PMC6315553<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/30467275\" target=\"_blank\">30467275<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.mdpi.com\/2306-5354\/5\/4\/102\" target=\"_blank\">http:\/\/www.mdpi.com\/2306-5354\/5\/4\/102<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Laboratory+Automation+in+Clinical+Microbiology&rft.jtitle=Bioengineering&rft.aulast=Burckhardt&rft.aufirst=Irene&rft.au=Burckhardt%2C%26%2332%3BIrene&rft.date=22+November+2018&rft.volume=5&rft.issue=4&rft.pages=102&rft_id=info:doi\/10.3390%2Fbioengineering5040102&rft.issn=2306-5354&rft_id=info:pmc\/PMC6315553&rft_id=info:pmid\/30467275&rft_id=http%3A%2F%2Fwww.mdpi.com%2F2306-5354%2F5%2F4%2F102&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-11\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-11\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Quiblier, Chantal; Jetter, Marion; Rominski, Mark; Mouttet, Forouhar; B\u00f6ttger, Erik C.; Keller, Peter M.; Hombach, Michael (1 March 2016). Onderdonk, A. B.. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.02577-15\" target=\"_blank\">\"Performance of Copan WASP for Routine Urine Microbiology\"<\/a> (in en). <i>Journal of Clinical Microbiology<\/i> <b>54<\/b> (3): 585\u2013592. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1128%2FJCM.02577-15\" target=\"_blank\">10.1128\/JCM.02577-15<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0095-1137\" target=\"_blank\">0095-1137<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC4767997\/\" target=\"_blank\">PMC4767997<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/26677255\" target=\"_blank\">26677255<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.02577-15\" target=\"_blank\">https:\/\/journals.asm.org\/doi\/10.1128\/JCM.02577-15<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Performance+of+Copan+WASP+for+Routine+Urine+Microbiology&rft.jtitle=Journal+of+Clinical+Microbiology&rft.aulast=Quiblier&rft.aufirst=Chantal&rft.au=Quiblier%2C%26%2332%3BChantal&rft.au=Jetter%2C%26%2332%3BMarion&rft.au=Rominski%2C%26%2332%3BMark&rft.au=Mouttet%2C%26%2332%3BForouhar&rft.au=B%C3%B6ttger%2C%26%2332%3BErik+C.&rft.au=Keller%2C%26%2332%3BPeter+M.&rft.au=Hombach%2C%26%2332%3BMichael&rft.date=1+March+2016&rft.volume=54&rft.issue=3&rft.pages=585%E2%80%93592&rft_id=info:doi\/10.1128%2FJCM.02577-15&rft.issn=0095-1137&rft_id=info:pmc\/PMC4767997&rft_id=info:pmid\/26677255&rft_id=https%3A%2F%2Fjournals.asm.org%2Fdoi%2F10.1128%2FJCM.02577-15&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-12\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-12\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Iversen, Jesper; Stendal, Gitta; Gerdes, Cecilie M.; Meyer, Christian H.; Andersen, Christian \u00d8stergaard; Frimodt-M\u00f8ller, Niels (1 February 2016). Ledeboer, N. A.. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.01718-15\" target=\"_blank\">\"Comparative Evaluation of Inoculation of Urine Samples with the Copan WASP and BD Kiestra InoqulA Instruments\"<\/a> (in en). <i>Journal of Clinical Microbiology<\/i> <b>54<\/b> (2): 328\u2013332. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1128%2FJCM.01718-15\" target=\"_blank\">10.1128\/JCM.01718-15<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0095-1137\" target=\"_blank\">0095-1137<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC4733172\/\" target=\"_blank\">PMC4733172<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/26607980\" target=\"_blank\">26607980<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.01718-15\" target=\"_blank\">https:\/\/journals.asm.org\/doi\/10.1128\/JCM.01718-15<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Comparative+Evaluation+of+Inoculation+of+Urine+Samples+with+the+Copan+WASP+and+BD+Kiestra+InoqulA+Instruments&rft.jtitle=Journal+of+Clinical+Microbiology&rft.aulast=Iversen&rft.aufirst=Jesper&rft.au=Iversen%2C%26%2332%3BJesper&rft.au=Stendal%2C%26%2332%3BGitta&rft.au=Gerdes%2C%26%2332%3BCecilie+M.&rft.au=Meyer%2C%26%2332%3BChristian+H.&rft.au=Andersen%2C%26%2332%3BChristian+%C3%98stergaard&rft.au=Frimodt-M%C3%B8ller%2C%26%2332%3BNiels&rft.date=1+February+2016&rft.volume=54&rft.issue=2&rft.pages=328%E2%80%93332&rft_id=info:doi\/10.1128%2FJCM.01718-15&rft.issn=0095-1137&rft_id=info:pmc\/PMC4733172&rft_id=info:pmid\/26607980&rft_id=https%3A%2F%2Fjournals.asm.org%2Fdoi%2F10.1128%2FJCM.01718-15&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:4-13\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:4_13-0\">13.0<\/a><\/sup> <sup><a href=\"#cite_ref-:4_13-1\">13.1<\/a><\/sup> <sup><a href=\"#cite_ref-:4_13-2\">13.2<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Croxatto, Antony; Marcelpoil, Rapha\u00ebl; Orny, C\u00e9drick; Morel, Didier; Prod'hom, Guy; Greub, Gilbert (1 December 2017). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2319417017302834\" target=\"_blank\">\"Towards automated detection, semi-quantification and identification of microbial growth in clinical bacteriology: A proof of concept\"<\/a> (in en). <i>Biomedical Journal<\/i> <b>40<\/b> (6): 317\u2013328. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.bj.2017.09.001\" target=\"_blank\">10.1016\/j.bj.2017.09.001<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC6138813\/\" target=\"_blank\">PMC6138813<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/29433835\" target=\"_blank\">29433835<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2319417017302834\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2319417017302834<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Towards+automated+detection%2C+semi-quantification+and+identification+of+microbial+growth+in+clinical+bacteriology%3A+A+proof+of+concept&rft.jtitle=Biomedical+Journal&rft.aulast=Croxatto&rft.aufirst=Antony&rft.au=Croxatto%2C%26%2332%3BAntony&rft.au=Marcelpoil%2C%26%2332%3BRapha%C3%ABl&rft.au=Orny%2C%26%2332%3BC%C3%A9drick&rft.au=Morel%2C%26%2332%3BDidier&rft.au=Prod%27hom%2C%26%2332%3BGuy&rft.au=Greub%2C%26%2332%3BGilbert&rft.date=1+December+2017&rft.volume=40&rft.issue=6&rft.pages=317%E2%80%93328&rft_id=info:doi\/10.1016%2Fj.bj.2017.09.001&rft_id=info:pmc\/PMC6138813&rft_id=info:pmid\/29433835&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2319417017302834&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-14\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-14\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Cheng, C.W.R.; Ong, C.H.; Chan, D.S.G. (1 September 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X20300288\" target=\"_blank\">\"Impact of BD Kiestra InoqulA streaking patterns on colony isolation and turnaround time of methicillin-resistant Staphylococcus aureus and carbapenem-resistant Enterobacterale surveillance samples\"<\/a> (in en). <i>Clinical Microbiology and Infection<\/i> <b>26<\/b> (9): 1201\u20131206. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.cmi.2020.01.006\" target=\"_blank\">10.1016\/j.cmi.2020.01.006<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X20300288\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X20300288<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Impact+of+BD+Kiestra+InoqulA+streaking+patterns+on+colony+isolation+and+turnaround+time+of+methicillin-resistant+Staphylococcus+aureus+and+carbapenem-resistant+Enterobacterale+surveillance+samples&rft.jtitle=Clinical+Microbiology+and+Infection&rft.aulast=Cheng&rft.aufirst=C.W.R.&rft.au=Cheng%2C%26%2332%3BC.W.R.&rft.au=Ong%2C%26%2332%3BC.H.&rft.au=Chan%2C%26%2332%3BD.S.G.&rft.date=1+September+2020&rft.volume=26&rft.issue=9&rft.pages=1201%E2%80%931206&rft_id=info:doi\/10.1016%2Fj.cmi.2020.01.006&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS1198743X20300288&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:5-15\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:5_15-0\">15.0<\/a><\/sup> <sup><a href=\"#cite_ref-:5_15-1\">15.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Burckhardt, Irene; Last, Katharina; Zimmermann, Stefan (28 January 2019). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/annlabmed.org\/journal\/view.html?doi=10.3343\/alm.2019.39.1.43\" target=\"_blank\">\"Shorter Incubation Times for Detecting Multi-drug Resistant Bacteria in Patient Samples: Defining Early Imaging Time Points Using Growth Kinetics and Total Laboratory Automation\"<\/a> (in en). <i>Annals of Laboratory Medicine<\/i> <b>39<\/b> (1): 43\u201349. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3343%2Falm.2019.39.1.43\" target=\"_blank\">10.3343\/alm.2019.39.1.43<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2234-3806\" target=\"_blank\">2234-3806<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC6143461\/\" target=\"_blank\">PMC6143461<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/30215229\" target=\"_blank\">30215229<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/annlabmed.org\/journal\/view.html?doi=10.3343\/alm.2019.39.1.43\" target=\"_blank\">http:\/\/annlabmed.org\/journal\/view.html?doi=10.3343\/alm.2019.39.1.43<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Shorter+Incubation+Times+for+Detecting+Multi-drug+Resistant+Bacteria+in+Patient+Samples%3A+Defining+Early+Imaging+Time+Points+Using+Growth+Kinetics+and+Total+Laboratory+Automation&rft.jtitle=Annals+of+Laboratory+Medicine&rft.aulast=Burckhardt&rft.aufirst=Irene&rft.au=Burckhardt%2C%26%2332%3BIrene&rft.au=Last%2C%26%2332%3BKatharina&rft.au=Zimmermann%2C%26%2332%3BStefan&rft.date=28+January+2019&rft.volume=39&rft.issue=1&rft.pages=43%E2%80%9349&rft_id=info:doi\/10.3343%2Falm.2019.39.1.43&rft.issn=2234-3806&rft_id=info:pmc\/PMC6143461&rft_id=info:pmid\/30215229&rft_id=http%3A%2F%2Fannlabmed.org%2Fjournal%2Fview.html%3Fdoi%3D10.3343%2Falm.2019.39.1.43&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-16\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-16\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Moreno-Camacho, Jos\u00e9 L; Calva-Espinosa, Diana Y; Leal-Leyva, Yoseli Y; Elizalde-Olivas, Dolores C; Campos-Romero, Abraham; Alc\u00e1ntar-Fern\u00e1ndez, Jonathan (1 January 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/academic.oup.com\/labmed\/article\/49\/1\/e1\/4743273\" target=\"_blank\">\"Transformation From a Conventional Clinical Microbiology Laboratory to Full Automation\"<\/a> (in en). <i>Laboratory Medicine<\/i> <b>49<\/b> (1): e1\u2013e8. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1093%2Flabmed%2Flmx079\" target=\"_blank\">10.1093\/labmed\/lmx079<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0007-5027\" target=\"_blank\">0007-5027<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/academic.oup.com\/labmed\/article\/49\/1\/e1\/4743273\" target=\"_blank\">https:\/\/academic.oup.com\/labmed\/article\/49\/1\/e1\/4743273<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Transformation+From+a+Conventional+Clinical+Microbiology+Laboratory+to+Full+Automation&rft.jtitle=Laboratory+Medicine&rft.aulast=Moreno-Camacho&rft.aufirst=Jos%C3%A9+L&rft.au=Moreno-Camacho%2C%26%2332%3BJos%C3%A9+L&rft.au=Calva-Espinosa%2C%26%2332%3BDiana+Y&rft.au=Leal-Leyva%2C%26%2332%3BYoseli+Y&rft.au=Elizalde-Olivas%2C%26%2332%3BDolores+C&rft.au=Campos-Romero%2C%26%2332%3BAbraham&rft.au=Alc%C3%A1ntar-Fern%C3%A1ndez%2C%26%2332%3BJonathan&rft.date=1+January+2018&rft.volume=49&rft.issue=1&rft.pages=e1%E2%80%93e8&rft_id=info:doi\/10.1093%2Flabmed%2Flmx079&rft.issn=0007-5027&rft_id=https%3A%2F%2Facademic.oup.com%2Flabmed%2Farticle%2F49%2F1%2Fe1%2F4743273&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:6-17\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:6_17-0\">17.0<\/a><\/sup> <sup><a href=\"#cite_ref-:6_17-1\">17.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Theparee, Talent; Das, Sanchita; Thomson, Richard B. (1 January 2018). Patel, Robin. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.01242-17\" target=\"_blank\">\"Total Laboratory Automation and Matrix-Assisted Laser Desorption Ionization\u2013Time of Flight Mass Spectrometry Improve Turnaround Times in the Clinical Microbiology Laboratory: a Retrospective Analysis\"<\/a> (in en). <i>Journal of Clinical Microbiology<\/i> <b>56<\/b> (1): e01242\u201317. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1128%2FJCM.01242-17\" target=\"_blank\">10.1128\/JCM.01242-17<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0095-1137\" target=\"_blank\">0095-1137<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC5744220\/\" target=\"_blank\">PMC5744220<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/29118171\" target=\"_blank\">29118171<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.01242-17\" target=\"_blank\">https:\/\/journals.asm.org\/doi\/10.1128\/JCM.01242-17<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Total+Laboratory+Automation+and+Matrix-Assisted+Laser+Desorption+Ionization%E2%80%93Time+of+Flight+Mass+Spectrometry+Improve+Turnaround+Times+in+the+Clinical+Microbiology+Laboratory%3A+a+Retrospective+Analysis&rft.jtitle=Journal+of+Clinical+Microbiology&rft.aulast=Theparee&rft.aufirst=Talent&rft.au=Theparee%2C%26%2332%3BTalent&rft.au=Das%2C%26%2332%3BSanchita&rft.au=Thomson%2C%26%2332%3BRichard+B.&rft.date=1+January+2018&rft.volume=56&rft.issue=1&rft.pages=e01242%E2%80%9317&rft_id=info:doi\/10.1128%2FJCM.01242-17&rft.issn=0095-1137&rft_id=info:pmc\/PMC5744220&rft_id=info:pmid\/29118171&rft_id=https%3A%2F%2Fjournals.asm.org%2Fdoi%2F10.1128%2FJCM.01242-17&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-18\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-18\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Cherkaoui, A.; Renzi, G.; Vuilleumier, N.; Schrenzel, J. (1 November 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X19301521\" target=\"_blank\">\"Copan WASPLab automation significantly reduces incubation times and allows earlier culture readings\"<\/a> (in en). <i>Clinical Microbiology and Infection<\/i> <b>25<\/b> (11): 1430.e5\u20131430.e12. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.cmi.2019.04.001\" target=\"_blank\">10.1016\/j.cmi.2019.04.001<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X19301521\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X19301521<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Copan+WASPLab+automation+significantly+reduces+incubation+times+and+allows+earlier+culture+readings&rft.jtitle=Clinical+Microbiology+and+Infection&rft.aulast=Cherkaoui&rft.aufirst=A.&rft.au=Cherkaoui%2C%26%2332%3BA.&rft.au=Renzi%2C%26%2332%3BG.&rft.au=Vuilleumier%2C%26%2332%3BN.&rft.au=Schrenzel%2C%26%2332%3BJ.&rft.date=1+November+2019&rft.volume=25&rft.issue=11&rft.pages=1430.e5%E2%80%931430.e12&rft_id=info:doi\/10.1016%2Fj.cmi.2019.04.001&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS1198743X19301521&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-19\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-19\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Thomson, Richard B.; McElvania, Erin (1 September 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0272271219300290\" target=\"_blank\">\"Total Laboratory Automation\"<\/a> (in en). <i>Clinics in Laboratory Medicine<\/i> <b>39<\/b> (3): 371\u2013389. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.cll.2019.05.002\" target=\"_blank\">10.1016\/j.cll.2019.05.002<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0272271219300290\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0272271219300290<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Total+Laboratory+Automation&rft.jtitle=Clinics+in+Laboratory+Medicine&rft.aulast=Thomson&rft.aufirst=Richard+B.&rft.au=Thomson%2C%26%2332%3BRichard+B.&rft.au=McElvania%2C%26%2332%3BErin&rft.date=1+September+2019&rft.volume=39&rft.issue=3&rft.pages=371%E2%80%93389&rft_id=info:doi\/10.1016%2Fj.cll.2019.05.002&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0272271219300290&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:7-20\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:7_20-0\">20.0<\/a><\/sup> <sup><a href=\"#cite_ref-:7_20-1\">20.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Cherkaoui, Abdessalam; Schrenzel, Jacques (3 February 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcimb.2022.807668\/full\" target=\"_blank\">\"Total Laboratory Automation for Rapid Detection and Identification of Microorganisms and Their Antimicrobial Resistance Profiles\"<\/a>. <i>Frontiers in Cellular and Infection Microbiology<\/i> <b>12<\/b>: 807668. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3389%2Ffcimb.2022.807668\" target=\"_blank\">10.3389\/fcimb.2022.807668<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2235-2988\" target=\"_blank\">2235-2988<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC8851030\/\" target=\"_blank\">PMC8851030<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/35186794\" target=\"_blank\">35186794<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcimb.2022.807668\/full\" target=\"_blank\">https:\/\/www.frontiersin.org\/articles\/10.3389\/fcimb.2022.807668\/full<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Total+Laboratory+Automation+for+Rapid+Detection+and+Identification+of+Microorganisms+and+Their+Antimicrobial+Resistance+Profiles&rft.jtitle=Frontiers+in+Cellular+and+Infection+Microbiology&rft.aulast=Cherkaoui&rft.aufirst=Abdessalam&rft.au=Cherkaoui%2C%26%2332%3BAbdessalam&rft.au=Schrenzel%2C%26%2332%3BJacques&rft.date=3+February+2022&rft.volume=12&rft.pages=807668&rft_id=info:doi\/10.3389%2Ffcimb.2022.807668&rft.issn=2235-2988&rft_id=info:pmc\/PMC8851030&rft_id=info:pmid\/35186794&rft_id=https%3A%2F%2Fwww.frontiersin.org%2Farticles%2F10.3389%2Ffcimb.2022.807668%2Ffull&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-21\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-21\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Cherkaoui, Abdessalam; Riat, Arnaud; Renzi, Gesuele; Fischer, Adrien; Schrenzel, Jacques (1 February 2023). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/link.springer.com\/10.1007\/s10096-022-04531-3\" target=\"_blank\">\"Diagnostic test accuracy of an automated device for the MALDI target preparation for microbial identification\"<\/a> (in en). <i>European Journal of Clinical Microbiology & Infectious Diseases<\/i> <b>42<\/b> (2): 153\u2013159. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2Fs10096-022-04531-3\" target=\"_blank\">10.1007\/s10096-022-04531-3<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0934-9723\" target=\"_blank\">0934-9723<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC9836989\/\" target=\"_blank\">PMC9836989<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/36469165\" target=\"_blank\">36469165<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/link.springer.com\/10.1007\/s10096-022-04531-3\" target=\"_blank\">https:\/\/link.springer.com\/10.1007\/s10096-022-04531-3<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Diagnostic+test+accuracy+of+an+automated+device+for+the+MALDI+target+preparation+for+microbial+identification&rft.jtitle=European+Journal+of+Clinical+Microbiology+%26+Infectious+Diseases&rft.aulast=Cherkaoui&rft.aufirst=Abdessalam&rft.au=Cherkaoui%2C%26%2332%3BAbdessalam&rft.au=Riat%2C%26%2332%3BArnaud&rft.au=Renzi%2C%26%2332%3BGesuele&rft.au=Fischer%2C%26%2332%3BAdrien&rft.au=Schrenzel%2C%26%2332%3BJacques&rft.date=1+February+2023&rft.volume=42&rft.issue=2&rft.pages=153%E2%80%93159&rft_id=info:doi\/10.1007%2Fs10096-022-04531-3&rft.issn=0934-9723&rft_id=info:pmc\/PMC9836989&rft_id=info:pmid\/36469165&rft_id=https%3A%2F%2Flink.springer.com%2F10.1007%2Fs10096-022-04531-3&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:8-22\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:8_22-0\">22.0<\/a><\/sup> <sup><a href=\"#cite_ref-:8_22-1\">22.1<\/a><\/sup> <sup><a href=\"#cite_ref-:8_22-2\">22.2<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Jacot, Damien; Sarton-Loh\u00e9ac, Garance; Coste, Alix T.; Bertelli, Claire; Greub, Gilbert; Prod'hom, Guy; Croxatto, Antony (1 August 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X20306042\" target=\"_blank\">\"Performance evaluation of the Becton Dickinson Kiestra\u2122 IdentifA\/SusceptA\"<\/a> (in en). <i>Clinical Microbiology and Infection<\/i> <b>27<\/b> (8): 1167.e9\u20131167.e17. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.cmi.2020.09.050\" target=\"_blank\">10.1016\/j.cmi.2020.09.050<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X20306042\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X20306042<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Performance+evaluation+of+the+Becton+Dickinson+Kiestra%E2%84%A2+IdentifA%2FSusceptA&rft.jtitle=Clinical+Microbiology+and+Infection&rft.aulast=Jacot&rft.aufirst=Damien&rft.au=Jacot%2C%26%2332%3BDamien&rft.au=Sarton-Loh%C3%A9ac%2C%26%2332%3BGarance&rft.au=Coste%2C%26%2332%3BAlix+T.&rft.au=Bertelli%2C%26%2332%3BClaire&rft.au=Greub%2C%26%2332%3BGilbert&rft.au=Prod%27hom%2C%26%2332%3BGuy&rft.au=Croxatto%2C%26%2332%3BAntony&rft.date=1+August+2021&rft.volume=27&rft.issue=8&rft.pages=1167.e9%E2%80%931167.e17&rft_id=info:doi\/10.1016%2Fj.cmi.2020.09.050&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS1198743X20306042&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-23\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-23\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Cherkaoui, Abdessalam; Renzi, Gesuele; Vuilleumier, Nicolas; Schrenzel, Jacques (18 August 2021). McElvania, Erin. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.00777-21\" target=\"_blank\">\"Performance of Fully Automated Antimicrobial Disk Diffusion Susceptibility Testing Using Copan WASP Colibri Coupled to the Radian In-Line Carousel and Expert System\"<\/a> (in en). <i>Journal of Clinical Microbiology<\/i> <b>59<\/b> (9): e00777\u201321. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1128%2FJCM.00777-21\" target=\"_blank\">10.1128\/JCM.00777-21<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0095-1137\" target=\"_blank\">0095-1137<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC8373016\/\" target=\"_blank\">PMC8373016<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/34160274\" target=\"_blank\">34160274<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.00777-21\" target=\"_blank\">https:\/\/journals.asm.org\/doi\/10.1128\/JCM.00777-21<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Performance+of+Fully+Automated+Antimicrobial+Disk+Diffusion+Susceptibility+Testing+Using+Copan+WASP+Colibri+Coupled+to+the+Radian+In-Line+Carousel+and+Expert+System&rft.jtitle=Journal+of+Clinical+Microbiology&rft.aulast=Cherkaoui&rft.aufirst=Abdessalam&rft.au=Cherkaoui%2C%26%2332%3BAbdessalam&rft.au=Renzi%2C%26%2332%3BGesuele&rft.au=Vuilleumier%2C%26%2332%3BNicolas&rft.au=Schrenzel%2C%26%2332%3BJacques&rft.date=18+August+2021&rft.volume=59&rft.issue=9&rft.pages=e00777%E2%80%9321&rft_id=info:doi\/10.1128%2FJCM.00777-21&rft.issn=0095-1137&rft_id=info:pmc\/PMC8373016&rft_id=info:pmid\/34160274&rft_id=https%3A%2F%2Fjournals.asm.org%2Fdoi%2F10.1128%2FJCM.00777-21&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:9-24\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:9_24-0\">24.0<\/a><\/sup> <sup><a href=\"#cite_ref-:9_24-1\">24.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Faron, Matthew L.; Buchan, Blake W.; Vismara, Chiara; Lacchini, Carla; Bielli, Alessandra; Gesu, Giovanni; Liebregts, Theo; van Bree, Anita <i>et al.<\/i> (1 March 2016). Richter, S. S.. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.02778-15\" target=\"_blank\">\"Automated Scoring of Chromogenic Media for Detection of Methicillin-Resistant Staphylococcus aureus by Use of WASPLab Image Analysis Software\"<\/a> (in en). <i>Journal of Clinical Microbiology<\/i> <b>54<\/b> (3): 620\u2013624. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1128%2FJCM.02778-15\" target=\"_blank\">10.1128\/JCM.02778-15<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0095-1137\" target=\"_blank\">0095-1137<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC4767952\/\" target=\"_blank\">PMC4767952<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/26719443\" target=\"_blank\">26719443<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.02778-15\" target=\"_blank\">https:\/\/journals.asm.org\/doi\/10.1128\/JCM.02778-15<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Automated+Scoring+of+Chromogenic+Media+for+Detection+of+Methicillin-Resistant+Staphylococcus+aureus+by+Use+of+WASPLab+Image+Analysis+Software&rft.jtitle=Journal+of+Clinical+Microbiology&rft.aulast=Faron&rft.aufirst=Matthew+L.&rft.au=Faron%2C%26%2332%3BMatthew+L.&rft.au=Buchan%2C%26%2332%3BBlake+W.&rft.au=Vismara%2C%26%2332%3BChiara&rft.au=Lacchini%2C%26%2332%3BCarla&rft.au=Bielli%2C%26%2332%3BAlessandra&rft.au=Gesu%2C%26%2332%3BGiovanni&rft.au=Liebregts%2C%26%2332%3BTheo&rft.au=van+Bree%2C%26%2332%3BAnita&rft.au=Jansz%2C%26%2332%3BArjan&rft.date=1+March+2016&rft.volume=54&rft.issue=3&rft.pages=620%E2%80%93624&rft_id=info:doi\/10.1128%2FJCM.02778-15&rft.issn=0095-1137&rft_id=info:pmc\/PMC4767952&rft_id=info:pmid\/26719443&rft_id=https%3A%2F%2Fjournals.asm.org%2Fdoi%2F10.1128%2FJCM.02778-15&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:10-25\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:10_25-0\">25.0<\/a><\/sup> <sup><a href=\"#cite_ref-:10_25-1\">25.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Faron, Matthew L.; Buchan, Blake W.; Coon, Christopher; Liebregts, Theo; van Bree, Anita; Jansz, Arjan R.; Soucy, Genevieve; Korver, John <i>et al.<\/i> (1 October 2016). Burnham, C.-A. D.. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.01040-16\" target=\"_blank\">\"Automatic Digital Analysis of Chromogenic Media for Vancomycin-Resistant-Enterococcus Screens Using Copan WASPLab\"<\/a> (in en). <i>Journal of Clinical Microbiology<\/i> <b>54<\/b> (10): 2464\u20132469. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1128%2FJCM.01040-16\" target=\"_blank\">10.1128\/JCM.01040-16<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0095-1137\" target=\"_blank\">0095-1137<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC5035414\/\" target=\"_blank\">PMC5035414<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/27413193\" target=\"_blank\">27413193<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.01040-16\" target=\"_blank\">https:\/\/journals.asm.org\/doi\/10.1128\/JCM.01040-16<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Automatic+Digital+Analysis+of+Chromogenic+Media+for+Vancomycin-Resistant-Enterococcus+Screens+Using+Copan+WASPLab&rft.jtitle=Journal+of+Clinical+Microbiology&rft.aulast=Faron&rft.aufirst=Matthew+L.&rft.au=Faron%2C%26%2332%3BMatthew+L.&rft.au=Buchan%2C%26%2332%3BBlake+W.&rft.au=Coon%2C%26%2332%3BChristopher&rft.au=Liebregts%2C%26%2332%3BTheo&rft.au=van+Bree%2C%26%2332%3BAnita&rft.au=Jansz%2C%26%2332%3BArjan+R.&rft.au=Soucy%2C%26%2332%3BGenevieve&rft.au=Korver%2C%26%2332%3BJohn&rft.au=Ledeboer%2C%26%2332%3BNathan+A.&rft.date=1+October+2016&rft.volume=54&rft.issue=10&rft.pages=2464%E2%80%932469&rft_id=info:doi\/10.1128%2FJCM.01040-16&rft.issn=0095-1137&rft_id=info:pmc\/PMC5035414&rft_id=info:pmid\/27413193&rft_id=https%3A%2F%2Fjournals.asm.org%2Fdoi%2F10.1128%2FJCM.01040-16&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-26\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-26\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Foschi, Claudio; Gaibani, Paolo; Lombardo, Donatella; Re, Maria Carla; Ambretti, Simone (1 June 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2213716519302668\" target=\"_blank\">\"Rectal screening for carbapenemase-producing Enterobacteriaceae: a proposed workflow\"<\/a> (in en). <i>Journal of Global Antimicrobial Resistance<\/i> <b>21<\/b>: 86\u201390. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.jgar.2019.10.012\" target=\"_blank\">10.1016\/j.jgar.2019.10.012<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2213716519302668\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2213716519302668<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Rectal+screening+for+carbapenemase-producing+Enterobacteriaceae%3A+a+proposed+workflow&rft.jtitle=Journal+of+Global+Antimicrobial+Resistance&rft.aulast=Foschi&rft.aufirst=Claudio&rft.au=Foschi%2C%26%2332%3BClaudio&rft.au=Gaibani%2C%26%2332%3BPaolo&rft.au=Lombardo%2C%26%2332%3BDonatella&rft.au=Re%2C%26%2332%3BMaria+Carla&rft.au=Ambretti%2C%26%2332%3BSimone&rft.date=1+June+2020&rft.volume=21&rft.pages=86%E2%80%9390&rft_id=info:doi\/10.1016%2Fj.jgar.2019.10.012&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2213716519302668&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-27\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-27\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Dauwalder, O.; Landrieve, L.; Laurent, F.; de Montclos, M.; Vandenesch, F.; Lina, G. (1 March 2016). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X15009787\" target=\"_blank\">\"Does bacteriology laboratory automation reduce time to results and increase quality management?\"<\/a> (in en). <i>Clinical Microbiology and Infection<\/i> <b>22<\/b> (3): 236\u2013243. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.cmi.2015.10.037\" target=\"_blank\">10.1016\/j.cmi.2015.10.037<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X15009787\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X15009787<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Does+bacteriology+laboratory+automation+reduce+time+to+results+and+increase+quality+management%3F&rft.jtitle=Clinical+Microbiology+and+Infection&rft.aulast=Dauwalder&rft.aufirst=O.&rft.au=Dauwalder%2C%26%2332%3BO.&rft.au=Landrieve%2C%26%2332%3BL.&rft.au=Laurent%2C%26%2332%3BF.&rft.au=de+Montclos%2C%26%2332%3BM.&rft.au=Vandenesch%2C%26%2332%3BF.&rft.au=Lina%2C%26%2332%3BG.&rft.date=1+March+2016&rft.volume=22&rft.issue=3&rft.pages=236%E2%80%93243&rft_id=info:doi\/10.1016%2Fj.cmi.2015.10.037&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS1198743X15009787&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-28\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-28\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Tabak, Ying P.; Vankeepuram, Latha; Ye, Gang; Jeffers, Kay; Gupta, Vikas; Murray, Patrick R. (1 December 2018). Carroll, Karen C.. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.00500-18\" target=\"_blank\">\"Blood Culture Turnaround Time in U.S. Acute Care Hospitals and Implications for Laboratory Process Optimization\"<\/a> (in en). <i>Journal of Clinical Microbiology<\/i> <b>56<\/b> (12): e00500\u201318. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1128%2FJCM.00500-18\" target=\"_blank\">10.1128\/JCM.00500-18<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0095-1137\" target=\"_blank\">0095-1137<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC6258864\/\" target=\"_blank\">PMC6258864<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/30135230\" target=\"_blank\">30135230<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.00500-18\" target=\"_blank\">https:\/\/journals.asm.org\/doi\/10.1128\/JCM.00500-18<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Blood+Culture+Turnaround+Time+in+U.S.+Acute+Care+Hospitals+and+Implications+for+Laboratory+Process+Optimization&rft.jtitle=Journal+of+Clinical+Microbiology&rft.aulast=Tabak&rft.aufirst=Ying+P.&rft.au=Tabak%2C%26%2332%3BYing+P.&rft.au=Vankeepuram%2C%26%2332%3BLatha&rft.au=Ye%2C%26%2332%3BGang&rft.au=Jeffers%2C%26%2332%3BKay&rft.au=Gupta%2C%26%2332%3BVikas&rft.au=Murray%2C%26%2332%3BPatrick+R.&rft.date=1+December+2018&rft.volume=56&rft.issue=12&rft.pages=e00500%E2%80%9318&rft_id=info:doi\/10.1128%2FJCM.00500-18&rft.issn=0095-1137&rft_id=info:pmc\/PMC6258864&rft_id=info:pmid\/30135230&rft_id=https%3A%2F%2Fjournals.asm.org%2Fdoi%2F10.1128%2FJCM.00500-18&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:11-29\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:11_29-0\">29.0<\/a><\/sup> <sup><a href=\"#cite_ref-:11_29-1\">29.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">De Socio, Giuseppe Vittorio; Di Donato, Francesco; Paggi, Riccardo; Gabrielli, Chiara; Belati, Alessandra; Rizza, Giuseppe; Savoia, Martina; Repetto, Antonella <i>et al.<\/i> (1 December 2018). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/s10096-018-3377-5\" target=\"_blank\">\"Laboratory automation reduces time to report of positive blood cultures and improves management of patients with bloodstream infection\"<\/a> (in en). <i>European Journal of Clinical Microbiology & Infectious Diseases<\/i> <b>37<\/b> (12): 2313\u20132322. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2Fs10096-018-3377-5\" target=\"_blank\">10.1007\/s10096-018-3377-5<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0934-9723\" target=\"_blank\">0934-9723<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/s10096-018-3377-5\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/s10096-018-3377-5<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Laboratory+automation+reduces+time+to+report+of+positive+blood+cultures+and+improves+management+of+patients+with+bloodstream+infection&rft.jtitle=European+Journal+of+Clinical+Microbiology+%26+Infectious+Diseases&rft.aulast=De+Socio&rft.aufirst=Giuseppe+Vittorio&rft.au=De+Socio%2C%26%2332%3BGiuseppe+Vittorio&rft.au=Di+Donato%2C%26%2332%3BFrancesco&rft.au=Paggi%2C%26%2332%3BRiccardo&rft.au=Gabrielli%2C%26%2332%3BChiara&rft.au=Belati%2C%26%2332%3BAlessandra&rft.au=Rizza%2C%26%2332%3BGiuseppe&rft.au=Savoia%2C%26%2332%3BMartina&rft.au=Repetto%2C%26%2332%3BAntonella&rft.au=Cenci%2C%26%2332%3BElio&rft.date=1+December+2018&rft.volume=37&rft.issue=12&rft.pages=2313%E2%80%932322&rft_id=info:doi\/10.1007%2Fs10096-018-3377-5&rft.issn=0934-9723&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2Fs10096-018-3377-5&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-30\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-30\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Vandenberg, Olivier; Durand, G\u00e9raldine; Hallin, Marie; Diefenbach, Andreas; Gant, Vanya; Murray, Patrick; Kozlakidis, Zisis; van Belkum, Alex (18 March 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/CMR.00057-19\" target=\"_blank\">\"Consolidation of Clinical Microbiology Laboratories and Introduction of Transformative Technologies\"<\/a> (in en). <i>Clinical Microbiology Reviews<\/i> <b>33<\/b> (2): e00057\u201319. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1128%2FCMR.00057-19\" target=\"_blank\">10.1128\/CMR.00057-19<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0893-8512\" target=\"_blank\">0893-8512<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC7048017\/\" target=\"_blank\">PMC7048017<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/32102900\" target=\"_blank\">32102900<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/CMR.00057-19\" target=\"_blank\">https:\/\/journals.asm.org\/doi\/10.1128\/CMR.00057-19<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Consolidation+of+Clinical+Microbiology+Laboratories+and+Introduction+of+Transformative+Technologies&rft.jtitle=Clinical+Microbiology+Reviews&rft.aulast=Vandenberg&rft.aufirst=Olivier&rft.au=Vandenberg%2C%26%2332%3BOlivier&rft.au=Durand%2C%26%2332%3BG%C3%A9raldine&rft.au=Hallin%2C%26%2332%3BMarie&rft.au=Diefenbach%2C%26%2332%3BAndreas&rft.au=Gant%2C%26%2332%3BVanya&rft.au=Murray%2C%26%2332%3BPatrick&rft.au=Kozlakidis%2C%26%2332%3BZisis&rft.au=van+Belkum%2C%26%2332%3BAlex&rft.date=18+March+2020&rft.volume=33&rft.issue=2&rft.pages=e00057%E2%80%9319&rft_id=info:doi\/10.1128%2FCMR.00057-19&rft.issn=0893-8512&rft_id=info:pmc\/PMC7048017&rft_id=info:pmid\/32102900&rft_id=https%3A%2F%2Fjournals.asm.org%2Fdoi%2F10.1128%2FCMR.00057-19&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-31\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-31\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Kulengowski, B.; Ribes, J.A.; Burgess, D.S. (1 January 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X18303434\" target=\"_blank\">\"Polymyxin B Etest\u00ae compared with gold-standard broth microdilution in carbapenem-resistant Enterobacteriaceae exhibiting a wide range of polymyxin B MICs\"<\/a> (in en). <i>Clinical Microbiology and Infection<\/i> <b>25<\/b> (1): 92\u201395. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.cmi.2018.04.008\" target=\"_blank\">10.1016\/j.cmi.2018.04.008<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X18303434\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1198743X18303434<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Polymyxin+B+Etest%C2%AE+compared+with+gold-standard+broth+microdilution+in+carbapenem-resistant+Enterobacteriaceae+exhibiting+a+wide+range+of+polymyxin+B+MICs&rft.jtitle=Clinical+Microbiology+and+Infection&rft.aulast=Kulengowski&rft.aufirst=B.&rft.au=Kulengowski%2C%26%2332%3BB.&rft.au=Ribes%2C%26%2332%3BJ.A.&rft.au=Burgess%2C%26%2332%3BD.S.&rft.date=1+January+2019&rft.volume=25&rft.issue=1&rft.pages=92%E2%80%9395&rft_id=info:doi\/10.1016%2Fj.cmi.2018.04.008&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS1198743X18303434&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-32\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-32\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Camarlinghi, Giulio; Parisio, Eva Maria; Antonelli, Alberto; Nardone, Maria; Coppi, Marco; Giani, Tommaso; Mattei, Romano; Rossolini, Gian Maria (1 January 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0732889318302505\" target=\"_blank\">\"Discrepancies in fosfomycin susceptibility testing of KPC-producing Klebsiella pneumoniae with various commercial methods\"<\/a> (in en). <i>Diagnostic Microbiology and Infectious Disease<\/i> <b>93<\/b> (1): 74\u201376. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.diagmicrobio.2018.07.014\" target=\"_blank\">10.1016\/j.diagmicrobio.2018.07.014<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0732889318302505\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0732889318302505<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Discrepancies+in+fosfomycin+susceptibility+testing+of+KPC-producing+Klebsiella+pneumoniae+with+various+commercial+methods&rft.jtitle=Diagnostic+Microbiology+and+Infectious+Disease&rft.aulast=Camarlinghi&rft.aufirst=Giulio&rft.au=Camarlinghi%2C%26%2332%3BGiulio&rft.au=Parisio%2C%26%2332%3BEva+Maria&rft.au=Antonelli%2C%26%2332%3BAlberto&rft.au=Nardone%2C%26%2332%3BMaria&rft.au=Coppi%2C%26%2332%3BMarco&rft.au=Giani%2C%26%2332%3BTommaso&rft.au=Mattei%2C%26%2332%3BRomano&rft.au=Rossolini%2C%26%2332%3BGian+Maria&rft.date=1+January+2019&rft.volume=93&rft.issue=1&rft.pages=74%E2%80%9376&rft_id=info:doi\/10.1016%2Fj.diagmicrobio.2018.07.014&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0732889318302505&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-33\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-33\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Simner, Patricia J.; Patel, Robin (17 December 2020). Burnham, Carey-Ann D.. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.00951-20\" target=\"_blank\">\"Cefiderocol Antimicrobial Susceptibility Testing Considerations: the Achilles' Heel of the Trojan Horse?\"<\/a> (in en). <i>Journal of Clinical Microbiology<\/i> <b>59<\/b> (1): e00951\u201320. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1128%2FJCM.00951-20\" target=\"_blank\">10.1128\/JCM.00951-20<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0095-1137\" target=\"_blank\">0095-1137<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC7771437\/\" target=\"_blank\">PMC7771437<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/32727829\" target=\"_blank\">32727829<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.00951-20\" target=\"_blank\">https:\/\/journals.asm.org\/doi\/10.1128\/JCM.00951-20<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Cefiderocol+Antimicrobial+Susceptibility+Testing+Considerations%3A+the+Achilles%27+Heel+of+the+Trojan+Horse%3F&rft.jtitle=Journal+of+Clinical+Microbiology&rft.aulast=Simner&rft.aufirst=Patricia+J.&rft.au=Simner%2C%26%2332%3BPatricia+J.&rft.au=Patel%2C%26%2332%3BRobin&rft.date=17+December+2020&rft.volume=59&rft.issue=1&rft.pages=e00951%E2%80%9320&rft_id=info:doi\/10.1128%2FJCM.00951-20&rft.issn=0095-1137&rft_id=info:pmc\/PMC7771437&rft_id=info:pmid\/32727829&rft_id=https%3A%2F%2Fjournals.asm.org%2Fdoi%2F10.1128%2FJCM.00951-20&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-34\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-34\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Antonelli, Alberto; Giani, Tommaso; Di Pilato, Vincenzo; Riccobono, Eleonora; Perriello, Gabriele; Mencacci, Antonella; Rossolini, Gian Maria (1 August 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/academic.oup.com\/jac\/article\/74\/8\/2464\/5477394\" target=\"_blank\">\"KPC-31 expressed in a ceftazidime\/avibactam-resistant Klebsiella pneumoniae is associated with relevant detection issues\"<\/a> (in en). <i>Journal of Antimicrobial Chemotherapy<\/i> <b>74<\/b> (8): 2464\u20132466. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1093%2Fjac%2Fdkz156\" target=\"_blank\">10.1093\/jac\/dkz156<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0305-7453\" target=\"_blank\">0305-7453<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/academic.oup.com\/jac\/article\/74\/8\/2464\/5477394\" target=\"_blank\">https:\/\/academic.oup.com\/jac\/article\/74\/8\/2464\/5477394<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=KPC-31+expressed+in+a+ceftazidime%2Favibactam-resistant+Klebsiella+pneumoniae+is+associated+with+relevant+detection+issues&rft.jtitle=Journal+of+Antimicrobial+Chemotherapy&rft.aulast=Antonelli&rft.aufirst=Alberto&rft.au=Antonelli%2C%26%2332%3BAlberto&rft.au=Giani%2C%26%2332%3BTommaso&rft.au=Di+Pilato%2C%26%2332%3BVincenzo&rft.au=Riccobono%2C%26%2332%3BEleonora&rft.au=Perriello%2C%26%2332%3BGabriele&rft.au=Mencacci%2C%26%2332%3BAntonella&rft.au=Rossolini%2C%26%2332%3BGian+Maria&rft.date=1+August+2019&rft.volume=74&rft.issue=8&rft.pages=2464%E2%80%932466&rft_id=info:doi\/10.1093%2Fjac%2Fdkz156&rft.issn=0305-7453&rft_id=https%3A%2F%2Facademic.oup.com%2Fjac%2Farticle%2F74%2F8%2F2464%2F5477394&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:12-35\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:12_35-0\">35.0<\/a><\/sup> <sup><a href=\"#cite_ref-:12_35-1\">35.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">De Socio, Giuseppe Vittorio; Rubbioni, Paola; Botta, Daniele; Cenci, Elio; Belati, Alessandra; Paggi, Riccardo; Pasticci, Maria Bruna; Mencacci, Antonella (1 December 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2213716519301237\" target=\"_blank\">\"Measurement and prediction of antimicrobial resistance in bloodstream infections by ESKAPE pathogens and Escherichia coli\"<\/a> (in en). <i>Journal of Global Antimicrobial Resistance<\/i> <b>19<\/b>: 154\u2013160. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.jgar.2019.05.013\" target=\"_blank\">10.1016\/j.jgar.2019.05.013<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2213716519301237\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2213716519301237<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Measurement+and+prediction+of+antimicrobial+resistance+in+bloodstream+infections+by+ESKAPE+pathogens+and+Escherichia+coli&rft.jtitle=Journal+of+Global+Antimicrobial+Resistance&rft.aulast=De+Socio&rft.aufirst=Giuseppe+Vittorio&rft.au=De+Socio%2C%26%2332%3BGiuseppe+Vittorio&rft.au=Rubbioni%2C%26%2332%3BPaola&rft.au=Botta%2C%26%2332%3BDaniele&rft.au=Cenci%2C%26%2332%3BElio&rft.au=Belati%2C%26%2332%3BAlessandra&rft.au=Paggi%2C%26%2332%3BRiccardo&rft.au=Pasticci%2C%26%2332%3BMaria+Bruna&rft.au=Mencacci%2C%26%2332%3BAntonella&rft.date=1+December+2019&rft.volume=19&rft.pages=154%E2%80%93160&rft_id=info:doi\/10.1016%2Fj.jgar.2019.05.013&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2213716519301237&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-36\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-36\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Culbreath, Karissa; Piwonka, Heather; Korver, John; Noorbakhsh, Mir (18 February 2021). McElvania, Erin. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.01969-20\" target=\"_blank\">\"Benefits Derived from Full Laboratory Automation in Microbiology: a Tale of Four Laboratories\"<\/a> (in en). <i>Journal of Clinical Microbiology<\/i> <b>59<\/b> (3): e01969\u201320. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1128%2FJCM.01969-20\" target=\"_blank\">10.1128\/JCM.01969-20<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0095-1137\" target=\"_blank\">0095-1137<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC8106725\/\" target=\"_blank\">PMC8106725<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/33239383\" target=\"_blank\">33239383<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.01969-20\" target=\"_blank\">https:\/\/journals.asm.org\/doi\/10.1128\/JCM.01969-20<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Benefits+Derived+from+Full+Laboratory+Automation+in+Microbiology%3A+a+Tale+of+Four+Laboratories&rft.jtitle=Journal+of+Clinical+Microbiology&rft.aulast=Culbreath&rft.aufirst=Karissa&rft.au=Culbreath%2C%26%2332%3BKarissa&rft.au=Piwonka%2C%26%2332%3BHeather&rft.au=Korver%2C%26%2332%3BJohn&rft.au=Noorbakhsh%2C%26%2332%3BMir&rft.date=18+February+2021&rft.volume=59&rft.issue=3&rft.pages=e01969%E2%80%9320&rft_id=info:doi\/10.1128%2FJCM.01969-20&rft.issn=0095-1137&rft_id=info:pmc\/PMC8106725&rft_id=info:pmid\/33239383&rft_id=https%3A%2F%2Fjournals.asm.org%2Fdoi%2F10.1128%2FJCM.01969-20&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:13-37\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:13_37-0\">37.0<\/a><\/sup> <sup><a href=\"#cite_ref-:13_37-1\">37.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Zimmermann, Stefan (18 February 2021). McElvania, Erin. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.02592-20\" target=\"_blank\">\"Laboratory Automation in the Microbiology Laboratory: an Ongoing Journey, Not a Tale?\"<\/a> (in en). <i>Journal of Clinical Microbiology<\/i> <b>59<\/b> (3): e02592\u201320. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1128%2FJCM.02592-20\" target=\"_blank\">10.1128\/JCM.02592-20<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0095-1137\" target=\"_blank\">0095-1137<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC8106703\/\" target=\"_blank\">PMC8106703<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/33361341\" target=\"_blank\">33361341<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.02592-20\" target=\"_blank\">https:\/\/journals.asm.org\/doi\/10.1128\/JCM.02592-20<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Laboratory+Automation+in+the+Microbiology+Laboratory%3A+an+Ongoing+Journey%2C+Not+a+Tale%3F&rft.jtitle=Journal+of+Clinical+Microbiology&rft.aulast=Zimmermann&rft.aufirst=Stefan&rft.au=Zimmermann%2C%26%2332%3BStefan&rft.date=18+February+2021&rft.volume=59&rft.issue=3&rft.pages=e02592%E2%80%9320&rft_id=info:doi\/10.1128%2FJCM.02592-20&rft.issn=0095-1137&rft_id=info:pmc\/PMC8106703&rft_id=info:pmid\/33361341&rft_id=https%3A%2F%2Fjournals.asm.org%2Fdoi%2F10.1128%2FJCM.02592-20&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-38\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-38\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation book\">Womack, James P.; Jones, Daniel T.; Roos, Daniel (1990). <i>The machine that changed the world: the story of lean production - Toyota\u00b4s secret weapon in the global car wars that is revolutionizing world industry<\/i> (1. paperback ed ed.). London: Free Press. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-0-7432-9979-4.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=The+machine+that+changed+the+world%3A+the+story+of+lean+production+-+Toyota%C2%B4s+secret+weapon+in+the+global+car+wars+that+is+revolutionizing+world+industry&rft.aulast=Womack&rft.aufirst=James+P.&rft.au=Womack%2C%26%2332%3BJames+P.&rft.au=Jones%2C%26%2332%3BDaniel+T.&rft.au=Roos%2C%26%2332%3BDaniel&rft.date=1990&rft.edition=1.+paperback+ed&rft.place=London&rft.pub=Free+Press&rft.isbn=978-0-7432-9979-4&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-39\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-39\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Yarbrough, Melanie L.; Lainhart, William; McMullen, Allison R.; Anderson, Neil W.; Burnham, Carey-Ann D. (1 December 2018). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/s10096-018-3391-7\" target=\"_blank\">\"Impact of total laboratory automation on workflow and specimen processing time for culture of urine specimens\"<\/a> (in en). <i>European Journal of Clinical Microbiology & Infectious Diseases<\/i> <b>37<\/b> (12): 2405\u20132411. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2Fs10096-018-3391-7\" target=\"_blank\">10.1007\/s10096-018-3391-7<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0934-9723\" target=\"_blank\">0934-9723<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/s10096-018-3391-7\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/s10096-018-3391-7<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Impact+of+total+laboratory+automation+on+workflow+and+specimen+processing+time+for+culture+of+urine+specimens&rft.jtitle=European+Journal+of+Clinical+Microbiology+%26+Infectious+Diseases&rft.aulast=Yarbrough&rft.aufirst=Melanie+L.&rft.au=Yarbrough%2C%26%2332%3BMelanie+L.&rft.au=Lainhart%2C%26%2332%3BWilliam&rft.au=McMullen%2C%26%2332%3BAllison+R.&rft.au=Anderson%2C%26%2332%3BNeil+W.&rft.au=Burnham%2C%26%2332%3BCarey-Ann+D.&rft.date=1+December+2018&rft.volume=37&rft.issue=12&rft.pages=2405%E2%80%932411&rft_id=info:doi\/10.1007%2Fs10096-018-3391-7&rft.issn=0934-9723&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2Fs10096-018-3391-7&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-40\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-40\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Burckhardt, Irene; Horner, Susanne; Burckhardt, Florian; Zimmermann, Stefan (1 September 2018). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/s10096-018-3308-5\" target=\"_blank\">\"Detection of MRSA in nasal swabs\u2014marked reduction of time to report for negative reports by substituting classical manual workflow with total lab automation\"<\/a> (in en). <i>European Journal of Clinical Microbiology & Infectious Diseases<\/i> <b>37<\/b> (9): 1745\u20131751. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2Fs10096-018-3308-5\" target=\"_blank\">10.1007\/s10096-018-3308-5<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0934-9723\" target=\"_blank\">0934-9723<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC6133036\/\" target=\"_blank\">PMC6133036<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/29943308\" target=\"_blank\">29943308<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/s10096-018-3308-5\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/s10096-018-3308-5<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Detection+of+MRSA+in+nasal+swabs%E2%80%94marked+reduction+of+time+to+report+for+negative+reports+by+substituting+classical+manual+workflow+with+total+lab+automation&rft.jtitle=European+Journal+of+Clinical+Microbiology+%26+Infectious+Diseases&rft.aulast=Burckhardt&rft.aufirst=Irene&rft.au=Burckhardt%2C%26%2332%3BIrene&rft.au=Horner%2C%26%2332%3BSusanne&rft.au=Burckhardt%2C%26%2332%3BFlorian&rft.au=Zimmermann%2C%26%2332%3BStefan&rft.date=1+September+2018&rft.volume=37&rft.issue=9&rft.pages=1745%E2%80%931751&rft_id=info:doi\/10.1007%2Fs10096-018-3308-5&rft.issn=0934-9723&rft_id=info:pmc\/PMC6133036&rft_id=info:pmid\/29943308&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2Fs10096-018-3308-5&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-41\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-41\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Seng, Piseth; Drancourt, Michel; Gouriet, Fr\u00e9d\u00e9rique; La Scola, Bernard; Fournier, Pierre\u2010Edouard; Rolain, Jean Marc; Raoult, Didier (15 August 2009). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/academic.oup.com\/cid\/article-lookup\/doi\/10.1086\/600885\" target=\"_blank\">\"Ongoing Revolution in Bacteriology: Routine Identification of Bacteria by Matrix\u2010Assisted Laser Desorption Ionization Time\u2010of\u2010Flight Mass Spectrometry\"<\/a> (in en). <i>Clinical Infectious Diseases<\/i> <b>49<\/b> (4): 543\u2013551. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1086%2F600885\" target=\"_blank\">10.1086\/600885<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1058-4838\" target=\"_blank\">1058-4838<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/academic.oup.com\/cid\/article-lookup\/doi\/10.1086\/600885\" target=\"_blank\">https:\/\/academic.oup.com\/cid\/article-lookup\/doi\/10.1086\/600885<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Ongoing+Revolution+in+Bacteriology%3A+Routine+Identification+of+Bacteria+by+Matrix%E2%80%90Assisted+Laser+Desorption+Ionization+Time%E2%80%90of%E2%80%90Flight+Mass+Spectrometry&rft.jtitle=Clinical+Infectious+Diseases&rft.aulast=Seng&rft.aufirst=Piseth&rft.au=Seng%2C%26%2332%3BPiseth&rft.au=Drancourt%2C%26%2332%3BMichel&rft.au=Gouriet%2C%26%2332%3BFr%C3%A9d%C3%A9rique&rft.au=La+Scola%2C%26%2332%3BBernard&rft.au=Fournier%2C%26%2332%3BPierre%E2%80%90Edouard&rft.au=Rolain%2C%26%2332%3BJean%C2%A0Marc&rft.au=Raoult%2C%26%2332%3BDidier&rft.date=15+August+2009&rft.volume=49&rft.issue=4&rft.pages=543%E2%80%93551&rft_id=info:doi\/10.1086%2F600885&rft.issn=1058-4838&rft_id=https%3A%2F%2Facademic.oup.com%2Fcid%2Farticle-lookup%2Fdoi%2F10.1086%2F600885&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-42\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-42\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Charnot-Katsikas, Angella; Tesic, Vera; Love, Nedra; Hill, Brandy; Bethel, Cindy; Boonlayangoor, Sue; Beavis, Kathleen G. (1 January 2018). Bourbeau, Paul. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.01166-17\" target=\"_blank\">\"Use of the Accelerate Pheno System for Identification and Antimicrobial Susceptibility Testing of Pathogens in Positive Blood Cultures and Impact on Time to Results and Workflow\"<\/a> (in en). <i>Journal of Clinical Microbiology<\/i> <b>56<\/b> (1): e01166\u201317. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1128%2FJCM.01166-17\" target=\"_blank\">10.1128\/JCM.01166-17<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0095-1137\" target=\"_blank\">0095-1137<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC5744213\/\" target=\"_blank\">PMC5744213<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/29118168\" target=\"_blank\">29118168<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/journals.asm.org\/doi\/10.1128\/JCM.01166-17\" target=\"_blank\">https:\/\/journals.asm.org\/doi\/10.1128\/JCM.01166-17<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Use+of+the+Accelerate+Pheno+System+for+Identification+and+Antimicrobial+Susceptibility+Testing+of+Pathogens+in+Positive+Blood+Cultures+and+Impact+on+Time+to+Results+and+Workflow&rft.jtitle=Journal+of+Clinical+Microbiology&rft.aulast=Charnot-Katsikas&rft.aufirst=Angella&rft.au=Charnot-Katsikas%2C%26%2332%3BAngella&rft.au=Tesic%2C%26%2332%3BVera&rft.au=Love%2C%26%2332%3BNedra&rft.au=Hill%2C%26%2332%3BBrandy&rft.au=Bethel%2C%26%2332%3BCindy&rft.au=Boonlayangoor%2C%26%2332%3BSue&rft.au=Beavis%2C%26%2332%3BKathleen+G.&rft.date=1+January+2018&rft.volume=56&rft.issue=1&rft.pages=e01166%E2%80%9317&rft_id=info:doi\/10.1128%2FJCM.01166-17&rft.issn=0095-1137&rft_id=info:pmc\/PMC5744213&rft_id=info:pmid\/29118168&rft_id=https%3A%2F%2Fjournals.asm.org%2Fdoi%2F10.1128%2FJCM.01166-17&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:14-43\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:14_43-0\">43.0<\/a><\/sup> <sup><a href=\"#cite_ref-:14_43-1\">43.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Arena, Fabio; Giani, Tommaso; Pollini, Simona; Viaggi, Bruno; Pecile, Patrizia; Rossolini, Gian Maria (1 April 2017). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.futuremedicine.com\/doi\/10.2217\/fmb-2017-0019\" target=\"_blank\">\"Molecular antibiogram in diagnostic clinical microbiology: advantages and challenges\"<\/a> (in en). <i>Future Microbiology<\/i> <b>12<\/b> (5): 361\u2013364. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.2217%2Ffmb-2017-0019\" target=\"_blank\">10.2217\/fmb-2017-0019<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1746-0913\" target=\"_blank\">1746-0913<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.futuremedicine.com\/doi\/10.2217\/fmb-2017-0019\" target=\"_blank\">https:\/\/www.futuremedicine.com\/doi\/10.2217\/fmb-2017-0019<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Molecular+antibiogram+in+diagnostic+clinical+microbiology%3A+advantages+and+challenges&rft.jtitle=Future+Microbiology&rft.aulast=Arena&rft.aufirst=Fabio&rft.au=Arena%2C%26%2332%3BFabio&rft.au=Giani%2C%26%2332%3BTommaso&rft.au=Pollini%2C%26%2332%3BSimona&rft.au=Viaggi%2C%26%2332%3BBruno&rft.au=Pecile%2C%26%2332%3BPatrizia&rft.au=Rossolini%2C%26%2332%3BGian+Maria&rft.date=1+April+2017&rft.volume=12&rft.issue=5&rft.pages=361%E2%80%93364&rft_id=info:doi\/10.2217%2Ffmb-2017-0019&rft.issn=1746-0913&rft_id=https%3A%2F%2Fwww.futuremedicine.com%2Fdoi%2F10.2217%2Ffmb-2017-0019&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-44\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-44\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Mitchell, Stephanie L.; Simner, Patricia J. (1 September 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0272271219300307\" target=\"_blank\">\"Next-Generation Sequencing in Clinical Microbiology\"<\/a> (in en). <i>Clinics in Laboratory Medicine<\/i> <b>39<\/b> (3): 405\u2013418. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.cll.2019.05.003\" target=\"_blank\">10.1016\/j.cll.2019.05.003<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0272271219300307\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0272271219300307<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Next-Generation+Sequencing+in+Clinical+Microbiology&rft.jtitle=Clinics+in+Laboratory+Medicine&rft.aulast=Mitchell&rft.aufirst=Stephanie+L.&rft.au=Mitchell%2C%26%2332%3BStephanie+L.&rft.au=Simner%2C%26%2332%3BPatricia+J.&rft.date=1+September+2019&rft.volume=39&rft.issue=3&rft.pages=405%E2%80%93418&rft_id=info:doi\/10.1016%2Fj.cll.2019.05.003&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0272271219300307&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-45\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-45\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Pitashny, Milena; Kadry, Balqees; Shalaginov, Raya; Gazit, Liat; Zohar, Yaniv; Szwarcwort, Moran; Stabholz, Yoav; Paul, Mical (19 October 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcimb.2022.955481\/full\" target=\"_blank\">\"NGS in the clinical microbiology settings\"<\/a>. <i>Frontiers in Cellular and Infection Microbiology<\/i> <b>12<\/b>: 955481. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3389%2Ffcimb.2022.955481\" target=\"_blank\">10.3389\/fcimb.2022.955481<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2235-2988\" target=\"_blank\">2235-2988<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC9627026\/\" target=\"_blank\">PMC9627026<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/36339334\" target=\"_blank\">36339334<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcimb.2022.955481\/full\" target=\"_blank\">https:\/\/www.frontiersin.org\/articles\/10.3389\/fcimb.2022.955481\/full<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=NGS+in+the+clinical+microbiology+settings&rft.jtitle=Frontiers+in+Cellular+and+Infection+Microbiology&rft.aulast=Pitashny&rft.aufirst=Milena&rft.au=Pitashny%2C%26%2332%3BMilena&rft.au=Kadry%2C%26%2332%3BBalqees&rft.au=Shalaginov%2C%26%2332%3BRaya&rft.au=Gazit%2C%26%2332%3BLiat&rft.au=Zohar%2C%26%2332%3BYaniv&rft.au=Szwarcwort%2C%26%2332%3BMoran&rft.au=Stabholz%2C%26%2332%3BYoav&rft.au=Paul%2C%26%2332%3BMical&rft.date=19+October+2022&rft.volume=12&rft.pages=955481&rft_id=info:doi\/10.3389%2Ffcimb.2022.955481&rft.issn=2235-2988&rft_id=info:pmc\/PMC9627026&rft_id=info:pmid\/36339334&rft_id=https%3A%2F%2Fwww.frontiersin.org%2Farticles%2F10.3389%2Ffcimb.2022.955481%2Ffull&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-46\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-46\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Anandan, Shalini (2015). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/jcdr.net\/article_fulltext.asp?issn=0973-709x&year=2015&volume=9&issue=9&page=DM01&issn=0973-709x&id=6530\" target=\"_blank\">\"Rapid Screening for Carbapenem Resistant Organisms: Current Results and Future Approaches\"<\/a>. <i>JOURNAL OF CLINICAL AND DIAGNOSTIC RESEARCH<\/i>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.7860%2FJCDR%2F2015%2F14246.6530\" target=\"_blank\">10.7860\/JCDR\/2015\/14246.6530<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC4606238\/\" target=\"_blank\">PMC4606238<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/26500909\" target=\"_blank\">26500909<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/jcdr.net\/article_fulltext.asp?issn=0973-709x&year=2015&volume=9&issue=9&page=DM01&issn=0973-709x&id=6530\" target=\"_blank\">http:\/\/jcdr.net\/article_fulltext.asp?issn=0973-709x&year=2015&volume=9&issue=9&page=DM01&issn=0973-709x&id=6530<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Rapid+Screening+for+Carbapenem+Resistant+Organisms%3A+Current+Results+and+Future+Approaches&rft.jtitle=JOURNAL+OF+CLINICAL+AND+DIAGNOSTIC+RESEARCH&rft.aulast=Anandan&rft.aufirst=Shalini&rft.au=Anandan%2C%26%2332%3BShalini&rft.date=2015&rft_id=info:doi\/10.7860%2FJCDR%2F2015%2F14246.6530&rft_id=info:pmc\/PMC4606238&rft_id=info:pmid\/26500909&rft_id=http%3A%2F%2Fjcdr.net%2Farticle_fulltext.asp%3Fissn%3D0973-709x%26year%3D2015%26volume%3D9%26issue%3D9%26page%3DDM01%26issn%3D0973-709x%26id%3D6530&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:15-47\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:15_47-0\">47.0<\/a><\/sup> <sup><a href=\"#cite_ref-:15_47-1\">47.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ambretti, Simone; Bassetti, Matteo; Clerici, Pierangelo; Petrosillo, Nicola; Tumietto, Fabio; Viale, Pierluigi; Rossolini, Gian Maria (1 December 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/aricjournal.biomedcentral.com\/articles\/10.1186\/s13756-019-0591-6\" target=\"_blank\">\"Screening for carriage of carbapenem-resistant Enterobacteriaceae in settings of high endemicity: a position paper from an Italian working group on CRE infections\"<\/a> (in en). <i>Antimicrobial Resistance & Infection Control<\/i> <b>8<\/b> (1): 136. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1186%2Fs13756-019-0591-6\" target=\"_blank\">10.1186\/s13756-019-0591-6<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2047-2994\" target=\"_blank\">2047-2994<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC6693230\/\" target=\"_blank\">PMC6693230<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/31423299\" target=\"_blank\">31423299<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/aricjournal.biomedcentral.com\/articles\/10.1186\/s13756-019-0591-6\" target=\"_blank\">https:\/\/aricjournal.biomedcentral.com\/articles\/10.1186\/s13756-019-0591-6<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Screening+for+carriage+of+carbapenem-resistant+Enterobacteriaceae+in+settings+of+high+endemicity%3A+a+position+paper+from+an+Italian+working+group+on+CRE+infections&rft.jtitle=Antimicrobial+Resistance+%26+Infection+Control&rft.aulast=Ambretti&rft.aufirst=Simone&rft.au=Ambretti%2C%26%2332%3BSimone&rft.au=Bassetti%2C%26%2332%3BMatteo&rft.au=Clerici%2C%26%2332%3BPierangelo&rft.au=Petrosillo%2C%26%2332%3BNicola&rft.au=Tumietto%2C%26%2332%3BFabio&rft.au=Viale%2C%26%2332%3BPierluigi&rft.au=Rossolini%2C%26%2332%3BGian+Maria&rft.date=1+December+2019&rft.volume=8&rft.issue=1&rft.pages=136&rft_id=info:doi\/10.1186%2Fs13756-019-0591-6&rft.issn=2047-2994&rft_id=info:pmc\/PMC6693230&rft_id=info:pmid\/31423299&rft_id=https%3A%2F%2Faricjournal.biomedcentral.com%2Farticles%2F10.1186%2Fs13756-019-0591-6&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-48\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-48\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Mangioni, Davide; Viaggi, Bruno; Giani, Tommaso; Arena, Fabio; D'Arienzo, Sara; Forni, Silvia; Tulli, Giorgio; Rossolini, Gian M (1 February 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.futuremedicine.com\/doi\/10.2217\/fmb-2018-0329\" target=\"_blank\">\"Diagnostic stewardship for sepsis: the need for risk stratification to triage patients for fast microbiology workflows\"<\/a> (in en). <i>Future Microbiology<\/i> <b>14<\/b> (3): 169\u2013174. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.2217%2Ffmb-2018-0329\" target=\"_blank\">10.2217\/fmb-2018-0329<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1746-0913\" target=\"_blank\">1746-0913<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.futuremedicine.com\/doi\/10.2217\/fmb-2018-0329\" target=\"_blank\">https:\/\/www.futuremedicine.com\/doi\/10.2217\/fmb-2018-0329<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Diagnostic+stewardship+for+sepsis%3A+the+need+for+risk+stratification+to+triage+patients+for+fast+microbiology+workflows&rft.jtitle=Future+Microbiology&rft.aulast=Mangioni&rft.aufirst=Davide&rft.au=Mangioni%2C%26%2332%3BDavide&rft.au=Viaggi%2C%26%2332%3BBruno&rft.au=Giani%2C%26%2332%3BTommaso&rft.au=Arena%2C%26%2332%3BFabio&rft.au=D%27Arienzo%2C%26%2332%3BSara&rft.au=Forni%2C%26%2332%3BSilvia&rft.au=Tulli%2C%26%2332%3BGiorgio&rft.au=Rossolini%2C%26%2332%3BGian+M&rft.date=1+February+2019&rft.volume=14&rft.issue=3&rft.pages=169%E2%80%93174&rft_id=info:doi\/10.2217%2Ffmb-2018-0329&rft.issn=1746-0913&rft_id=https%3A%2F%2Fwww.futuremedicine.com%2Fdoi%2F10.2217%2Ffmb-2018-0329&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-49\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-49\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Skodvin, Brita; Aase, Karina; Brekken, Anita L\u00f8v\u00e5s; Charani, Esmita; Lindemann, Paul Christoffer; Smith, Ingrid (1 September 2017). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/academic.oup.com\/jac\/article\/72\/9\/2666\/3867670\" target=\"_blank\">\"Addressing the key communication barriers between microbiology laboratories and clinical units: a qualitative study\"<\/a> (in en). <i>Journal of Antimicrobial Chemotherapy<\/i> <b>72<\/b> (9): 2666\u20132672. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1093%2Fjac%2Fdkx163\" target=\"_blank\">10.1093\/jac\/dkx163<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0305-7453\" target=\"_blank\">0305-7453<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC5890706\/\" target=\"_blank\">PMC5890706<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/28633405\" target=\"_blank\">28633405<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/academic.oup.com\/jac\/article\/72\/9\/2666\/3867670\" target=\"_blank\">https:\/\/academic.oup.com\/jac\/article\/72\/9\/2666\/3867670<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Addressing+the+key+communication+barriers+between+microbiology+laboratories+and+clinical+units%3A+a+qualitative+study&rft.jtitle=Journal+of+Antimicrobial+Chemotherapy&rft.aulast=Skodvin&rft.aufirst=Brita&rft.au=Skodvin%2C%26%2332%3BBrita&rft.au=Aase%2C%26%2332%3BKarina&rft.au=Brekken%2C%26%2332%3BAnita+L%C3%B8v%C3%A5s&rft.au=Charani%2C%26%2332%3BEsmita&rft.au=Lindemann%2C%26%2332%3BPaul+Christoffer&rft.au=Smith%2C%26%2332%3BIngrid&rft.date=1+September+2017&rft.volume=72&rft.issue=9&rft.pages=2666%E2%80%932672&rft_id=info:doi\/10.1093%2Fjac%2Fdkx163&rft.issn=0305-7453&rft_id=info:pmc\/PMC5890706&rft_id=info:pmid\/28633405&rft_id=https%3A%2F%2Facademic.oup.com%2Fjac%2Farticle%2F72%2F9%2F2666%2F3867670&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-50\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-50\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Tiseo, Giusy; Brigante, Gioconda; Giacobbe, Daniele Roberto; Maraolo, Alberto Enrico; Gona, Floriana; Falcone, Marco; Giannella, Maddalena; Grossi, Paolo <i>et al.<\/i> (1 August 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0924857922001236\" target=\"_blank\">\"Diagnosis and management of infections caused by multidrug-resistant bacteria: guideline endorsed by the Italian Society of Infection and Tropical Diseases (SIMIT), the Italian Society of Anti-Infective Therapy (SITA), the Italian Group for Antimicrobial Stewardship (GISA), the Italian Association of Clinical Microbiologists (AMCLI) and the Italian Society of Microbiology (SIM)\"<\/a> (in en). <i>International Journal of Antimicrobial Agents<\/i> <b>60<\/b> (2): 106611. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.ijantimicag.2022.106611\" target=\"_blank\">10.1016\/j.ijantimicag.2022.106611<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0924857922001236\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0924857922001236<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Diagnosis+and+management+of+infections+caused+by+multidrug-resistant+bacteria%3A+guideline+endorsed+by+the+Italian+Society+of+Infection+and+Tropical+Diseases+%28SIMIT%29%2C+the+Italian+Society+of+Anti-Infective+Therapy+%28SITA%29%2C+the+Italian+Group+for+Antimicrobial+Stewardship+%28GISA%29%2C+the+Italian+Association+of+Clinical+Microbiologists+%28AMCLI%29+and+the+Italian+Society+of+Microbiology+%28SIM%29&rft.jtitle=International+Journal+of+Antimicrobial+Agents&rft.aulast=Tiseo&rft.aufirst=Giusy&rft.au=Tiseo%2C%26%2332%3BGiusy&rft.au=Brigante%2C%26%2332%3BGioconda&rft.au=Giacobbe%2C%26%2332%3BDaniele+Roberto&rft.au=Maraolo%2C%26%2332%3BAlberto+Enrico&rft.au=Gona%2C%26%2332%3BFloriana&rft.au=Falcone%2C%26%2332%3BMarco&rft.au=Giannella%2C%26%2332%3BMaddalena&rft.au=Grossi%2C%26%2332%3BPaolo&rft.au=Pea%2C%26%2332%3BFederico&rft.date=1+August+2022&rft.volume=60&rft.issue=2&rft.pages=106611&rft_id=info:doi\/10.1016%2Fj.ijantimicag.2022.106611&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0924857922001236&rfr_id=info:sid\/en.wikipedia.org:Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<\/ol><\/div><\/div>\n<h2><span class=\"mw-headline\" id=\"Notes\">Notes<\/span><\/h2>\n<p>This presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added. The original article lists references alphabetically; they are listed by order of appearance for this version, by design.\n<\/p>\n<!-- \nNewPP limit report\nCached time: 20231215165840\nCache expiry: 86400\nDynamic content: false\nComplications: []\nCPU time usage: 1.347 seconds\nReal time usage: 1.476 seconds\nPreprocessor visited node count: 56236\/1000000\nPost\u2010expand include size: 543867\/2097152 bytes\nTemplate argument size: 164989\/2097152 bytes\nHighest expansion depth: 25\/40\nExpensive parser function count: 0\/100\nUnstrip recursion depth: 0\/20\nUnstrip post\u2010expand size: 146549\/5000000 bytes\n-->\n<!--\nTransclusion expansion time report (%,ms,calls,template)\n100.00% 1210.909 1 -total\n 90.35% 1094.078 1 Template:Reflist\n 69.69% 843.914 50 Template:Citation\/core\n 68.54% 829.929 48 Template:Cite_journal\n 11.62% 140.684 50 Template:Date\n 10.23% 123.888 133 Template:Citation\/identifier\n 5.55% 67.167 1 Template:Infobox_journal_article\n 4.62% 55.961 1 Template:Infobox\n 4.16% 50.419 1 Template:Cite_web\n 3.18% 38.474 266 Template:Hide_in_print\n-->\n\n<!-- Saved in parser cache with key limswiki:pcache:idhash:14345-0!canonical and timestamp 20231215165838 and revision id 52730. Serialized with JSON.\n -->\n<\/div><\/div><div class=\"printfooter\">Source: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.limswiki.org\/index.php\/Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology\">https:\/\/www.limswiki.org\/index.php\/Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology<\/a><\/div>\n<!-- end content --><div class=\"visualClear\"><\/div><\/div><\/div><div class=\"visualClear\"><\/div><\/div><!-- end of the left (by default at least) column --><div class=\"visualClear\"><\/div><\/div>\n\n\n\n<\/body>","49083cfbd81897f5701da971794583e5_images":["https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/6\/66\/Fig1_Mencacci_FrontCellInfectMicro2023_13.jpg"],"49083cfbd81897f5701da971794583e5_timestamp":1702682172,"9ace6d7c38d417b5bea5133d24ffe1a9_type":"article","9ace6d7c38d417b5bea5133d24ffe1a9_title":"Establishing reliable research data management by integrating measurement devices utilizing intelligent digital twins (Lehmann et al. 2023)","9ace6d7c38d417b5bea5133d24ffe1a9_url":"https:\/\/www.limswiki.org\/index.php\/Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins","9ace6d7c38d417b5bea5133d24ffe1a9_plaintext":"\n\nJournal:Establishing reliable research data management by integrating measurement devices utilizing intelligent digital twinsFrom LIMSWikiJump to navigationJump to searchFull article title\n \nEstablishing reliable research data management by integrating measurement devices utilizing intelligent digital twinsJournal\n \nSensorsAuthor(s)\n \nLehmann, Joel; Schorz, Stefan; Rache, Alessa; H\u00e4u\u00dfermann, Tim; R\u00e4dle, Matthias; Reichwald, JulianAuthor affiliation(s)\n \nMannheim University of Applied SciencesPrimary contact\n \nEmail: j dot lehmann at hs dash mannheim dot deYear published\n \n2023Volume and issue\n \n23(1)Article #\n \n468DOI\n \n10.3390\/s23010468ISSN\n \n1424-8220Distribution license\n \nCreative Commons Attribution 4.0 InternationalWebsite\n \nhttps:\/\/www.mdpi.com\/1424-8220\/23\/1\/468Download\n \nhttps:\/\/www.mdpi.com\/1424-8220\/23\/1\/468\/pdf (PDF)\n\nContents \n\n1 Abstract \n2 Introduction \n3 Related work \n\n3.1 Research data management \n3.2 Digital twins \n\n\n4 Concept architecture \n5 Research landscape and use case description of a photometrical measurement device \n6 Implementation \n\n6.1 Physical Space \n6.2 Digital Twin Space \n6.3 RDM Core Space \n6.4 Smart Application Space \n\n\n7 Proof of concept and evaluation \n\n7.1 Knowledge-based recommendation of measuring devices \n7.2 Interfacing the digital twin of a photometrical measurement device \n\n\n8 Discussion \n9 Conclusions and future work \n10 Abbreviations, acronyms, and initialisms \n11 Appendix A \n12 Acknowledgements \n\n12.1 Author contributions \n12.2 Conflicts of interest \n\n\n13 References \n14 Notes \n\n\n\nAbstract \nOne of the main topics within research activities is the management of research data. Large amounts of data acquired by heterogeneous scientific devices, sensor systems, measuring equipment, and experimental setups have to be processed and ideally managed by FAIR (findable, accessible, interoperable, and reusable) data management approaches in order to preserve their intrinsic value to researchers throughout the entire data lifecycle. The symbiosis of heterogeneous measuring devices, FAIR principles, and digital twin technologies is considered to be ideally suited to realize the foundation of reliable, sustainable, and open research data management. This paper contributes a novel architectural approach for gathering and managing research data aligned with the FAIR principles. A reference implementation as well as a subsequent proof of concept is given, leveraging the utilization of digital twins to overcome common data management issues at equipment-intense research institutes. To facilitate implementation, a top-level knowledge graph has been developed to convey metadata from research devices along with the produced data. In addition, a reactive digital twin implementation of a specific measurement device was devised to facilitate reconfigurability and minimized design effort.\nKeywords: cyber\u2013physical system, sensor data, research data management, FAIR, digital twin, research 4.0, knowledge graph, ontology\n\nIntroduction \nInitiated through the ongoing efforts of digitization, one of the new fields of activity within research concerns the management of research data. New technologies and the related increase in computing power can now generate large amounts of data, providing new paths to scientific knowledge.[1] Research is increasingly adopting toolsets and techniques raised by Industry 4.0 while gearing itself up for Research 4.0.[2] The requirement for reliable research data management (RDM) can be managed by FAIR data management principles, which indicate that data must be findable, accessible, interoperable, and reusable through the entire data lifecycle in order to provide value to researchers.[3] In practice, however, implementation often fails due to the high heterogeneity of hardware and software, as well as outdated or decentralized data backup mechanisms.[4][5] This experience can be confirmed by the work at the Center for Mass Spectrometry and Optical Spectroscopy (CeMOS), a research institute at the Mannheim University of Applied Sciences which employs approximately 80 interdisciplinary scientific staff. In the various fields within the institute\u2019s research landscape\u2014including medical technology, biotechnology, artificial intelligence (AI), and digital transformation, a wide variety of hardware and software is required to collect and process the data that are generated, which in initial efforts is posing a significant challenge for achieving holistic data integration.\nTo cater to the respective disciplines, researchers of the institute develop experimental equipment such as middle infrared (MIR) scanners for the rapid detection and imaging of biochemical substances in medical tissue sections, multimodal imaging systems generating hyperspectral images of tissue slices, or photometrical measurement devices for detection of particle concentration. Nevertheless, they also use non-customizable equipment such as mass spectrometers, microscopes, and cell imagers for their experiments. These appliances provide great benefits for further development within the respective research disciplines, which is why the data are of immense value and must be brought together accordingly in a reliable RDM system.\nResearch practice shows that the step into the digital world seems to be associated with obstacles. As an innovative technology, the digital twin (DT) can be seen as a secure data source, as it mirrors a physical device (also called a physical twin or PT) into the digital world through a bilateral communication stream.[6] DTs are key actors for the implementation of Industry 4.0 prospects.[7] Consequently, additional reconfigurability of hardware and software of the digitally imaged devices becomes a reality. The data mapped by the DT thus enable the bridge to the digital world and hence to the digital use and management of the data.[1] Depending on the domain and use case, industry and research are creating new types of standardization-independent DTs. In most cases, only a certain part of the twin\u2019s life cycle is reflected. Only when utilized over the entire life cycle of the physical entity does the DT becomes a powerful tool of digitization.[8][9] With the development of semantic modeling, hardware, and communication technology, there are more degrees of freedom to leverage the semantic representation of DTs, improving their usability.[10] For the internal interconnection in particular, the referencing of knowledge correlations distinguishes intelligent DTs.[11] The analysis of relevant literature reveals a research gap in the combination of both approaches (RDM and DTs), which the authors intend to address with this work.\nIn this paper, a centralized solution-based approach for data processing and storage is chosen, which is in contrast to the decentralized practice in RDM. Common problems of data management include having many locally, decentrally distributed research data; missing access authorizations; and missing experimental references, which is why the results become unusable over long periods of time. The resulting replication of data is followed by inconsistencies and interoperability issues.[12] Furthermore, these circumstances were also determined by empirical surveys at the authors\u2019 institute. Therefore, a holistic infrastructure for data management is introduced, starting with the collection of the measurement series of the physical devices, up to the final reliable reusability of the data. Relevant requirements for a sustainable RDM leveraged by intelligent DTs are elaborated based on the related work. By enhancing with DT paradigms, the efficiency of a reliable RDM can be further extended. This forms the basis for an architectural concept for reliable data integration into the infrastructure with the DTs of the fully mapped physical devices. \nDue to the broad spectrum and interdisciplinarity of the institution, myriad data of different origins, forms, and quantities are created. The generic concept of DT allows evaluation units to be created agnostically from their specific use cases. Not only do the physical measuring devices and apparatuses benefit in the form of flexible reconfiguration through the possibilities of providing their virtual representation with intelligent functions, but also directly through the great variety of harmonized data structures and interfaces made possible by DTs. The bidirectional communication stream between the twins enables the physical devices to be directly influenced. Accordingly, parameterization of the physical device takes place dynamically using the DT, instead of statically using firmware as is usually the case. In addition, due to the real-time data transmission and the seamless integration of the DT, an immediate and reliable response to outliers is possible. Both data management and DTs as disruptive technology are mutual enablers in terms of their realization.[1] Therefore, the designed infrastructure is based on the interacting functionality of both technologies to leverage their synergies providing sustainable and reliable data management. In order to substantiate the feasibility and practicability, a demo implementation of a measuring device within the realized infrastructure is carried out using a photometrical measuring device developed at the institute. This also forms the basis for the proof of concept and the evaluation of the overall system.\nAs main contributions, the paper (1) presents a new type of approach for dealing with large amounts of research data according to FAIR principles; (2) identifies the need for the use of DTs to break down barriers for the digital transformation in research institutes in order to arm them for Research 4.0; (3) elaborates a high-level knowledge graph that addresses the pending issues of interoperability and meta-representation of experimental data and associated devices; (4) devises an implementation variant for reactive DTs as a basis for later proactive realizations going beyond DTs as pure, passive state representations; and (5) works out a design approach that is highly reconfigurable, using the example of a photometer, which opens up completely new possibilities with less development effort in hardware and software engineering by using the DT rather than the physical device itself.\nThis paper is organized as follows. The next section points out the state of the art and the related work in terms of RDM and DTs. Both subsections derive architectural requirements, which serve to evolve an architecture for sustainable and reliable RDM. Next, specifically picked use cases of the authors\u2019 institute are outlined, followed by their implementation and subsequent proof of concept and evaluation. Finally, after a discussion that relates the predefined requirements with each other and the implemented infrastructure, the work will be concluded and future challenges will be prospected.\n\nRelated work \nIn order to better situate the present work in the state of the art, the following subsections first show the foundations of RDM, then the developments in the field of DTs. For both focal points, requirements for the development of the later introduced architecture are elaborated, which provides a basis for discussion at the end.\n\nResearch data management \nThe motivating force for reliable RDM should not be the product per se, but rather the necessity to build a body of knowledge enabling the subsequent integration and reuse of data and knowledge by the research community through a reliable RDM process.[3] Therefore, the primary objective of RDM is to capture data in order to pave the way for new scientific knowledge in the long term.\nTo bridge the gap from simple information to actual knowledge generation in order to bring greater value to researchers, data are the fundamental resource that enables the integration of the physical world with the virtual world, and finally, the interaction with each other.[13] The DT as an innovative concept of Industry 4.0 enables the convergence of the physical world with the virtual world through its definition-given bilateral data exchange. Data from physical reality are seamlessly transferred into virtual reality, allowing developed applications and services to influence the behavior and impact on the physical reality. Data are the underlying structure that enables the DT; as such, having good data management practices in place provides the realization of the concept.[1]\nSpecifically, in the context of the ongoing advances in innovative technologies, data have evolved from being merely static in nature to being a continuous stream of information.[14] In practice, the data generated in research activities are commonly stored in a decentralized manner on the computers of individual researchers or on local data mediums.[5] A recent study showed that only 12 percent of research data is stored in reliable repositories accessible by others. The far greater part, the so-called \u201cshadow data,\u201d remains in the hands of the researchers, resulting in the loss of non-reproducible data sets, devoid of the possibility of extracting further knowledge from this data.[4] In addition, the backed-up data may become inconsistent and lose significance without the entire measurement series being available. According to Schadt et al.[15], the most efficient method currently available for transmitting large amounts of data to collaborative partners entails copying the data to a sufficiently large storage drive, which is then sent to the intended recipient. This observation can also be confirmed within CeMOS, where this practice of data transfer prevails. Not only is this method inefficient and a barrier to data sharing, but it can also become a security issue when dealing with sensitive data. With such an abundance of data flows, large amounts of data need to be processed and reliably stored, causing RDM to gain momentum within the researcher\u2019s community.[14]\nBased on the increasing awareness and the initiated ambition towards a reformation of publishing and communication systems in research, the international coalition of Wilkinson et al.[3] proposed the FAIR Data Principles in 2016. These principles are intended to serve as a guide for those seeking to improve the reusability of their data assets, according to which data are expected to be findable, accessible, interoperable, and reusable (FAIR) throughout the data lifecycle. The FAIR principles are briefly outlined below within the context of the technical requirements, as modeled by Wilkinson et al.[3]:\n\nFindable: Data are described with extensive metadata, which are given a globally unique and persistent identifier and are stored in a searchable resource.\nAccessible: Metadata are retrievable by their individual indicators through a standardized protocol, which is publicly free and universally implementable, as well as enabling an authentication procedure. The metadata must remain accessible even if the data are no longer available.\nInteroperable: (Meta)-data utilize a formal, broadly applicable language and follow FAIR principles; moreover, references exist between (meta)-data.\nReusable: (Meta)-data are characterized by relevant attributes and released on the basis of clear data usage licenses. The origin of the (meta)-data is clearly referenced. In addition, (meta)-data comply with domain-relevant community standards.\nWhile the FAIR principles define the core foundation for a reliable RDM, there is also a need to ensure that the necessary scientific infrastructure is in place to support RDM.[16] In addition to the FAIR criteria, the concept of a data management plan (DMP) has a significant impact on the success of any RDM effort. The DMP is a comprehensive document that details the management of a research project\u2019s data throughout its entire lifecycle.[17] A standard DMP in fact does not exist, as it must be individually tailored to the requirements of the respective research project. This requires an extensive understanding of the individual research project and an awareness of the complexity and project-specific research data. The actual implementation of a DMP often creates additional work for researchers, such as data preparation or documentation.[16] With the aim of providing researchers with a useful instrument, a number of web-based collaborative tools for creating DMPs has since emerged, such as DMPTool, DMPonline, and Research Data Management Organizer (RDMO).[18]\nIn addition to the benefits already mentioned, the use of a research data infrastructure facilitates the visibility of scientists\u2019 research as well as identifying new collaboration partners in industry, research, or funding bodies.[5][16] In the meantime, funding bodies in particular have recognized the necessity of effective RDM, making it a prerequisite for the submission of research proposals.[16][17]\n\nDigital twins \nThe first pioneering principles for twinning systems can be dated back to training and simulation facilities of the National Aeronautics and Space Administration (NASA). In 1970, these facilities gained particular prominence during the thirteenth mission of the Apollo lunar landing program. Using a full-scale simulation environment of the command and lunar landing capsule, NASA engineers on Earth mirrored the condition of the seriously damaged spacecraft and tested all necessary operations for a successful return of the astronauts. All the possibilities could thus be simulated and validated before executing the real protocol to avoid the potential fatal outcome of a mishandling.[19][20] The actual paradigm of a virtual representation of physical entities was initiated later in 2002. After the first introduction, Michael Grieves further developed his product life cycle (PLC) model, which was later given the term \"digital twin\" by NASA engineer John Vickers. The mirrored systems approach was popularized in 2010 when it was incorporated into NASA\u2019s technical road map.[21][22][23]\nThe fundamental concept can be divided into the duality of the physical and virtual world. According to Figure 1, the physical world or space contains tangible components, i.e., machines, apparatuses, production assets, measurement devices, or even physical processes, the so-called PTs. In this context, the illustration shows a stylized device of arbitrary complexity on the left-hand side. On the right side, its virtual counterpart is shown in the virtual world or space. The coexistence of both is ensured by the bilateral stream of data and information, which is introduced as a digital thread. All raw data accumulated from the physical world are sent by the PT to its DT, which aggregates them and provides accessibility. Vice versa, by processing these data, the DT provides the PT with refined analytical information. Each PT is allocated to precisely one DT. One of the goals is to transfer work activities from the physical world to the virtual world so that efficiency and resources are preserved.[23] Systems with a multitude of devices especially require flexible approaches for orchestration. Processes and devices must be able to be varied, rescheduled, and reconfigured.[24] Twin technologies as enablers for this, providing the greatest possible degree of freedom.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 1. Concept of digital twins according to Grieves.[6][23]\n\n\n\nIn the evolution of DTs, gradations concerning integration depth can be identified. A distinction is made between \"digital model,\" \"digital shadow,\" and the \"digital twin\" itself. Sepasgozar[25] investigates this coherence and elaborates that digital models are created before the actual physical life cycle of a DT, whereas digital shadows have a unidirectional mirroring of a physical entity. To meet the characteristics of a real twin, communication must be in a bivalent way. Van der Valk et al.[26] deduce DT archetypes from characteristics as well as industry interviews. Starting from basic digital twins which, similar to a digital shadow, just represent the state of a physical object, up to increasingly complex twin variations, the following archetypes are further differentiated: enriched digital twin, autonomous control twin, enhanced autonomous control twin, exhaustive twin. Starting with the autonomous control twin, the DT emerges from its passive role and receives autonomous, intelligent features, which reach their completion in the exhaustive twin. While the first three archetypes can already be found in industry, the more advanced approaches are rather domain-limited or limited to research activities. Grieves[23] also criticizes this and argues that intelligent DTs must shift from their passive role and become active, online, goal-seeking, and anticipatory. DT technologies still need a long time to reveal their full potential. Just by identifying and focusing on the domain-specific challenges, this lack of utilizing the opportunities can be tackled.[8][27]\nAddressing some of these problems, semantic web technologies are inevitably needed labeling the required data streams.[28] Lehmann et al.[29] show that a knowledge-based approach for the representation of DTs is indispensable. Only then interaction between intelligent DTs can take place, and they are able to proactively negotiate with others so that, for example, optimal process flows emerge. Sahlab et al.[11] use knowledge graphs to refine intelligent DTs. Particularly in industrial applications, these approaches are distinguished by the management of dynamically emerging DTs. Only through reasoning over the knowledge graphs do opportunities for self-adaptation and self-adaptation emerge. G\u00f6ppert et al.[30] develop a reference architecture for the development of DTs based on an end-to-end workflow that addresses definition, modeling, and deployment for the description of a pipeline for ontology-based DT creation. Zhang et al.[31] combine DTs, dynamic knowledge bases, and knowledge-based intelligent skills to realize an autonomous framework for manufacturing cells. Due to the manufacturing context, other ontologies are relevant for the definition phase. Therefore, various other requirements and constraints are needed for different applications.\nFinally, Segovia and Garcia-Alfaro[32] investigate DTs in terms of design, modeling, and implementation and derive functional specifications. Lober et al.[33] also elaborate general specifications for DTs in their work on improving control systems based on them. Introducing a general framework and use case studies, Onaji et al.[34] show important characteristics that DTs must fulfil. \nIn accordance with the insights outlined above and the specifically developed guidelines, the following requirements have to be considered in the present work to realize proper virtual representations of physical entities:\n\nReplication, representation, and interoperability: The virtual counterpart of a physical entity should be as detailed as possible, but at the same time as less complex as required without violating the fidelity of the replicated device. A representation should not only include the data of a device but also describe the meaning of this data to lay the foundations for autonomous interoperability.\nInterconnectivity and data acquisition: All physical devices must be connected bi-directionally via suitable communication standards. The incoming data must be processed in a time-appropriate manner and reflected in the twin. The data forms to be taken into account can be of a descriptive, static, or dynamic nature and must be considered accordingly during processing. Processed information from the DT must also be reflected back into the PT.\nData storage: All aggregated data must be stored agnostic of format immediately. For reusability, it is necessary to store the data with reference and labeling in suitable storage forms. Not only time but also version, as well as change management, are useful options regarding this.\nSynchronization: Whenever possible, the bivalent data connection should be carried out in real-time and under adequate latency conditions. Both twins should replicate the condition of their counterparts if possible.\nInterface and interaction In order to enable collaboration and interaction between and with the twins, suitable interfaces are required. On the one hand, it must be possible for data to be exchanged and accessed by machines, and on the other hand, data must be readable and interpretable by humans providing suitable interaction modes.\nOptimization, analytics, simulation, and decision-making: To gain further advantages, additional features should be accessible through the DTs. Thus, real-time analyses and optimizations, as well as independent algorithms for data evaluation, can be applied to the data basis of the DT. It should be possible to use AI technologies, establish decision making, or use far-reaching simulations, for example. The DT is intended to create context awareness and to facilitate collaborative approaches to reliably choreograph the twins.\nSecurity: Each entity must comply with current security standards, i.e., authorization, policies, and encryption. Both privacy and integrity must be preserved. Optionally, the DT could monitor the current security through \"what-if\" scenarios and initiate countermeasures.\nAfter a detailed examination of both concepts (RDM and DTs), it is obvious that symbiosis of both can draw certain advantages. The requirements for reliable and sustainable RDM especially align well with the DT characteristics described above. Both data management and DTs\u2014as disruptive technology\u2014are mutual enablers in terms of their realization.[1] The works analyzed in this section reveal a gap in research, which this paper attempts to address. The synthesized architecture built out of these pillars is presented subsequently.\n\nConcept architecture \nAfter the relevant requirements for a sustainable RDM leveraged by intelligent DTs have been elaborated, the architectural concept will be introduced and aligned regarding these requirements. Subsequently, a decentralized RDM infrastructure is presented facing general data management problems within a research and development institute. Due to the wide range and interdisciplinarity of the institution, countless data of different origins, formats, and quantities are generated. In the authors\u2019 context, data from mass spectroscopy and spectrometry must be specifically assumed, especially in the field of process analytics and medical technology. However, general infrastructure approaches are to be built up agnostically so that the RDM can also be operated independently of use cases. By enriching it with DT paradigms, the efficiency of a reliable RDM can be further expanded. Thus, not only the physical measuring devices and appliances benefit in the form of flexible reconfiguration facilitated by the possibilities of incorporating their virtual representation with intelligent features, but also directly the wide variety of harmonized data structures and interfaces that are empowered by DTs. Hence, typical data management problems\u2014i.e., many locally, decentrally distributed research data, missing access permissions, and missing experimental references, which is why the results become unusable over long periods of time\u2014are addressed. The proposed approach depicted in Figure 2 tries to overcome these issues.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 2. Conceptual RDM architecture.\n\n\n\nThe overall architecture is split into two main areas: the Physical Space in which the physical measuring devices and research equipment are settled, and the RDM infrastructure itself, which is subdivided into three functional layers: the Digital Twin Space, the RDM Core Space, and the Smart Application Space.\nFrom the bottom up, Figure 2 shows there are the physical devices generically referred to as PT 1\u2013PT n. In accordance with Grieves\u2019 twinning paradigm, these are uniquely linked to their digital counterparts situated within the Digital Twin Space. The data-driven representations must be enriched with semantic information content. In this way, it is possible to derive a machine-readable information model. Due to the asynchronous nature of many measurement procedures, the bivalent data pipeline between the two twins is l-driven so that data synchronicity is preserved. The DTs in the Digital Twin Space consequently aggregate all data and static, structural information from the Physical Space and provide it in a harmonized form to the superimposed infrastructure layers through standard communication interfaces.\nThe middle layer of the infrastructure, the RDM Core Space, contains the main elements required for reliable and sustainable RDM. These include a knowledge graph, a storage environment, and a messaging broker. Special attention should be paid to the DT Orchestration Service (DTOS), which takes over the choreography of the DTs with all the aggregated data, information, and requests from all participants of the RDM infrastructure that arise.\nStarting with the knowledge graph, it offers itself as an environment for storing all domain-specific knowledge through an ontology. It is intended to organize the entire semantic information of the DT information models. As a sub-discipline of AI, such a knowledge-based approach should bring with it possibilities for reasoning and inferring complex system interrelationships. In this way, DT should be harnessed with intelligence through the knowledge graph.\nThe second pillar of the RDM Core Space involves examining storage approaches for all accruing forms of data. Because of the different measurement methods and data sources, it also needs different concepts for storage to be considered. For example, some devices deliver a continuous data stream, others asynchronous data points or data sets, and still others preprocessed data, i.e., from imaging measurement procedures. This requires, on the one hand, the necessity of archiving time series data and, on the other hand, a conventional repository-based file system approach. To ensure reusability and interoperability, experiment-specific data must be labeled and versioned. This is also done on a semantic basis so that the experiment data can also be located within the knowledge graph in order to create intelligent links at later stages and to be able to put data sets into context to other ones.\nA messaging broker is also envisaged as a central RDM Core Space element that can be accessed anywhere within the architecture. This constitutes an asynchronous, event-driven communications interface for live data of all intended layers and ensures that everyone has non-discriminatory access to all necessary data.\nThe last major component of the RDM Core Space, the DTOS, has extra intersections with the lower as well as the superordinate layers. The DTOS manages and orchestrates the entire RDM Core Space and thus simultaneously enables intelligent interplay of the DTs. A living part of the DTs is located in the DTOS and gives them functional freedom of action beyond the mostly passive Digital Twin Space. Combining the DTs with the stored knowledge within the knowledge graph results in powerful tools for superposed smart applications. Hence, the responsibility for reading out the DT information models and creating them within the knowledge graph also lies here. For both data and DTs, lifecycle management is established, so that sustainability and reliability in RDM are created. The DTOS should also provide access to the administration of the DTs as well as the versioning and management of the achieved data in storage. The DTOS can also be utilized by the top-level Smart Application Space through interfaces in order to trigger intelligent functionalities between the DT and the accumulated data.\nThe top-level Smart Application Space allows arbitrary services to consume data via the interfaces of the messaging broker or the DTOS and use it for their purposes. It would also be conceivable for smart applications to proactively offer their capabilities as a service to the DTs of the devices, or even as a service twin. Realizing this, they could also be represented in the knowledge graph and the Digital Twin Space and get into contact with other DTs.\nDue to the nature of the research devices, a partly decentralized (data acquisition and preprocessing are commonly facilitated decentrally), mostly asynchronous event-based architecture is needed. Instead of choosing a monolithic software approach, which makes perfect sense on a central system, independent microservices are utilized here. A microservice-based architecture offers the greatest possible advantages in the context of RDM through separate areas of responsibility, independence, autarky, scalability, and fault tolerance through modularity.[35] In order to make the later implementation approaches more comprehensible, a typical use case of the authors\u2019 research institute is presented subsequently.\n\nResearch landscape and use case description of a photometrical measurement device \nAfter the basic concept architecture for RDM has been presented, the subsequent implementation of a PT will take place on the basis of a specific measuring device and its use cases in order to integrate it prototypically within the RDM architecture as a first application example. Therefore, the research and device landscape of the institute will be considered first. The CeMOS conducts interdisciplinary research in the fields of medical biotechnology or medical technology and intelligent sensor technology in order to create synergies between mass spectrometry and optical device development. Based on a variety of covered research areas, several devices from different manufacturers, as well as self-built ones, are used to create a wide-ranging heterogeneous equipment landscape. It includes microscopes, cell imagers, and various mass spectrometers for generating hyperspectral images, as well as other hyperspectral imagers for specific use cases and optical measuring devices. Some of these imagers and measuring devices were developed, built, and are currently operating at the institute itself. \nRepresentatives of these self-developed and manufactured measuring devices are the MIR scanner[36], the Multimodal Imaging System[37], and a multipurpose, multichannel photometer. The MIR scanner is used for generating hyperspectral images of tissue sections and consists of a laser unit with four lasers with different wavenumbers, a detector unit, a focusing unit, an agile mirror unit, and a movable object slide. It is used for frozen section analysis in tumor detection for the identification of tissue morphologies or tumor margins.[36] The Multimodal Imaging System also generates hyperspectral data for tissue sections with various procedures and consists of a modular upright light microscope combined with a Raman spectrometer, a visible (VIS) \/ near-Infrared (NIR) reflectance spectrometer, and a detector unit. Its applications are in brightfield, darkfield, and polarization microscopy of normal mouse brain tissue, and an exemplary application provides the ability to make a distinction between white and grey matter.[37] The majority of all devices currently do not use a network interface. The resulting measurement data are mostly stored locally and manually collected for analysis purposes. Due to the decentralized processing of the data, no problems arose with regard to security and confidentiality. Likewise, due to the low level of automation, no problems occurred with regard to emerging experimental errors or technical failures. Manual intervention could directly mitigate these errors. In the future, with an increased degree of automation of the RDM infrastructure, issues regarding security, confidentiality, and functional safety have to be taken into account. In addition to the previously mentioned devices, the photometer also serves as an essential component of the institute\u2019s research. For the subsequent implementation of the presented architecture, the feasibility is to be proven on the basis of the photometer.\nThe developed photometer system, schematically shown in Figure 3, essentially consists of three main components: the parts for digitizing the analog sensors, the driver for controlling the light sources, and a powerful microcontroller.[38] These components and their interaction are described in the following. Depending on the application and measuring principle (transmission, reflection), photodiodes with different spectral sensitivities are used. Ideally, these sensors should have high photon sensitivity, fast response time, and low capacitance. Since these criteria are in mutual interaction, an application case-individual consideration is necessary. Exposure of the photodiode causes electrons to be released from the photocathode, resulting in a slight change in the diode\u2019s dark current. A current-to-voltage converter integrated circuit (IC)[39] senses this photocurrent from the diode. The application-specific integrated circuit (ASIC) is a low-noise sensor interface and is suitable for coupling optical sensors with current output. These input currents are quantized into a digital output signal (with up to 16 bits, depending on the integration time). The integration time can be varied between 1 ms and 1024 ms, and the current sensitivity can be varied in steps from 20 fA\/Least Significant Bit (LSB) to 5000 pA\/LSB. Measurements can be continuous or manually triggered. An advantage of the integration of the input signals performed by the device is the resulting significant increase in the dynamic range. Furthermore, high-frequency components are filtered, and periodic disturbances with a multiple of the period duration are suppressed.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 3. Schematic layout of the physical photometer system.\n\n\n\nThe microcontroller, connected via an inter-integrated circuit (I\u00b2C) interface, is configured in 400 kHz fast mode to communicate with the sensor interfaces. The parameters for writing and reading the analog-to-digital converters (ADCs) must follow a format specified by the manufacturer. The current state of each ADC (i.e., measurement running, measurement finished) can also be queried by reading special registers. Likewise, dedicated general purpose input\/output (GPIO) pins can be used to signal the status of the ADCs to the controller as an interrupt request (IRQ). This avoids permanent polling of the corresponding register or GPIO pin and saves resources. After a completed measurement, the controller reads the corresponding data packet via the internal interface. Subsequently, a new measurement can be initiated.\nIn addition to communication with the ADCs, the controller has the task of controlling the LEDs. These light sources are controlled via switchable constant current sources. The selection of the light-emitting diodes used is again very much dependent on the selected measuring principle and the detector. In addition to the wavelength and power of the light source, the rise times (\ud835\udc61\ud835\udc5f\ud835\udc56\ud835\udc60\ud835\udc52) and fall times (\ud835\udc61\ud835\udc53\ud835\udc4e\ud835\udc59\ud835\udc59) in particular must be taken into account in the LED selection. These times can be stored in the controller as parameters of each light source individually. Before starting and after finishing each measurement, these specific times are taken into account. Especially for complex measurement setups, low detection limits, and\/or short measurement duration, these parameters have a significant influence on the results and the maximum possible scanning speed. Via an integrated USB connection, all parameters can be configured between the photometer and the computer, and the raw measurement data can be sent. By means of the built-in ethernet PHY IC DP83825 from Texas Instruments[40], 10\/100 MBit communication via Ethernet is also possible. An internet of things (IoT) interface implemented on the software side, consisting of a Hypertext Transfer Protocol (HTTP) server and Message Queueing Telemetry Transport (MQTT) client, enables the connection to further IT infrastructure or web services.\nThe design of the photometer is highly flexible given its configurability and modularity. According to the selected configuration of the individual components, measurements can be performed in wavelength ranges of ultraviolet (UV), VIS, NIR, and infrared (IR). Furthermore, measurements of, e.g., particle sizes can be performed with special probe designs adapted to the task. It is even possible to conduct Raman measurements with probes that are extended by additional optical components. Some specific examples are listed below:\n\nUse of a scattered light sensor for monitoring the dispersed surface in crystallization: The specific surface area of the dispersed phase in suspensions, emulsions, bubble columns, and aerosols plays a decisive role in the increment of heat and mass transfer processes. This has a direct effect on the space-time yield in large-scale chemical\/process engineering production plants. An easy-to-install optical backscatter sensor outputs the dispersed surface area as a direct primary signal under certain boundary conditions. The sensor works even in highly concentrated suspensions and emulsions, where conventional nephelometry already fails. Several trends and limitations have been found so far for the sensor, which can be used in-line in batch and continuously operated crystallizers, even in harsh production environments, and in potentially explosive zones. The specific dispersed surface is directly detected as the primary measurand.[41]\nDevelopment and application of optical sensors and measurement devices for the detection of deposits during reaction fouling: In many chemical\/pharmaceutical processes, the technically viable efficiencies and throughputs have not been achieved yet because of the reduction in heat transfer (e.g., in heat transfer units, reactors, etc.) due to the formation of wall deposits. Considerable amounts of energy can be saved by reducing or entirely preventing this problematic area. Therefore, a measurement device and its optical and electronic parts were developed for the detection and measurement of deposits in polymerization reactors, simultaneously aiming the in-line monitoring. The design strategy was carried out systematically via theoretical calculations\u2014such as optical ray tracing and photon flux analysis\u2014via test designs, laboratory investigations, and then industrial use. The developed sensors are based on fiber-optic technology and thus can be integrated into the smallest and most complex apparatus, even in explosion-hazardous areas. Critical product and process states in the reactant are detected at an early stage by combining several multi-spectral backscattering technologies. Thus, the formation of deposits can be prevented by changing process parameters.[42]\nPhotometric inline monitoring of the pigment concentration of highly filled coatings: This involves inline monitoring of particle concentration in highly filled dispersions and paint systems using fiber-optic backscatter sensors. Due to the miniaturization of the distance between emitter and receiver fiber to <600 \u00b5m, the transmitted light can also penetrate high dispersion phase fractions of up to 60%. Due to the measurement setup, both transmission and scattering influences are found in the resulting signal. In this setup, the photometer is configured with detectors and light sources for the red wavelength range (660 nm). The measurement interval of 128 ms is sufficiently small to allow very close monitoring of the measured values.[43]\nAs described above, a number of hardware and software settings and modifications have to be made, especially during the pre-test phase, in order to fulfil the intended task. This usually requires a modification of the firmware of the photometer with the corresponding parameters of the installed components. This time-consuming step, which cannot be performed by every end user, can be eliminated by utilizing the DT of the physical device. Hardware-specific settings can be made comfortably via a graphical user interface (GUI). At the same time, a plausibility check of the selected parameters can be realized in a simple way. A misconfiguration of the device can be made more difficult, and the end user has the option to check the settings again.\n\nImplementation \nThrough the proposed architectural approach on the one hand and the described use case on the other hand, the prototypical implementation should be outlined on this foundation subsequently. Thus, the general structure and deployment of the main components and their interrelationship will be presented.\nFigure 4 illustrates the further developed concept architecture. In each individual layer, the utilized microservice instances are depicted. Instead of just using templates of PTs and their DTs in theory, as shown in the concept, the photometer presented in the use case facilitates a complete integration scenario within the RDM infrastructure. It will proceed from the bottom up beginning with the development of the physical photometers representation to further derive its information model for the corresponding DT. Afterward, the entire integration of the RDM Core Space is executed by the DTOS. Underlining the implementation and the integration scenario of the photometer, a proof of concept will be carried out later.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 4. Implementation of the conceptual RDM architecture.\n\n\n\nEvery implemented component is built up microservice-based. These microservices are deployed within a distributed server environment at the research institute. Depending on required performance and space, such a microservice infrastructure can be deployed as scalable via Docker, a Docker swarm, or even a Kubernetes cluster. Because of the high complexity of such an infrastructure which a reliable and sustainable RDM requires, this section will be further subdivided into several subsections. After prospecting the physical setup of the Physical Space, the Digital Twin Space will be illuminated. Followed by the RDM Core Space, in which the interrelations between main objective functionalities of RDM are laid down, and a brief overview regarding utilized, as well as potentially realizable, smart applications within the Smart Application Space are shown.\n\nPhysical Space \nThe lower layer of the architecture is the Physical Space, which contains all physical devices. In this case, the implementation of the Physical Space is exemplarily reduced to the photometer introduced in the research landscape. The other pre-presented devices are featured at the proof of concept level, demonstrating and validating the functionality of the RDM infrastructure. Later, this layer should be extended by the heterogeneous research equipment of the institute.\nThe photometer previously described above will be used to demonstrate the seamless integration of a research measuring device. For the connection, the IoT interface of the photometer PT is foreseen. The device logs into the Digital Twin Space on every boot sequence via its integrated Representational State Transfer (REST) interface and transmits its structural configuration to it. This auto-deployment ensures that the state between PT and DT is always up to date. All the configurations are embedded in the form of an information model in a JavaScript Object Notation (JSON) file which is directly readable for the overlaying architecture layers. The JSON-formatted information model is recognized in Appendix A and exactly mirrors the measuring capabilities and functionalities for the setup of experiments, which was outlined before. Important parameters for identification and policy are declared at the beginning of the document. Then the attribute part describes the semantic meta contexts and capabilities of the device. Finally, the setting parameters, actuators, and sensors are described as features. For reasons of clarity and space, LED3\u2013LED6, as well as adc2\u2013adc4, have been substituted. Their structure is analogous to the ones shown. Based on the physical structure of the PT, it is one of the main components of the later DT representation. Thus, it serves not only as a data basis for all applications infrastructurally settled above it but also as a semantically enriched information model, which the DTOS uses to describe the device within the knowledge graph.\n\nDigital Twin Space \nAt the base of the RDM infrastructure itself, the Digital Twin Space contains the DTs of the physical measuring devices. It is based on the open-source project Eclipse Ditto[44], which aims to cope with representations of DTs. With a scalable basis, the Ditto project offers the possibility of integrating physical devices and their digital representations at a high abstraction level. Not only the organization but also the entire physical-virtual interaction is thus made possible for further back-end applications in a simplified manner. The PT and its DT can be accessed bi-directionally via the provided application programming interface (API). As a result, the Physical Space can be influenced by changes within the Digital Twin Space. Eclipse Ditto is, as the rest of the authors\u2019 infrastructure, built on various microservices. Individual scalability, space-saving deployment, and separation of different task areas as a robust, distributed system, let the project become a universal middleware for the provision of DTs. The essential system components of Eclipse Ditto are briefly outlined below.\n\nConnectivity service: Ensuring frictionless communication between physical devices, their virtual counterparts, and data consuming back-end applications, the Connectivity service provides a direct interface for various protocols and communication standards such as HTTP, Websockets, MQTT, or Advanced Message Queuing Protocol (AMQP). A specially developed, unified JSON-based Ditto Protocol as the payload of messages of the listed communication standards opens up numerous interaction possibilities. For example, messages can be also mapped via scripts for preprocessing and post-processing, as well as structuring. Furthermore, by using the Ditto Protocol, the entire Ditto instance can be managed, thereby a complete interface is established to interact efficiently with the DTs and their physical counterparts.\nThings service: The Things service contains the actual structure and telemetry representation of the PTs. This abstract representation consists of a simple JSON file. While the first part of the JSON includes the static describing attributes of a DT, such as a unique identifier, the assigned policy, or other semantically describing properties, the second part contains the dynamic features to which all telemetry data belong. These mirror the constantly changing status of the PTs.\nPolicies service: Individual permissions for access and management of the twins, preserving privacy and integrity, are managed by the policy microservice. In order to grant finely graded read and write permissions to certain subjects, Eclipse Ditto offers the Policies service concept that can be easily modified via specific Ditto Protocol communication patterns. In addition to extensible certificate-based security mechanisms which Eclipse Ditto naively offers, this setup forms the foundation for the fulfilment of modern security standards.\nTo substantiate the advantages which are brought by Eclipse Ditto, the photometer DT should be further instantiated at the Things service. Therefore the before introduced representation form is used and aligned to the requirements of the Ditto Protocol. As a result Appendix A with its photometer JSON representation can be reviewed. The part with the key attributes at the beginning of the file contains all static and semantic necessary information to draw later benefits from. The second part with the key features contains the dynamic telemetry data, which are transmitted while operating constantly via MQTT from the PT to the DT and are further consumed by back-end applications or storage purposes. The responsibility for proper connections in direction of the superordinated architectural layers is preserved by the Connectivity service. On top of this middleware-like DT abstraction layer, value-generating features, i.e., the subsequently introduced RDM infrastructure, can be constructed.\n\nRDM Core Space \nOn top of the Digital Twin Space, the RDM Core Space layer is settled. Here, the orchestration of the infrastructure and DTs takes place, the generated data are managed, and the communication service is provided. The RDM Core Space includes the DTOS, Apache Jena Fuseki as the knowledge graph, a combination of InfluxDB and Dataverse for storage, and Eclipse Mosquitto as the message broker. These instances provide various functionalities that are necessary for the microservices of the RDM infrastructure. Starting on the left with Apache Jena Fuseki, the individual infrastructure components are explained in order to subsequently characterize the features of the DTOS and thus fully cover the RDM Core Space later on.\nApache Jena Fuseki is a web ontology server that stands out from other alternatives such as Neo4j due to its higher performance. Although Jena Fuseki offers less flexibility in the area of integration of multiple sources, performance is the key criterion for this infrastructure.[45] In comparison to JanusGraph, another alternative, Jena Fuseki also predominates in terms of performance.[46][47] In addition, Jena Fuseki uses the common query language SPARQL, while Neo4j or JanusGraph use the less common languages Cypher and Gremlin. Based on these mentioned arguments, Jena Fuseki is implemented within the infrastructure; however, a more extensive analysis of the suitability of Apache Jena Fuseki will be conducted in the future.[48] \nApache Jena Fuseki is implemented on a dedicated server in a Docker container and offers the ability to receive and answer SPARQL queries. It includes a REST API that is used to create the semantic representation of the DTs and to communicate with the microservices across the infrastructure. In addition, a web interface can be used to submit SPARQL queries directly through an input form.[49] In order to process queries, it is necessary that Apache Jena Fuseki contains a domain-specific ontology in which the entire semantic information of the DTs information model can be captured. The domain-specific ontology was designed with Prot\u00e9g\u00e9 according to the requirements of RDM and the DT representation. Figure 5 depicts the top-level ontology for RDM, allowing for the development of further complex sublevel ontologies in the future due to its modular structure. It enables the DTs to be described semantically with minimal complexity, along with contextualizing the generated data of their PTs. The ontology\u2019s basic structure is inspired by Lehmann et al., who presented an ontology for production resources and products.[29]\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 5. Top-level knowledge graph for RDM.\n\n\n\nIn the context of this work, the ontology was adapted and further fitted to the needs of measurement devices and RDM. At the top hierarchy, the ontology is divided into five logical sections, corresponding to the classes Resource (yellow), Service (green), Target (red), Data (blue), and Manufacturer (black). As such, due to the underlying use case, it has been developed starting from the Resource class. The relationship between the individual classes is as follows. Each Resource has a Manufacturer and provides a specific Service for a particular Target and generates specific Data from it. These relations can equally be expressed in an inverse manner on the basis of the generated reasoning and inferences, as Figure 5 shows. The ontology\u2019s four top-level classes of Resource, Service, Target, and Data are further divided into sub-classes as shown by the logical sections. The Resource (yellow) has the SubResource \"Measurement Resource,\" which contains the sub-class \"Sensor\" and enables the Measurement Resource to gather data. In order for the Measurement Resource to collect data, it must provide a specific Service. This Service (green) is provided in the context of the Services sub-class as the \"Measurement Service.\" For a more precise specification of the given device landscape, the top-level class Target (red) is divided into the three sub-classes within the ontology: Tissue Slice, Surface, and Suspension. The classification of the top-level Data (blue) is thereby based on the degree of structuration into structured, semi-structured, and unstructured data. In the development of this domain-specific ontology, great efforts were made to ensure the best possible foundation for representing the semantic characteristics of the DTs and their data. At the same time, due to its modular structure, it offers future connecting points to roll out the ontology to the entire context of the institute and its requirements. Furthermore, the demonstrated ontology, in association with the Apache Jena Fuseki web server, offers the possibility to provide knowledge-based recommendations via use case specifications, as demonstrated in the proof of concept.\nThe next essential part of the implementation is the storage, which is implemented as a combination of InfluxDB and Dataverse. The hereby united different storage concepts cover a maximum number of use cases and fulfil the RDM and DT requirements as effectively as possible. For storing discrete data points, InfluxDB, an open-source time series database, is used.[50] The InfluxDB stands out from other popular time series databases such as Prometheus, Druid, or OpenTSDB due to its query response time.[51] Compared to Prometheus, InfluxDB offers an SQL-like query language, the possibility to manage user rights, and in-memory capabilities.[52] In addition, the InfluxDB features a better compression ratio than Druid and OpenTSDB. These advantages make InfluxDB suitable for storing time series data within the infrastructure.[51] InfluxDB is able to sign incoming data with a timestamp and classify it into corresponding buckets, which can be assigned to sensors in even more detail with the help of further criteria from the DT. InfluxDB comes with a REST API which allows data to be queried or sent, as well as a comprehensive web interface, which enables buckets to be searched and data to be visualized, further making data more findable and accessible. Moreover, the web interface allows the creation of dashboards to enable live monitoring. \nBesides the discrete data points in the sense of measurement series, other data, such as hyperspectral images, are generated at the institute. InfluxDB is not suitable for storing this type of data, which is why Dataverse is also implemented and serves other storage concepts. It is an open-source web application that allows publishing, storing, citing, and providing research data in associated repositories.[53] Besides Dataverse, there are other alternatives such as Zenodo for storing various research data in repositories. Dataverse is distinguished from Zenodo by its more advanced authentication options and the superior concept of version control.[54] As a result, Dataverse is implemented within the infrastructure. The repositories in the Dataverse are called Dataverses and can be subdivided for example by working groups or projects. Within the Dataverses so-called Datasets can be created, in which data, e.g., from measurements, can be saved. The Dataverse Project offers extensive metadata management and makes it possible to describe the individual Datasets more exactly, which facilitates interoperability of data. Furthermore, the Datasets can be directly linked to a publication with a digital object identifier (DOI), allowing the user to extract corresponding citations directly from the web application. Metadata management within Dataverse contextualizes the stored data and provides a high level of reusability for other users.[55] Dataverse also provides a REST API to upload or query data, which simultaneously can be uploaded and searched with different filters for the metadata in a simplified manner via the web application. Thus, requirements of different users are served by it. Analogous to Apache Jena Fuski, InfluxDB and Dataverse are also implemented on a provided server within the distributed environment.\nThe next element of the implementation in the RDM Core Space is the message broker, which is implemented by Eclipse Mosquitto. It is a message broker that supports the MQTT communication protocol and enables interaction within the infrastructure and its microservices.[56] At this stage of development, MQTT is used due to its less required deployment resources. In the future, however, this communication interface will be enhanced by AMQP due to the buffering and the larger range of functions, in order to thus be able to serve a wider scope of requirements.[57] In this context, Eclipse Mosquitto provides the link between the PTs and DTs, the DTOS and DTs, the DTOS and Node-RED, and the DTOS and smart applications in general. By connecting the PTs and the DTs, a bidirectional connection of both units is formed and their interconnectivity is ensured. Eclipse Mosquitto provides different security levels for the message traffic, ranging from \"no security\" to \"encryption\" with Transport Layer Security (TLS) and TLS with a client certificate. The implementation currently provides no encryption for the message traffic, but this will be addressed in the future. The broker is located on the provided server for the infrastructure and operates beside the previously presented elements of the RDM Core Space.\nIn addition to the presented instances within the RDM Core Space, the DTOS represents a central component at this layer of the infrastructure, which performs different orchestration tasks as an event-driven microservice and is currently implemented as a Python-based microservice. It captures the registration of new DTs, and it creates their semantic representations in Jena Fuseki and a file system in the InfluxDB. Furthermore, the DTOS connects the Digital Twin Space with the Smart Application Space and thereby enables intelligent cooperation between the DTs. The sequence chart shown in Figure 6 illustrates the operations inside the infrastructure and the DTOS, which are outlined in more detail below.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 6. Registration of a digital twin within the RDM Core Space.\n\n\n\nThe entire process starts with a PT being activated and registering with its underlying information in the DT information model via a HTTP registration request. Through this registration request, a DT is provided for the PT with the help of Eclipse Ditto. After the DT is successfully instantiated, it notifies the DTOS of its registration via MQTT. This is the event trigger for the DTOS, which now submits a HTTP request for the DT\u2019s information model. The DT then provides its information model to the DTOS. This information model contains, as shown in Appendix A exemplarily, the photometer, attributes, and features besides general properties. The general properties and the structure of the JSON with the keys attributes and features originate from the Eclipse Ditto information model and are necessary for registration, as well as instantiation. All components of the device like sensors or ADCs are included in the key features. Additionally, information for the DTOS is included with the variable regComplete, which is set to false by default. This variable is altered from false to true with the registration of the DT within the infrastructure and controlled by the DTOS to prevent the duplicate creation of the file system in the InfluxDB and the semantic information on the Apache Jena Fuseki server. The DTOS starts to extract the semantic information about the DT from the information model if the check of the variable regComplete results in the fact that the DT has not yet been registered. These semantic information are contained in the key attributes and structured as a resource description framework (RDF) triple according to the Web Ontology Language (OWL) to simplify its creation in the knowledge graph. After the positive check of the regComplete variable, the DTOS parses the content of the key attributes and generates a SPARQL query using the HTTP POST method and thereby transmits it to Apache Jena Fuskei, instantiating the DT in the knowledge graph and all its sensors as new individuals inside the presented ontology. Simultaneously, the DTOS creates a file system for the DT in the InfluxDB via HTTP to store its generated data. Subsequently, the DTOS reports the complete registration process to the DT, and the DT returns the notification of successful registration to its PT via MQTT. Thus, the PT is fully integrated into the infrastructure and can start generating data that now are saved via the DTOS in the associated file system and made accessible to other microservices as well as the smart applications in the Smart Application Space. In addition, the semantic information can now be queried through the knowledge graph and taken into account in queries about specific device properties.\nBesides the orchestration of the DTs, the creation of file systems and semantic representations, the DTOS takes over the lifecycle management for the generated data as well as the DTs. For this purpose, the DTOS monitors the DTs registered in the infrastructure and enables their deregistration if required. During deregistration, the DTOS removes the semantic representations from the knowledge graph and moves the DT\u2019s data to the long-term archive, which is provided via the Dataverse, as well as deletes them when they reached the end of their lifecycle. Similarly, the DTOS monitors the generated data over its lifecycle from creation to publication to archiving. In doing so, the DTOS links the data to the associated publications and then moves the data to the archive until it deletes it at the end of the life cycle.\n\nSmart Application Space \nSmart applications in the RDM infrastructure are settled at the top of the hierarchy. Applications for the evaluation of experiments and the creation of added value are to be located according to the infrastructure modalities that are as open as possible. Examples of such smart features would be AI algorithms for the evaluation of medical image data, context-based correlation of multi-dimensional parameter fields, optimization procedures for measurement arrangements, and much more. Some of the applications already implemented and those planned for the near future are discussed below.\nOriginally developed by IBM, the open-source software Node-RED is a tool for flow-based programming and is used for connecting hardware components, APIs, and online services. Node-RED provides an editor through the web browser that enables a graphically supported creation of flows with different nodes.[58] With Node-RED further microservices for the RDM infrastructure are implemented, such as the knowledge-based recommendation system for measuring devices. With the knowledge-based recommendation system, a tool is implemented in the Smart Application Space that facilitates the selection of measuring devices for the end user. For this purpose, an input mask is set up in a Node-RED dashboard with which the parameters for a measurement to be performed can be specified. Based on these specifications, the microservice generates a SPARQL query and sends it to Apache Jena Fuseki. The response from the knowledge graph is output in tabular form with the required parameters. With the help of the input mask the required service, the target, and the output data can be defined. In addition, the required wavelength can be specified either as a specific value or as a range. All entries are optional and serve the refinement of the search filter. This recommendation system is featured in the proof of concept for the RDM infrastructures functionality validation.\nIn the future, further smart applications will be integrated into the infrastructure, such as scientific trial management, which provides two essential features for reliable and sustainable RDM. The first aspect of scientific trial management is the standardized creation of Dataverses and Datasets within the Dataverse. This is made possible via a web-based GUI in which using standardized catalogs Dataverses can be created or the metadata for projects Dataset can be specified. During the creation of a Dataset, the affiliation to a corresponding Dataverse can be established. The list of existing Dataverses is continuously updated to avoid duplicates. The use of standardized catalogs prevents different spellings and establishes a joint terminology among the institute\u2019s researchers. This increases the findability of the data in the Dataverse via the metadata search. \nThe second aspect of scientific trial management is the export of timer series data from the InfluxDB into a Dataset within the Dataverse. The export of time series data enables the movement of PT data to the Dataverse after a measurement series and thus increases its findability, accessibility, and reusability. An input mask is used to select the PT for which the data needs to be extracted and specify the Metadata on the basis of the standardized catalogs. The process gets triggered by an integrated button in the input mask. Based on the entered name, the corresponding bucket is determined in the InfluxDB and the data for the specified period is retrieved. The microservice then creates a Dataset in the Dataverse according to the specifications and metadata of the input mask and saves the data in a structured, neutral tabular format to improve interoperability and reusability.\nIn addition to the scientific trial management efforts, prospective activities include the implementation of an application for the user-friendly creation of data management plans in the Smart Application Space in order to fully meet the requirements for a reliable RDM, including the FAIR criteria to fully and sustainably document the project\u2019s own data lifecycle. For this purpose, the established software RDMO will be integrated into the infrastructure, which will be connected to the existing microservices using its native API, offering the researchers a centralized and standardized tool for the creation of data management plans.\n\nProof of concept and evaluation \nFollowing the implementation of the presented RDM architecture, a two-stage proof of concept with a respective concluding evaluation will demonstrate the advantages and the practicability of the authors\u2019 architectural approach. First, a knowledge-based recommendation system is outlined to illustrate one use case and the benefits of a knowledge graph. Subsequently, the practical implementation of a DT is presented using the photometer. Afterward, it is demonstratively visualized by a real measurement series. Although the demonstration is based on a measurement series, the concluding evaluation is only qualitative. The focus of this work is the introduction of a novel infrastructure for RDM, not the investigation of a specific experimental context.\n\nKnowledge-based recommendation of measuring devices \nThe first stage of the proof of concept is performed with a knowledge-based recommendation system to demonstrate the simplified identification of suitable measurement devices for generating research data enabled by the developed RDM infrastructure. For this purpose, three measuring devices have been selected and equipped with the necessary control system for their integration into the infrastructure and registration ability within Eclipse Ditto.\nIn addition to the photometer, the MIR scanner and the Multimodal Imaging System have been chosen because of their representative character. To enable this proof of concept, the individuals OpticalMeasurementService, MFG_1 (corresponding to the photometer), MFG_2 (corresponding to the Multimodal Imaging System), MFG_3 (corresponding to the MIR scanner), DiscreteDataPoint, HyperspectralImage, SkinLikeLiquid, and MouseBrain have been inserted into the ontology. The OpticalMeasurementService describes the type of offered service which is the same for all three devices. The individuals MFG_1, MFG_2, and MFG_3 represent the device\u2019s manufacturer, and the individuals DiscreteDataPoint and HyperspectralImage describe the generated data. The last two individuals SkinLikeLiquid and MouseBrain describe the target of the offered measurements, whereby they are only exemplary.\nWith the prepared ontology, the three devices are started, which triggers the process described in the implementation. After their boot, the devices register themselves in Ditto and are provided with a DT. Afterward, the DTs notify the DTOS of their registration, and the semantic representations are created automatically, which can now be queried. Figure 7 shows the GUI. It is divided into the three areas: Demand, Query, and Result. In the Demand area, various filters can be used to specify the demands on a measurement or a measuring device. The Query area displays the SPARQL query generated from the set filters by the microservice, which is sent to the knowledge graph. The Result area lists the device recommendations based on the submitted query. The fewer filters are set, the more comprehensive the results are. Figure 7 therefore features all devices presented in the research landscape, as only the service is defined as an OpticalMeasurementService.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 7. Node-RED dashboard, showing query functionality for specification-driven device recommendation.\n\n\n\nSubsequently, the Node-RED dashboard is used for a device recommendation based on the present use case. This use case requires an optical measurement service for a skin-like liquid to provide discrete data points as measured values. The liquid needs to be measured with a wavelength range of 300 to 450 nanometers. Figure 8 illustrates the result of the query with the specified filters. In the Result section, the recommended devices for the defined requirements are shown. For this use case, the photometer\u2014which is able to cover the required wavelength range with its four sensors, provides an optical measurement service, and delivers discrete data points\u2014is recommended.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 8. Node-RED dashboard, showing the detailed results of the query functionality for specification-driven device recommendation.\n\n\n\nInterfacing the digital twin of a photometrical measurement device \nIn the second step, the continuous integration of the photometer and its DT is proven. In this case, Node-RED, which is located in the Smart Application Space, was utilized again to provide a GUI to the DT. This ensures an interface to the DT via the DTOS and Eclipse Ditto for parameterizing, operating, and monitoring the physical measurement experiment. The practical procedures of the demonstration experiment are described below.\nA dilution series was performed on a skin-like liquid suspension with variable concentration of New Coccine (E124 Sigma Aldrich, St. Louis, MO, United States) (Figure 9). On the detector side, two EPD-660-1-0.9[59] from Roithner Lasertechnik were used. An LED, type ELD-650-523[60] from Roithner Lasertechnik, was utilized as the light source. The connection between the photometer and the probe is realized by optical fibers. Adc1 measures the reflected light signal caused by different concentrations of New Coccine. Adc2 measures the relevant ambient interfering light. The obtained digital values can be converted into a corresponding current value based on the set parameters. The resulting measurement signal, cleaned of interfering signals, can then be determined by subtracting these two values.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 9. Node-RED dashboard, showing the visualization results and GUI of the photometer digital twin recording an actual measurement of a skin-like liquid suspension (adc#1) and reference ambient interference measurement (adc#2).\n\r\n \n\n\n\n\nFor a brief qualitative evaluation, the comparison with the previously applied procedure for data processing and storage can be referred to. Since the photometer did not have a network interface, data could only be read out via the serial interface of the controller. Other researchers[41][42][43] have employed manual methods to read out the measurement series using tools like Matlab and Labview or directly writing down the data arriving via the serial interface within a text file. Afterward, they used tools like Excel or Matlab to evaluate the measurement results manually. As a consequence, the typical problems of data management arise again, i.e., many locally, decentrally distributed research data, missing access authorizations, and missing experimental references. The new concept of holistic engagement within the RDM infrastructure overcomes these problems and provides an integration platform. All data are labeled and referenced in the database with relevant experimental information and thus made accessible for future investigations. The parameterization of the photometer no longer has to be done by reprogramming the controller, but can now be conveniently adjusted via its GUI. This approach prevents misoperation and saves expert knowledge, workload, and, consequently, time. Henceforth, measurement apparatuses can be developed independently of use cases and used for experiments without any subsequent effort.\n\nDiscussion \nThis work elaborates an architecture for reliable as well as sustainable RDM tailored for the increasing amount of gathered data at a large interdisciplinary research institute. Facing the problems of multiple measuring sources, experimental devices, and the associated mass data that must be processed, the need for modern RDM arises. Guiding away from the conservative treatment of research data in a decentralized and passive archival manner, accompanied by issues in data loss, accessibility, interpretability, etc., a holonic infrastructure for managing devices, experiments, and their resulting data, entirely new opportunities arise to exploit the extensive potential of digitized RDM in research institutions. Addressing the obstacles of transposing RDM in a complete digitized form, the envisioned concept is leveraged by the DT paradigm. DTs act use-case independently as a disruptive enabler technology in digital transformation. Therefore, the state of the art was examined and several requirements were identified in terms of RDM (R-RDM1\u2013R-RDM4) and DTs (R-DT1\u2013R-DT6). Aligning the architecture to a sustainable modern RDM practice, the broadly accepted FAIR principles were utilized. Additionally, the derived requirements for DTs naturally fit well with the aforementioned FAIR principles and jointly build the foundation on the herewithin outlined RDM infrastructure. Subsequently, all the requirements are set into context, followed by a brief overview of how they have been satisfied in the previous sections. In particular:\n\nR-DT1: Replication, representation, and interoperability meets perfectly with R-RDM1: Findable, R-RDM3: Interoperable, and R-RDM4: Reusable. Every PT is precisely described as DT within the Digital Twin Space facilitated by Eclipse Ditto and its JSON-based twin representation examined by the exemplary Photometer implementation. Herein embedded are all necessary structural and semantic information, which are further consumed by the DTOS, which instantiates this information into Apache Jena Fuseki\u2019s knowledge graph and thus establishes the pillar of later interaction and querying of all metadata within the RDM Core Space. To do so, the twins and their knowledge representation are unambiguously connected with each other.\nR-DT2: Interconnectivity and data acquisition fits well with R-RDM2: Accessible. The DTs settled within Eclipse Ditto are connected bi-directionally via various standard IoT interfaces (e.g., MQTT, HTTP, etc.) to their physical pendants. The different types of the DT\u2019s data, including its metadata, are all covered by the dynamically updated JSON representation.\nR-DT3: Data storage can be aligned with R-RDM1: Findable, R-RDM2: Accessible, R-RDM3: Interoperable, and R-RDM4: Reusable. All types of data are managed, homogeneously stored, and labeled by the DTOS within both applied storage approaches. While InfluxDB is serving a time-series technique, the Dataverse offers a repository-based approach. The labeling relates to the metadata managed by the DTOS and the instantiated individuals within Apache Jena Fuseki\u2019s knowledge graph. Data access can be achieved by calling the DTOS API or in a two-staged manner by querying the knowledge graph and afterward pulling the data from the resulting storage locations.\nR-DT4: Synchronization could be satisfied in the demo implementation by using MQTT realized by Eclipse Mosquitto for the connection between the twins. Every change of state actualizes the DT and superordinated components or vice versa the PT.\nR-DT5: Interface and interaction meets with R-RDM2: Accessible and R-RDM3: Interoperable. Eclipse Ditto, as well as the entire RDM Core Space components, offer open APIs to interact and request data. Even Node-RED, located in the Smart Application Space, embodies basic GUI and interaction schemes of the DTs as a demonstrative implementation.\nR-DT6: Optimization, analytics, simulation, and decision-making addresses several value-adding features on top of DTs in accordance with every FAIR principle. The Smart Application Space is intended to be the habitat of these value-adding features and applications, which is founded on the subordinated three spaces. So far, just Node-RED represents one demonstrative approach to highlight potential future functionalities. With the introduced RDM infrastructure in place, there are no restrictions and obstacles in the potential magnitude of later developable smart applications or tools.\nThe last requirement R-DT6: Security is suitable regarding R-RDM2: Accessible. Eclipse Ditto supports state-of-the-art security standards, including encryption, policy, and tenant-based DT management to gain proper access to required entities.\nThe contribution of this paper demonstrates that DTs are a perfectly suitable enabling technology for the central management of RDM entities and a reliable RDM itself. Further advantages can be drawn in the generation of knowledge and the reuse of data from other experiments or devices. Diversified datasets can be correlated with each other to gain entirely new insights. Especially for non-trivial human-readable data structures, i.e., multidimensional parameter arrays, this brings tremendous benefits. By processing research data in the way shown, inconsistencies, accessibility problems, data loss, etc., are no longer issues, also paving the way for more sustainability in research. Even the reusability of experimental knowledge can dissolve the need of reproducing difficult and energy-consuming experiment setups if still examined and well-labeled data are available. This also contributes to the minimization of the environmental footprint in research.\nA self-developed photometer from the authors\u2019 research institute is used as the first demo implementation of a PT and its DT. Subsequently, it is shown that the maximum depth of integration allows access to all the functionalities of the RDM infrastructure. Especially the reconfigurability of already manufactured physical devices through their DT offers great modification opportunities. i.e., measuring devices can be built use-case-agnostic and later parameterized by their DT as proven before.\nThe introduced knowledge graph is dedicated as the heart of the RDM infrastructure. Paving the way for intelligent interaction behavior between the DTs, it was initially proven that a measuring equipment recommendation can be established based on the DT\u2019s knowledge representation.\nHowever, the presented implementation covers mandatory sub-parts of the overall RDM infrastructure, and the individual parts will need to be investigated at a much more fine-grained level in future proceedings. The authors are aware of the fact that this kind of infrastructure is only reasonable in large research institutions, where large amounts of diverse data accumulate. The optimum potential of such twinning architectures comes with a critical count of DTs. Improved scenarios for collaboration and interaction can then be explored. Another challenge arises from more complex measuring devices like non-customizable mass spectrometers. The ability of all research devices to communicate via the network and automatically aggregate data for the RDM involves an increased security risk and vulnerability to functional errors. Thus, future work has to cope with the analysis and integration of extremely heterogeneous research equipment into the RDM infrastructure. Furthermore, the development of experiment-specific representative metrics to ease the correlation between various previously examined results must be tackled. Additionally, the knowledge graph as a foundation for intelligent behavior must be further developed to realize a higher degree of action within the Smart Application Space. Henceforth, with a higher level of automation of the RDM infrastructure, matters of security, confidentiality, and functional safety will be considered in the authors\u2019 future work.\nSummarizing with the overall rationales, the RDM infrastructure is intended to be introduced and used at the authors\u2019 research institute within the upcoming three years, empowered by a research project. All the researchers should be sensitized in terms of FAIR-compliant RDM. Thus, better research-domain-independent cooperation between people should take place tackling problems together. To get them all on board and involved, it is absolutely necessary to pay the highest attention to usability and user-friendliness to avoid acceptance problems later on. This can be achieved by involving researchers from every domain in the development process of the RDM infrastructure.\n\nConclusions and future work \nIn order to cope with the increasing amount of generated data at research institutions, this paper introduced an infrastructure for handling research data produced by manifold heterogeneous measurement devices and experimental setups. Facing rapidly growing requirements on RDM, the DT paradigm is utilized and highlighted as suitable enabler technology.\nAccording to the analysis of relevant literature, the combination of both approaches results in a research gap, which this paper attempts to address. Requirements on reliable RDM, especially the FAIR principles, preserve value to researchers through the entire data lifecycle. In symbiosis with DT requirements, these principles could be afterward conceptually derived. Underlining the subsequent implementation, some of the typical measuring devices and apparatuses of the authors\u2019 institute, the CeMOS, have been highlighted as well as the specific use case of a photometer which was implemented afterward. Built upon four hierarchical key pillars, the architecture splits from the bottom up in the Physical Space, the Digital Twin Space, the RDM Core Space, and the Smart Application Space. Through the example of the photometer, a complete integration scenario was shown, including every mandatory part of the RDM infrastructure. Further, a proof of concept showed the feasibility and advantages of the utilization of knowledge graphs as well as the beneficial functionalities of DTs. In the subsequent discussion, the individual requirements were put into context with each other along with the implemented architecture. The discussion revealed that DTs are the perfect companion for the realization of a reliable and sustainable RDM to gain added value.\nAs its main contributions, this paper (1) introduced a novel approach for handling large amounts of research data according to the FAIR principles managing them in a centralized, structured manner; (2) obtained the necessity of utilizing DTs to overcome obstacles of the digital transformation within research institutes gearing them for Research 4.0; (3) developed a top-level knowledge graph addressing upcoming issues of interoperability and meta representation of experimental data and associated devices, paving the way for correlation of complex experimental data; (4) elaborated an implementation variant for reactive DTs as the base for later proactive realizations going beyond DTs as pure, passive state representations; and (5) outlined a highly reconfigurable design approach shown by a photometer opening up entirely new possibilities with less development efforts in hardware and software engineering by utilizing the DT, not the physical device itself.\nHowever, several limitations force future research rationales. To fully exploit the potential of the architecture, much research data needs to be collected, which is only feasible for large research institutes. A lot of work regarding the integration of measuring devices and experimental setups (e.g., non-customizable mass spectrometers) needs to be done. Interaction schemes based on the knowledge graph must be elaborated to rise DTs to proactive behavior. Experiment-specific representative metrics must be envisioned to facilitate the correlation of the resulting data. In addition to that, smart applications have to be integrated into the Smart Application Space to gain the genuine added value of such infrastructures. This work covers mandatory sub-parts of the overall RDM infrastructure, but the single parts have to be examined in a much more fine-grained manner in the authors\u2019 future work. Likewise, issues of security, confidentiality, and functional security will be considered forthcoming. In order to substantiate the practicability, future publications with experimental use-case-specific data are planned.\n\n Abbreviations, acronyms, and initialisms \nADC: analog-to-digital converter\nAI: artificial intelligence\nAMQP: Advanced Message Queuing Protocol\nAPI: application programming interface\nASIC: application-specific integrated circuit\nCeMOS: Center for Mass Spectrometry and Optical Spectroscopy\nDMP: data management plan\nDOI: digital object identifier\nDT: digital twin\nDTOS: DT Orchestration Service\nFAIR: findable, accessible, interoperable, and reusable\nGPIO: general purpose input\/output\nGUI: graphical user interface\nHTTP: Hypertext Transfer Protocol\nI\u00b2C: inter-integrated circuit\nIC: integrated circuit\nIoT: internet of things\nIR: infrared\nIRQ: interrupt request\nJSON: JavaScript Object Notation\nLSB: Least Significant Bit\nMIR: middle infrared\nMQTT: Message Queueing Telemetry Transport\nNIR: near infrared\nNASA: National Aeronautics and Space Administration\nPT: physical twin\nREST: Representational State Transfer\nRDM: research data management\nRDMO: Research Data Management Organizer\nUV: ultraviolet\nVIS: visible\nAppendix A \nhe JSON-formatted information model of Eclipse Ditto is shown in Figure A1. For reasons of clarity and space, LED3\u2013LED6, as well as adc2\u2013adc4, have been substituted. Their structure is analogous to the ones shown. Based on the physical structure of a PT, it is one of the main components of the DT representation. Thus, it serves not only as a data basis for all applications infrastructurally settled above it but also as a semantically enriched information model, which the DTOS uses to describe the device within the knowledge graph.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure A1. Eclipse Ditto information model.\n\r\n \n\n\n\n\nAcknowledgements \nParts of this work presented in this paper were supported by a grant from the German Ministry of Education and Research (BMBF), grant number 16FDFH125.\n\nAuthor contributions \nConceptualization, J.L., S.S., A.R. and T.H.; methodology, J.L., S.S., A.R. and T.H.; software, J.L., S.S. and T.H.; validation, J.L., S.S., A.R. and T.H.; investigation, J.L., A.R. and T.H.; writing\u2014original draft preparation, J.L., S.S., A.R. and T.H.; supervision, M.R. and J.R.; project administration, J.L. All authors have read and agreed to the published version of the manuscript.\n\nConflicts of interest \nThe authors declare no conflict of interest.\n\nReferences \n\n\n\u2191 1.0 1.1 1.2 1.3 1.4 Raptis, Theofanis P.; Passarella, Andrea; Conti, Marco (2019). \"Data Management in Industry 4.0: State of the Art and Open Challenges\". IEEE Access 7: 97052\u201397093. doi:10.1109\/ACCESS.2019.2929296. ISSN 2169-3536. https:\/\/ieeexplore.ieee.org\/document\/8764545\/ .   \n \n\n\u2191 Jones, E.; Kalantery, N.; Glover, B. (1 October 2019). \"Research 4.0: Interim Report\". Analysis & Policy Observatory. https:\/\/apo.org.au\/node\/262636 . Retrieved 06 December 2022 .   \n \n\n\u2191 3.0 3.1 3.2 3.3 Wilkinson, Mark D.; Dumontier, Michel; Aalbersberg, IJsbrand Jan; Appleton, Gabrielle; Axton, Myles; Baak, Arie; Blomberg, Niklas; Boiten, Jan-Willem et al. (15 March 2016). \"The FAIR Guiding Principles for scientific data management and stewardship\" (in en). Scientific Data 3 (1): 160018. doi:10.1038\/sdata.2016.18. ISSN 2052-4463. PMC PMC4792175. PMID 26978244. https:\/\/www.nature.com\/articles\/sdata201618 .   \n \n\n\u2191 4.0 4.1 Mons, Barend (9 March 2018) (in en). Data Stewardship for Open Science: Implementing FAIR Principles (1 ed.). Chapman and Hall\/CRC. doi:10.1201\/9781315380711. ISBN 978-1-315-38071-1. https:\/\/www.taylorfrancis.com\/books\/9781498753180 .   \n \n\n\u2191 5.0 5.1 5.2 Diepenbroek, M.; Gl\u00f6ckner, F.O.; Grobe, P. et al. (2014). \"Towards an integrated biodiversity and ecological research data management and archiving platform: the German federation for the curation of biological data (GFBio)\". Informatik 2014. Gesellschaft f\u00fcr Informatik e.V. pp. 1711\u201321. ISBN 978-3-88579-626-8. https:\/\/dl.gi.de\/items\/618c1a92-fff2-423e-8c60-a0d6a65fe04f .   \n \n\n\u2191 6.0 6.1 Grieves, Michael (2016). Origins of the Digital Twin Concept. doi:10.13140\/RG.2.2.26367.61609. http:\/\/rgdoi.net\/10.13140\/RG.2.2.26367.61609 .   \n \n\n\u2191 Mihai, Stefan; Yaqoob, Mahnoor; Hung, Dang V.; Davis, William; Towakel, Praveer; Raza, Mohsin; Karamanoglu, Mehmet; Barn, Balbir et al. (24\/2022). \"Digital Twins: A Survey on Enabling Technologies, Challenges, Trends and Future Prospects\". IEEE Communications Surveys & Tutorials 24 (4): 2255\u20132291. doi:10.1109\/COMST.2022.3208773. ISSN 1553-877X. https:\/\/ieeexplore.ieee.org\/document\/9899718\/ .   \n \n\n\u2191 8.0 8.1 Massonet, Angela; Kiesel, Raphael; Schmitt, Robert H. (7 April 2020). \"Der Digitale Zwilling \u00fcber den Produktlebenszyklus: Das Konzept des Digitalen Zwillings verstehen und gewinnbringend einsetzen\" (in en). Zeitschrift f\u00fcr wirtschaftlichen Fabrikbetrieb 115 (s1): 97\u2013100. doi:10.3139\/104.112324. ISSN 0947-0085. https:\/\/www.degruyter.com\/document\/doi\/10.3139\/104.112324\/html .   \n \n\n\u2191 Semeraro, Concetta; Lezoche, Mario; Panetto, Herv\u00e9; Dassisti, Michele (1 September 2021). \"Digital twin paradigm: A systematic literature review\" (in en). Computers in Industry 130: 103469. doi:10.1016\/j.compind.2021.103469. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0166361521000762 .   \n \n\n\u2191 Bao, Qiangwei; Zhao, Gang; Yu, Yong; Dai, Sheng; Wang, Wei (1 November 2021). \"The ontology-based modeling and evolution of digital twin for assembly workshop\" (in en). The International Journal of Advanced Manufacturing Technology 117 (1-2): 395\u2013411. doi:10.1007\/s00170-021-07773-1. ISSN 0268-3768. https:\/\/link.springer.com\/10.1007\/s00170-021-07773-1 .   \n \n\n\u2191 11.0 11.1 Sahlab, Nada; Kamm, Simon; Muller, Timo; Jazdi, Nasser; Weyrich, Michael (10 May 2021). \"Knowledge Graphs as Enhancers of Intelligent Digital Twins\". 2021 4th IEEE International Conference on Industrial Cyber-Physical Systems (ICPS) (Victoria, BC, Canada: IEEE): 19\u201324. doi:10.1109\/ICPS49255.2021.9468219. ISBN 978-1-7281-6207-2. https:\/\/ieeexplore.ieee.org\/document\/9468219\/ .   \n \n\n\u2191 Pang, Candy; Szafron, Duane (2014), Franch, Xavier; Ghose, Aditya K.; Lewis, Grace A. et al.., eds., \"Single Source of Truth (SSOT) for Service Oriented Architecture (SOA)\" (in en), Service-Oriented Computing (Berlin, Heidelberg: Springer Berlin Heidelberg) 8831: 575\u2013589, doi:10.1007\/978-3-662-45391-9_50, ISBN 978-3-662-45390-2, http:\/\/link.springer.com\/10.1007\/978-3-662-45391-9_50 . Retrieved 2023-09-12   \n \n\n\u2191 Miksa, Tomasz; Cardoso, Joao; Borbinha, Jose (1 December 2018). \"Framing the scope of the common data model for machine-actionable Data Management Plans\". 2018 IEEE International Conference on Big Data (Big Data) (Seattle, WA, USA: IEEE): 2733\u20132742. doi:10.1109\/BigData.2018.8622618. ISBN 978-1-5386-5035-6. https:\/\/ieeexplore.ieee.org\/document\/8622618\/ .   \n \n\n\u2191 14.0 14.1 Gray, Jim; Liu, David T.; Nieto-Santisteban, Maria; Szalay, Alex; DeWitt, David J.; Heber, Gerd (1 December 2005). \"Scientific data management in the coming decade\" (in en). ACM SIGMOD Record 34 (4): 34\u201341. doi:10.1145\/1107499.1107503. ISSN 0163-5808. https:\/\/dl.acm.org\/doi\/10.1145\/1107499.1107503 .   \n \n\n\u2191 Schadt, Eric E.; Linderman, Michael D.; Sorenson, Jon; Lee, Lawrence; Nolan, Garry P. (1 September 2010). \"Computational solutions to large-scale data management and analysis\" (in en). Nature Reviews Genetics 11 (9): 647\u2013657. doi:10.1038\/nrg2857. ISSN 1471-0056. PMC PMC3124937. PMID 20717155. https:\/\/www.nature.com\/articles\/nrg2857 .   \n \n\n\u2191 16.0 16.1 16.2 16.3 Andreas, F\u00fcrholz; Martin, Jaekel (2021) (in en). Data life cycle management pilot projects and implications for research data management at universities of applied sciences. doi:10.21256\/ZHAW-23073. https:\/\/digitalcollection.zhaw.ch\/handle\/11475\/23073 .   \n \n\n\u2191 17.0 17.1 Redkina, N. S. (1 April 2019). \"Current Trends in Research Data Management\" (in en). Scientific and Technical Information Processing 46 (2): 53\u201358. doi:10.3103\/S0147688219020035. ISSN 0147-6882. http:\/\/link.springer.com\/10.3103\/S0147688219020035 .   \n \n\n\u2191 Engelhardt, C.; Enke, H.; Klar, J. et al. (2017). \"Research Data Management Organiser\" (PDF). Proceedings of the 14th International Conference on Digital Preservation: 25\u201329. http:\/\/www-archive.cseas.kyoto-u.ac.jp\/ipres2017.jp\/wp-content\/uploads\/27Claudia-Engelhardt.pdf .   \n \n\n\u2191 Rosen, Roland; von Wichert, Georg; Lo, George; Bettenhausen, Kurt D. (2015). \"About The Importance of Autonomy and Digital Twins for the Future of Manufacturing\" (in en). IFAC-PapersOnLine 48 (3): 567\u2013572. doi:10.1016\/j.ifacol.2015.06.141. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2405896315003808 .   \n \n\n\u2191 Wang, Zongyan (18 March 2020), B\u00e1nyai, Tam\u00e1s; Petrilloand Fabio De Felice, Antonella, eds., \"Digital Twin Technology\" (in en), Industry 4.0 - Impact on Intelligent Logistics and Manufacturing (IntechOpen), doi:10.5772\/intechopen.80974, ISBN 978-953-51-6996-3, https:\/\/www.intechopen.com\/books\/industry-4-0-impact-on-intelligent-logistics-and-manufacturing\/digital-twin-technology . Retrieved 2023-09-12   \n \n\n\u2191 Barricelli, Barbara Rita; Casiraghi, Elena; Fogli, Daniela (2019). \"A Survey on Digital Twin: Definitions, Characteristics, Applications, and Design Implications\". IEEE Access 7: 167653\u2013167671. doi:10.1109\/ACCESS.2019.2953499. ISSN 2169-3536. https:\/\/ieeexplore.ieee.org\/document\/8901113\/ .   \n \n\n\u2191 Shafto, M.; Conroy, M.; Doyle, R. et al. (April 2012). \"Modeling, Simulation, Information Technology & Processing Roadmap\" (PDF). National Aeronautics and Space Administration. https:\/\/www.nasa.gov\/sites\/default\/files\/501321main_TA11-ID_rev4_NRC-wTASR.pdf .   \n \n\n\u2191 23.0 23.1 23.2 23.3 Grieves, Michael (25 May 2022). \"Intelligent digital twins and the development and management of complex systems\" (in en). Digital Twin 2: 8. doi:10.12688\/digitaltwin.17574.1. ISSN 2752-5783. https:\/\/digitaltwin1.org\/articles\/2-8\/v1 .   \n \n\n\u2191 Dorofeev, Kirill; Zoitl, Alois (1 July 2018). \"Skill-based Engineering Approach using OPC UA Programs\". 2018 IEEE 16th International Conference on Industrial Informatics (INDIN) (Porto: IEEE): 1098\u20131103. doi:10.1109\/INDIN.2018.8471978. ISBN 978-1-5386-4829-2. https:\/\/ieeexplore.ieee.org\/document\/8471978\/ .   \n \n\n\u2191 Sepasgozar, Samad M. E. (2 April 2021). \"Differentiating Digital Twin from Digital Shadow: Elucidating a Paradigm Shift to Expedite a Smart, Sustainable Built Environment\" (in en). Buildings 11 (4): 151. doi:10.3390\/buildings11040151. ISSN 2075-5309. https:\/\/www.mdpi.com\/2075-5309\/11\/4\/151 .   \n \n\n\u2191 van der Valk, Hendrik; Ha\u00dfe, Hendrik; M\u00f6ller, Frederik; Otto, Boris (1 June 2022). \"Archetypes of Digital Twins\" (in en). Business & Information Systems Engineering 64 (3): 375\u2013391. doi:10.1007\/s12599-021-00727-7. ISSN 2363-7005. https:\/\/link.springer.com\/10.1007\/s12599-021-00727-7 .   \n \n\n\u2191 Singh, Maulshree; Fuenmayor, Evert; Hinchy, Eoin; Qiao, Yuansong; Murray, Niall; Devine, Declan (24 May 2021). \"Digital Twin: Origin to Future\" (in en). Applied System Innovation 4 (2): 36. doi:10.3390\/asi4020036. ISSN 2571-5577. https:\/\/www.mdpi.com\/2571-5577\/4\/2\/36 .   \n \n\n\u2191 Zehnder, Philipp; Riemer, Dominik (1 December 2018). \"Representing Industrial Data Streams in Digital Twins using Semantic Labeling\". 2018 IEEE International Conference on Big Data (Big Data) (Seattle, WA, USA: IEEE): 4223\u20134226. doi:10.1109\/BigData.2018.8622400. ISBN 978-1-5386-5035-6. https:\/\/ieeexplore.ieee.org\/document\/8622400\/ .   \n \n\n\u2191 29.0 29.1 Lehmann, Joel; Lober, Andreas; Rache, Alessa; Baumg\u00e4rtel, Hartwig; Reichwald, Julian (2022). \"Collaboration of Semantically Enriched Digital Twins based on a Marketplace Approach:\". Proceedings of the 19th International Conference on Wireless Networks and Mobile Systems (Lisbon, Portugal: SCITEPRESS - Science and Technology Publications): 35\u201345. doi:10.5220\/0011141200003286. ISBN 978-989-758-592-0. https:\/\/www.scitepress.org\/DigitalLibrary\/Link.aspx?doi=10.5220\/0011141200003286 .   \n \n\n\u2191 G\u00f6ppert, Amon; Grahn, Lea; Rachner, Jonas; Grunert, Dennis; Hort, Simon; Schmitt, Robert H. (1 June 2023). \"Pipeline for ontology-based modeling and automated deployment of digital twins for planning and control of manufacturing systems\" (in en). Journal of Intelligent Manufacturing 34 (5): 2133\u20132152. doi:10.1007\/s10845-021-01860-6. ISSN 0956-5515. https:\/\/link.springer.com\/10.1007\/s10845-021-01860-6 .   \n \n\n\u2191 Zhang, Chao; Zhou, Guanghui; He, Jun; Li, Zhi; Cheng, Wei (2019). \"A data- and knowledge-driven framework for digital twin manufacturing cell\" (in en). Procedia CIRP 83: 345\u2013350. doi:10.1016\/j.procir.2019.04.084. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2212827119306985 .   \n \n\n\u2191 Segovia, Mariana; Garcia-Alfaro, Joaquin (20 July 2022). \"Design, Modeling and Implementation of Digital Twins\" (in en). Sensors 22 (14): 5396. doi:10.3390\/s22145396. ISSN 1424-8220. PMC PMC9318241. PMID 35891076. https:\/\/www.mdpi.com\/1424-8220\/22\/14\/5396 .   \n \n\n\u2191 Lober, Andreas; Lehmann, Joel; Hausermann, Tim; Reichwald, Julian; Baumgartel, Hartwig (22 November 2022). \"Improving the Engineering Process of Control Systems Based on Digital Twin Specifications\". 2022 4th International Conference on Emerging Trends in Electrical, Electronic and Communications Engineering (ELECOM) (Mauritius: IEEE): 1\u20136. doi:10.1109\/ELECOM54934.2022.9965259. ISBN 978-1-6654-6697-4. https:\/\/ieeexplore.ieee.org\/document\/9965259\/ .   \n \n\n\u2191 Onaji, Igiri; Tiwari, Divya; Soulatiantork, Payam; Song, Boyang; Tiwari, Ashutosh (3 August 2022). \"Digital twin in manufacturing: conceptual framework and case studies\" (in en). International Journal of Computer Integrated Manufacturing 35 (8): 831\u2013858. doi:10.1080\/0951192X.2022.2027014. ISSN 0951-192X. https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/0951192X.2022.2027014 .   \n \n\n\u2191 Blinowski, Grzegorz; Ojdowska, Anna; Przybylek, Adam (2022). \"Monolithic vs. Microservice Architecture: A Performance and Scalability Evaluation\". IEEE Access 10: 20357\u201320374. doi:10.1109\/ACCESS.2022.3152803. ISSN 2169-3536. https:\/\/ieeexplore.ieee.org\/document\/9717259\/ .   \n \n\n\u2191 36.0 36.1 K\u00fcmmel, Tim; van Marwick, Bj\u00f6rn; Rittel, Miriam; Ramallo Guevara, Carina; W\u00fchler, Felix; Teumer, Tobias; W\u00e4ngler, Bj\u00f6rn; Hopf, Carsten et al. (28 May 2021). \"Rapid brain structure and tumour margin detection on whole frozen tissue sections by fast multiphotometric mid-infrared scanning\" (in en). Scientific Reports 11 (1): 11307. doi:10.1038\/s41598-021-90777-4. ISSN 2045-2322. PMC PMC8163866. PMID 34050224. https:\/\/www.nature.com\/articles\/s41598-021-90777-4 .   \n \n\n\u2191 37.0 37.1 Heintz, Annabell; Sold, Sebastian; W\u00fchler, Felix; Dyckow, Julia; Schirmer, Lucas; Beuermann, Thomas; R\u00e4dle, Matthias (23 May 2021). \"Design of a Multimodal Imaging System and Its First Application to Distinguish Grey and White Matter of Brain Tissue. A Proof-of-Concept-Study\" (in en). Applied Sciences 11 (11): 4777. doi:10.3390\/app11114777. ISSN 2076-3417. https:\/\/www.mdpi.com\/2076-3417\/11\/11\/4777 .   \n \n\n\u2191 \"i.MX RT1060 Crossover Processors for Consumer Products\" (PDF). NXP B.V. April 2019. https:\/\/www.pjrc.com\/teensy\/IMXRT1060CEC_rev0_1.pdf . Retrieved 08 December 2022 .   \n \n\n\u2191 \"AS89010 Current-to-Digital Converter\". ams-OSRAM AG. 2017. https:\/\/ams.com\/as89010 . Retrieved 06 December 2022 .   \n \n\n\u2191 \"DP83825I: Smallest form factor (3-mm by 3-mm), low-power 10\/100-Mbps Ethernet PHY transceiver with 50-MHz c\". Texas Instruments. August 2019. https:\/\/www.ti.com\/product\/DP83825I . Retrieved 30 November 2022 .   \n \n\n\u2191 41.0 41.1 Schmitt, Lukas; Meyer, Conrad; Schorz, Stefan; Manser, Steffen; Scholl, Stephan; R\u00e4dle, and Matthias (1 August 2022). \"Use of a Scattered Light Sensor for Monitoring the Dispersed Surface in Crystallization\" (in de). Chemie Ingenieur Technik 94 (8): 1177\u20131184. doi:10.1002\/cite.202200076. ISSN 0009-286X. https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/cite.202200076 .   \n \n\n\u2191 42.0 42.1 Teumer, Tobias; Medina, Isabel; Strischakov, Johann; Schorz, Stefan; Kumari, Pooja; Hohlen, Annika; Scholl, Stephan; Schwede, Christian et al. (2021) (in en). Development and application of optical sensors and measurement devices for the detection of deposits during reaction fouling 13th ECCE and 6th ECAB -post 31762. doi:10.13140\/RG.2.2.25284.55681. http:\/\/rgdoi.net\/10.13140\/RG.2.2.25284.55681 .   \n \n\n\u2191 43.0 43.1 Guffart, Julia; Bus, Yannick; Nachtmann, Marcel; Lettau, Markus; Schorz, Stefan; Nieder, Helmut; Repke, Jens\u2010Uwe; R\u00e4dle, Matthias (1 June 2020). \"Photometric Inline Monitoring of Pigment Concentration in Highly Filled Lacquers\" (in en). Chemie Ingenieur Technik 92 (6): 729\u2013735. doi:10.1002\/cite.201900186. ISSN 0009-286X. https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/cite.201900186 .   \n \n\n\u2191 \"Eclipse Ditto documentation overview\". Eclipse Ditto Documentation. Eclipse Foundation. https:\/\/eclipse.dev\/ditto\/intro-overview.html . Retrieved 09 November 2022 .   \n \n\n\u2191 Nguyen, Dat Tien; Do, Hao Duc (2021), Tran, Duc-Tan; Jeon, Gwanggil; Nguyen, Thi Dieu Linh et al.., eds., \"Research on Large-Scale Knowledge Base Management Frameworks for Open-Domain Question Answering Systems\" (in en), Intelligent Systems and Networks (Singapore: Springer Singapore) 243: 87\u201392, doi:10.1007\/978-981-16-2094-2_11, ISBN 978-981-16-2093-5, https:\/\/link.springer.com\/10.1007\/978-981-16-2094-2_11 . Retrieved 2023-09-12   \n \n\n\u2191 Hong, Seokyong; Lee, Sangkeun; Lim, Seung-Hwan; Sukumar, Sreenivas R.; Vatsavai, Ranga Raju (31 May 2016). \"Evaluation of Pattern Matching Workloads in Graph Analysis Systems\" (in en). Proceedings of the 25th ACM International Symposium on High-Performance Parallel and Distributed Computing (Kyoto Japan: ACM): 263\u2013266. doi:10.1145\/2907294.2907305. ISBN 978-1-4503-4314-5. https:\/\/dl.acm.org\/doi\/10.1145\/2907294.2907305 .   \n \n\n\u2191 Lissandrini, Matteo; Brugnara, Martin; Velegrakis, Yannis (1 December 2018). \"Beyond macrobenchmarks: microbenchmark-based graph database evaluation\" (in en). Proceedings of the VLDB Endowment 12 (4): 390\u2013403. doi:10.14778\/3297753.3297759. ISSN 2150-8097. https:\/\/dl.acm.org\/doi\/10.14778\/3297753.3297759 .   \n \n\n\u2191 Vrgoc, Domagoj; Rojas, Carlos; Angles, Renzo; Arenas, Marcelo; Arroyuelo, Diego; Aranda, Carlos Buil; Hogan, Aidan; Navarro, Gonzalo et al. (2021). MillenniumDB: A Persistent, Open-Source, Graph Database. doi:10.48550\/ARXIV.2111.01540. https:\/\/arxiv.org\/abs\/2111.01540 .   \n \n\n\u2191 \"Apache Jena\". The Apache Software Foundation. https:\/\/jena.apache.org\/ . Retrieved 22 November 2022 .   \n \n\n\u2191 \"Get started with InfluxDB OSS 2.5\". InfluxData Documentation. InfluxData. 2023. https:\/\/docs.influxdata.com\/influxdb\/v2.5\/ . Retrieved 22 November 2023 .   \n \n\n\u2191 51.0 51.1 Hao, Yuanzhe; Qin, Xiongpai; Chen, Yueguo; Li, Yaru; Sun, Xiaoguang; Tao, Yu; Zhang, Xiao; Du, Xiaoyong (1 April 2021). \"TS-Benchmark: A Benchmark for Time Series Databases\". 2021 IEEE 37th International Conference on Data Engineering (ICDE) (Chania, Greece: IEEE): 588\u2013599. doi:10.1109\/ICDE51399.2021.00057. ISBN 978-1-7281-9184-3. https:\/\/ieeexplore.ieee.org\/document\/9458659\/ .   \n \n\n\u2191 Department of Computing and Informatics, Mazoon College, Muscat, Sultanate of Oman.; Nasar, Mohammad; Kausar, Mohammad Abu; Department of Information Systems, University of Nizwa, Nizwa, Sultanate of Oman. (30 August 2019). \"Suitability Of Influxdb Database For Iot Applications\". International Journal of Innovative Technology and Exploring Engineering 8 (10): 1850\u20131857. doi:10.35940\/ijitee.J9225.0881019. https:\/\/www.ijitee.org\/portfolio-item\/J92250881019\/ .   \n \n\n\u2191 \"Dataverse Documentation v. 6.0\". Dataverse Guides. Dataverse Project. 2022. https:\/\/guides.dataverse.org\/en\/latest\/ . Retrieved 22 November 2022 .   \n \n\n\u2191 Stall, Shelley; Martone, Maryann E.; Chandramouliswaran, Ishwar; Crosas, Merc\u00e8; Federer, Lisa; Gautier, Julian; Hahnel, Mark; Larkin, Jennie et al. (15 July 2020). Generalist Repository Comparison Chart. doi:10.5281\/ZENODO.3946720. https:\/\/zenodo.org\/record\/3946720 .   \n \n\n\u2191 Wittenberg, Marion; Tykhonov, Vyacheslav; Indarto, Eko; Steinhoff, Wilko; Veld, Laura Huis In 'T; Kasberger, Stefan; Conzett, Philipp; Concordia, Cesare et al. (31 March 2022). D5.5 'Archive in a Box' repository software and proof of concept of centralised installation in the cloud. doi:10.5281\/ZENODO.6676391. https:\/\/zenodo.org\/record\/6676391 .   \n \n\n\u2191 Light, Roger A (26 May 2017). \"Mosquitto: server and client implementation of the MQTT protocol\". The Journal of Open Source Software 2 (13): 265. doi:10.21105\/joss.00265. ISSN 2475-9066. http:\/\/joss.theoj.org\/papers\/10.21105\/joss.00265 .   \n \n\n\u2191 Uy, Nguyen Quoc; Nam, Vu Hoai (1 December 2019). \"A comparison of AMQP and MQTT protocols for Internet of Things\". 2019 6th NAFOSTED Conference on Information and Computer Science (NICS) (Hanoi, Vietnam: IEEE): 292\u2013297. doi:10.1109\/NICS48868.2019.9023812. ISBN 978-1-7281-5163-2. https:\/\/ieeexplore.ieee.org\/document\/9023812\/ .   \n \n\n\u2191 \"Node-RED Documentation\". Node-RED. OpenJS Foundation. https:\/\/nodered.org\/docs\/ . Retrieved 23 November 2022 .   \n \n\n\u2191 \"EPD-660-1-0.9 Datasheet (PDF) - EPIGAP optoelectronic GmbH\". Optoelektronik GmbH. 16 May 2022. https:\/\/pdf1.alldatasheet.com\/datasheet-pdf\/view\/332642\/EPIGAP\/EPD-660-1-0.9.html . Retrieved 08 December 2022 .   \n \n\n\u2191 \"ELD-650-523\" (PDF). Roithner LaserTechnik. http:\/\/www.roithner-laser.com\/datasheets\/led_div\/eld_650_523.pdf . Retrieved 08 December 2022 .   \n \n\n\nNotes \nThis presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added.\n\n\n\n\n\nSource: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.limswiki.org\/index.php\/Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\">https:\/\/www.limswiki.org\/index.php\/Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins<\/a>\nCategories: LIMSwiki journal articles (added in 2023)LIMSwiki journal articles (all)LIMSwiki journal articles on data management and sharingLIMSwiki journal articles on laboratory informaticsLIMSwiki journal articles on researchNavigation menuPage actionsJournalDiscussionView sourceHistoryPage actionsJournalDiscussionMoreToolsIn other languagesPersonal toolsLog inNavigationMain pageEncyclopedic articlesRecent changesRandom pageHelp about MediaWikiSearch\u00a0 ToolsWhat links hereRelated changesSpecial pagesPermanent linkPage informationPopular publications\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\nPrint\/exportCreate a bookDownload as PDFDownload as PDFDownload as Plain textPrintable version This page was last edited on 12 September 2023, at 23:50.Content is available under a Creative Commons Attribution-ShareAlike 4.0 International License unless otherwise noted.This page has been accessed 709 times.Privacy policyAbout LIMSWikiDisclaimers\n\n\n\n","9ace6d7c38d417b5bea5133d24ffe1a9_html":"<body class=\"mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-206 ns-subject page-Journal_Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins rootpage-Journal_Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins skin-monobook action-view skin--responsive\"><div id=\"rdp-ebb-globalWrapper\"><div id=\"rdp-ebb-column-content\"><div id=\"rdp-ebb-content\" class=\"mw-body\" role=\"main\"><a id=\"rdp-ebb-top\"><\/a>\n<h1 id=\"rdp-ebb-firstHeading\" class=\"firstHeading\" lang=\"en\">Journal:Establishing reliable research data management by integrating measurement devices utilizing intelligent digital twins<\/h1><div id=\"rdp-ebb-bodyContent\" class=\"mw-body-content\"><!-- start content --><div id=\"rdp-ebb-mw-content-text\" lang=\"en\" dir=\"ltr\" class=\"mw-content-ltr\"><div class=\"mw-parser-output\">\n\n\n<h2><span class=\"mw-headline\" id=\"Abstract\">Abstract<\/span><\/h2>\n<p>One of the main topics within <a href=\"https:\/\/www.limswiki.org\/index.php\/Research\" title=\"Research\" class=\"wiki-link\" data-key=\"409634fd90113f119362927fe222f549\">research<\/a> activities is the <a href=\"https:\/\/www.limswiki.org\/index.php\/Information_management\" title=\"Information management\" class=\"wiki-link\" data-key=\"f8672d270c0750a858ed940158ca0a73\">management of research data<\/a>. Large amounts of data acquired by heterogeneous scientific devices, sensor systems, measuring equipment, and experimental setups have to be processed and ideally managed by <a href=\"https:\/\/www.limswiki.org\/index.php\/Journal:The_FAIR_Guiding_Principles_for_scientific_data_management_and_stewardship\" title=\"Journal:The FAIR Guiding Principles for scientific data management and stewardship\" class=\"wiki-link\" data-key=\"e5903ddcc7734415af1d91fcd258da90\">FAIR<\/a> (findable, accessible, interoperable, and reusable) data management approaches in order to preserve their intrinsic value to researchers throughout the entire data lifecycle. The symbiosis of heterogeneous measuring devices, FAIR principles, and <a href=\"https:\/\/www.limswiki.org\/index.php\/Digital_twin\" title=\"Digital twin\" class=\"wiki-link\" data-key=\"6eb3827d476e956178a962cd0382ec3f\">digital twin<\/a> technologies is considered to be ideally suited to realize the foundation of reliable, sustainable, and open research data management. This paper contributes a novel architectural approach for gathering and managing research data aligned with the FAIR principles. A reference implementation as well as a subsequent proof of concept is given, leveraging the utilization of digital twins to overcome common data management issues at equipment-intense research institutes. To facilitate implementation, a top-level knowledge graph has been developed to convey <a href=\"https:\/\/www.limswiki.org\/index.php\/Metadata\" title=\"Metadata\" class=\"wiki-link\" data-key=\"f872d4d6272811392bafe802f3edf2d8\">metadata<\/a> from research devices along with the produced data. In addition, a reactive digital twin implementation of a specific measurement device was devised to facilitate reconfigurability and minimized design effort.\n<\/p><p><b>Keywords<\/b>: cyber\u2013physical system, sensor data, research data management, FAIR, digital twin, research 4.0, knowledge graph, ontology\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Introduction\">Introduction<\/span><\/h2>\n<p>Initiated through the ongoing efforts of digitization, one of the new fields of activity within <a href=\"https:\/\/www.limswiki.org\/index.php\/Research\" title=\"Research\" class=\"wiki-link\" data-key=\"409634fd90113f119362927fe222f549\">research<\/a> concerns the <a href=\"https:\/\/www.limswiki.org\/index.php\/Information_management\" title=\"Information management\" class=\"wiki-link\" data-key=\"f8672d270c0750a858ed940158ca0a73\">management of research data<\/a>. New technologies and the related increase in computing power can now generate large amounts of data, providing new paths to scientific knowledge.<sup id=\"rdp-ebb-cite_ref-:0_1-0\" class=\"reference\"><a href=\"#cite_note-:0-1\">[1]<\/a><\/sup> Research is increasingly adopting toolsets and techniques raised by Industry 4.0 while gearing itself up for Research 4.0.<sup id=\"rdp-ebb-cite_ref-2\" class=\"reference\"><a href=\"#cite_note-2\">[2]<\/a><\/sup> The requirement for reliable research data management (RDM) can be managed by <a href=\"https:\/\/www.limswiki.org\/index.php\/Journal:The_FAIR_Guiding_Principles_for_scientific_data_management_and_stewardship\" title=\"Journal:The FAIR Guiding Principles for scientific data management and stewardship\" class=\"wiki-link\" data-key=\"e5903ddcc7734415af1d91fcd258da90\">FAIR<\/a> data management principles, which indicate that data must be findable, accessible, interoperable, and reusable through the entire data lifecycle in order to provide value to researchers.<sup id=\"rdp-ebb-cite_ref-:1_3-0\" class=\"reference\"><a href=\"#cite_note-:1-3\">[3]<\/a><\/sup> In practice, however, implementation often fails due to the high heterogeneity of hardware and software, as well as outdated or decentralized data <a href=\"https:\/\/www.limswiki.org\/index.php\/Backup\" title=\"Backup\" class=\"wiki-link\" data-key=\"e12548e6bf5f28bfee99099fe8662dde\">backup<\/a> mechanisms.<sup id=\"rdp-ebb-cite_ref-:2_4-0\" class=\"reference\"><a href=\"#cite_note-:2-4\">[4]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:3_5-0\" class=\"reference\"><a href=\"#cite_note-:3-5\">[5]<\/a><\/sup> This experience can be confirmed by the work at the Center for Mass Spectrometry and Optical Spectroscopy (CeMOS), a research institute at the Mannheim University of Applied Sciences which employs approximately 80 interdisciplinary scientific staff. In the various fields within the institute\u2019s research landscape\u2014including medical technology, <a href=\"https:\/\/www.limswiki.org\/index.php\/Biotechnology\" title=\"Biotechnology\" class=\"wiki-link\" data-key=\"115005039d4cf0b4ef55ec14dc6d66da\">biotechnology<\/a>, <a href=\"https:\/\/www.limswiki.org\/index.php\/Artificial_intelligence\" title=\"Artificial intelligence\" class=\"wiki-link\" data-key=\"0c45a597361ca47e1cd8112af676276e\">artificial intelligence<\/a> (AI), and digital transformation, a wide variety of hardware and software is required to collect and process the data that are generated, which in initial efforts is posing a significant challenge for achieving holistic <a href=\"https:\/\/www.limswiki.org\/index.php\/Data_integration\" title=\"Data integration\" class=\"wiki-link\" data-key=\"fd01c635859e1d5b9583e43e31ef6718\">data integration<\/a>.\n<\/p><p>To cater to the respective disciplines, researchers of the institute develop experimental equipment such as middle infrared (MIR) scanners for the rapid detection and <a href=\"https:\/\/www.limswiki.org\/index.php\/Imaging\" class=\"mw-disambig wiki-link\" title=\"Imaging\" data-key=\"c99dd47b045eb67ecc822556afcbda57\">imaging<\/a> of biochemical substances in medical tissue sections, multimodal imaging systems generating hyperspectral images of tissue slices, or photometrical measurement devices for detection of particle concentration. Nevertheless, they also use non-customizable equipment such as <a href=\"https:\/\/www.limswiki.org\/index.php\/Mass_spectrometry\" title=\"Mass spectrometry\" class=\"wiki-link\" data-key=\"fb548eafe2596c35d7ea741849aa83d4\">mass spectrometers<\/a>, <a href=\"https:\/\/www.limswiki.org\/index.php\/Microscope\" title=\"Microscope\" class=\"wiki-link\" data-key=\"88edff09f2745648524350d3f7be8354\">microscopes<\/a>, and cell imagers for their experiments. These appliances provide great benefits for further development within the respective research disciplines, which is why the data are of immense value and must be brought together accordingly in a reliable RDM system.\n<\/p><p>Research practice shows that the step into the digital world seems to be associated with obstacles. As an innovative technology, the <a href=\"https:\/\/www.limswiki.org\/index.php\/Digital_twin\" title=\"Digital twin\" class=\"wiki-link\" data-key=\"6eb3827d476e956178a962cd0382ec3f\">digital twin<\/a> (DT) can be seen as a secure data source, as it mirrors a physical device (also called a physical twin or PT) into the digital world through a bilateral communication stream.<sup id=\"rdp-ebb-cite_ref-:8_6-0\" class=\"reference\"><a href=\"#cite_note-:8-6\">[6]<\/a><\/sup> DTs are key actors for the implementation of Industry 4.0 prospects.<sup id=\"rdp-ebb-cite_ref-7\" class=\"reference\"><a href=\"#cite_note-7\">[7]<\/a><\/sup> Consequently, additional reconfigurability of hardware and software of the digitally imaged devices becomes a reality. The data mapped by the DT thus enable the bridge to the digital world and hence to the digital use and management of the data.<sup id=\"rdp-ebb-cite_ref-:0_1-1\" class=\"reference\"><a href=\"#cite_note-:0-1\">[1]<\/a><\/sup> Depending on the domain and use case, industry and research are creating new types of standardization-independent DTs. In most cases, only a certain part of the twin\u2019s life cycle is reflected. Only when utilized over the entire life cycle of the physical entity does the DT becomes a powerful tool of digitization.<sup id=\"rdp-ebb-cite_ref-:9_8-0\" class=\"reference\"><a href=\"#cite_note-:9-8\">[8]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-9\" class=\"reference\"><a href=\"#cite_note-9\">[9]<\/a><\/sup> With the development of semantic modeling, hardware, and communication technology, there are more degrees of freedom to leverage the semantic representation of DTs, improving their usability.<sup id=\"rdp-ebb-cite_ref-10\" class=\"reference\"><a href=\"#cite_note-10\">[10]<\/a><\/sup> For the internal interconnection in particular, the referencing of knowledge correlations distinguishes intelligent DTs.<sup id=\"rdp-ebb-cite_ref-:10_11-0\" class=\"reference\"><a href=\"#cite_note-:10-11\">[11]<\/a><\/sup> The analysis of relevant literature reveals a research gap in the combination of both approaches (RDM and DTs), which the authors intend to address with this work.\n<\/p><p>In this paper, a centralized solution-based approach for data processing and storage is chosen, which is in contrast to the decentralized practice in RDM. Common problems of data management include having many locally, decentrally distributed research data; missing access authorizations; and missing experimental references, which is why the results become unusable over long periods of time. The resulting replication of data is followed by inconsistencies and interoperability issues.<sup id=\"rdp-ebb-cite_ref-12\" class=\"reference\"><a href=\"#cite_note-12\">[12]<\/a><\/sup> Furthermore, these circumstances were also determined by empirical surveys at the authors\u2019 institute. Therefore, a holistic infrastructure for data management is introduced, starting with the collection of the measurement series of the physical devices, up to the final reliable reusability of the data. Relevant requirements for a sustainable RDM leveraged by intelligent DTs are elaborated based on the related work. By enhancing with DT paradigms, the efficiency of a reliable RDM can be further extended. This forms the basis for an architectural concept for reliable data integration into the infrastructure with the DTs of the fully mapped physical devices. \n<\/p><p>Due to the broad spectrum and interdisciplinarity of the institution, myriad data of different origins, forms, and quantities are created. The generic concept of DT allows evaluation units to be created agnostically from their specific use cases. Not only do the physical measuring devices and apparatuses benefit in the form of flexible reconfiguration through the possibilities of providing their virtual representation with intelligent functions, but also directly through the great variety of harmonized data structures and interfaces made possible by DTs. The bidirectional communication stream between the twins enables the physical devices to be directly influenced. Accordingly, parameterization of the physical device takes place dynamically using the DT, instead of statically using firmware as is usually the case. In addition, due to the real-time data transmission and the seamless integration of the DT, an immediate and reliable response to outliers is possible. Both data management and DTs as disruptive technology are mutual enablers in terms of their realization.<sup id=\"rdp-ebb-cite_ref-:0_1-2\" class=\"reference\"><a href=\"#cite_note-:0-1\">[1]<\/a><\/sup> Therefore, the designed infrastructure is based on the interacting functionality of both technologies to leverage their synergies providing sustainable and reliable data management. In order to substantiate the feasibility and practicability, a demo implementation of a measuring device within the realized infrastructure is carried out using a photometrical measuring device developed at the institute. This also forms the basis for the proof of concept and the evaluation of the overall system.\n<\/p><p>As main contributions, the paper (1) presents a new type of approach for dealing with large amounts of research data according to FAIR principles; (2) identifies the need for the use of DTs to break down barriers for the digital transformation in research institutes in order to arm them for Research 4.0; (3) elaborates a high-level knowledge graph that addresses the pending issues of interoperability and meta-representation of experimental data and associated devices; (4) devises an implementation variant for reactive DTs as a basis for later proactive realizations going beyond DTs as pure, passive state representations; and (5) works out a design approach that is highly reconfigurable, using the example of a photometer, which opens up completely new possibilities with less development effort in hardware and software engineering by using the DT rather than the physical device itself.\n<\/p><p>This paper is organized as follows. The next section points out the state of the art and the related work in terms of RDM and DTs. Both subsections derive architectural requirements, which serve to evolve an architecture for sustainable and reliable RDM. Next, specifically picked use cases of the authors\u2019 institute are outlined, followed by their implementation and subsequent proof of concept and evaluation. Finally, after a discussion that relates the predefined requirements with each other and the implemented infrastructure, the work will be concluded and future challenges will be prospected.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Related_work\">Related work<\/span><\/h2>\n<p>In order to better situate the present work in the state of the art, the following subsections first show the foundations of RDM, then the developments in the field of DTs. For both focal points, requirements for the development of the later introduced architecture are elaborated, which provides a basis for discussion at the end.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Research_data_management\">Research data management<\/span><\/h3>\n<p>The motivating force for reliable RDM should not be the product per se, but rather the necessity to build a body of knowledge enabling the subsequent integration and reuse of data and knowledge by the research community through a reliable RDM process.<sup id=\"rdp-ebb-cite_ref-:1_3-1\" class=\"reference\"><a href=\"#cite_note-:1-3\">[3]<\/a><\/sup> Therefore, the primary objective of RDM is to capture data in order to pave the way for new scientific knowledge in the long term.\n<\/p><p>To bridge the gap from simple <a href=\"https:\/\/www.limswiki.org\/index.php\/Information\" title=\"Information\" class=\"wiki-link\" data-key=\"6300a14d9c2776dcca0999b5ed940e7d\">information<\/a> to actual knowledge generation in order to bring greater value to researchers, data are the fundamental resource that enables the integration of the physical world with the virtual world, and finally, the interaction with each other.<sup id=\"rdp-ebb-cite_ref-13\" class=\"reference\"><a href=\"#cite_note-13\">[13]<\/a><\/sup> The DT as an innovative concept of Industry 4.0 enables the convergence of the physical world with the virtual world through its definition-given bilateral data exchange. Data from physical reality are seamlessly transferred into virtual reality, allowing developed applications and services to influence the behavior and impact on the physical reality. Data are the underlying structure that enables the DT; as such, having good data management practices in place provides the realization of the concept.<sup id=\"rdp-ebb-cite_ref-:0_1-3\" class=\"reference\"><a href=\"#cite_note-:0-1\">[1]<\/a><\/sup>\n<\/p><p>Specifically, in the context of the ongoing advances in innovative technologies, data have evolved from being merely static in nature to being a continuous stream of information.<sup id=\"rdp-ebb-cite_ref-:4_14-0\" class=\"reference\"><a href=\"#cite_note-:4-14\">[14]<\/a><\/sup> In practice, the data generated in research activities are commonly stored in a decentralized manner on the computers of individual researchers or on local data mediums.<sup id=\"rdp-ebb-cite_ref-:3_5-1\" class=\"reference\"><a href=\"#cite_note-:3-5\">[5]<\/a><\/sup> A recent study showed that only 12 percent of research data is stored in reliable repositories accessible by others. The far greater part, the so-called \u201cshadow data,\u201d remains in the hands of the researchers, resulting in the loss of non-reproducible data sets, devoid of the possibility of extracting further knowledge from this data.<sup id=\"rdp-ebb-cite_ref-:2_4-1\" class=\"reference\"><a href=\"#cite_note-:2-4\">[4]<\/a><\/sup> In addition, the <a href=\"https:\/\/www.limswiki.org\/index.php\/Backup\" title=\"Backup\" class=\"wiki-link\" data-key=\"e12548e6bf5f28bfee99099fe8662dde\">backed-up<\/a> data may become inconsistent and lose significance without the entire measurement series being available. According to Schadt <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-15\" class=\"reference\"><a href=\"#cite_note-15\">[15]<\/a><\/sup>, the most efficient method currently available for transmitting large amounts of data to collaborative partners entails copying the data to a sufficiently large storage drive, which is then sent to the intended recipient. This observation can also be confirmed within CeMOS, where this practice of data transfer prevails. Not only is this method inefficient and a barrier to data sharing, but it can also become a security issue when dealing with sensitive data. With such an abundance of data flows, large amounts of data need to be processed and reliably stored, causing RDM to gain momentum within the researcher\u2019s community.<sup id=\"rdp-ebb-cite_ref-:4_14-1\" class=\"reference\"><a href=\"#cite_note-:4-14\">[14]<\/a><\/sup>\n<\/p><p>Based on the increasing awareness and the initiated ambition towards a reformation of publishing and communication systems in research, the international coalition of Wilkinson <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-:1_3-2\" class=\"reference\"><a href=\"#cite_note-:1-3\">[3]<\/a><\/sup> proposed the FAIR Data Principles in 2016. These principles are intended to serve as a guide for those seeking to improve the reusability of their data assets, according to which data are expected to be findable, accessible, interoperable, and reusable (FAIR) throughout the data lifecycle. The FAIR principles are briefly outlined below within the context of the technical requirements, as modeled by Wilkinson <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-:1_3-3\" class=\"reference\"><a href=\"#cite_note-:1-3\">[3]<\/a><\/sup>:\n<\/p>\n<ul><li><b>Findable<\/b>: Data are described with extensive <a href=\"https:\/\/www.limswiki.org\/index.php\/Metadata\" title=\"Metadata\" class=\"wiki-link\" data-key=\"f872d4d6272811392bafe802f3edf2d8\">metadata<\/a>, which are given a globally unique and persistent identifier and are stored in a searchable resource.<\/li>\n<li><b>Accessible<\/b>: Metadata are retrievable by their individual indicators through a standardized protocol, which is publicly free and universally implementable, as well as enabling an authentication procedure. The metadata must remain accessible even if the data are no longer available.<\/li>\n<li><b>Interoperable<\/b>: (Meta)-data utilize a formal, broadly applicable language and follow FAIR principles; moreover, references exist between (meta)-data.<\/li>\n<li><b>Reusable<\/b>: (Meta)-data are characterized by relevant attributes and released on the basis of clear data usage licenses. The origin of the (meta)-data is clearly referenced. In addition, (meta)-data comply with domain-relevant community standards.<\/li><\/ul>\n<p>While the FAIR principles define the core foundation for a reliable RDM, there is also a need to ensure that the necessary scientific infrastructure is in place to support RDM.<sup id=\"rdp-ebb-cite_ref-:5_16-0\" class=\"reference\"><a href=\"#cite_note-:5-16\">[16]<\/a><\/sup> In addition to the FAIR criteria, the concept of a data management plan (DMP) has a significant impact on the success of any RDM effort. The DMP is a comprehensive document that details the management of a research project\u2019s data throughout its entire lifecycle.<sup id=\"rdp-ebb-cite_ref-:6_17-0\" class=\"reference\"><a href=\"#cite_note-:6-17\">[17]<\/a><\/sup> A standard DMP in fact does not exist, as it must be individually tailored to the requirements of the respective research project. This requires an extensive understanding of the individual research project and an awareness of the complexity and project-specific research data. The actual implementation of a DMP often creates additional work for researchers, such as data preparation or documentation.<sup id=\"rdp-ebb-cite_ref-:5_16-1\" class=\"reference\"><a href=\"#cite_note-:5-16\">[16]<\/a><\/sup> With the aim of providing researchers with a useful instrument, a number of web-based collaborative tools for creating DMPs has since emerged, such as DMPTool, DMPonline, and Research Data Management Organizer (RDMO).<sup id=\"rdp-ebb-cite_ref-18\" class=\"reference\"><a href=\"#cite_note-18\">[18]<\/a><\/sup>\n<\/p><p>In addition to the benefits already mentioned, the use of a research data infrastructure facilitates the visibility of scientists\u2019 research as well as identifying new collaboration partners in industry, research, or funding bodies.<sup id=\"rdp-ebb-cite_ref-:3_5-2\" class=\"reference\"><a href=\"#cite_note-:3-5\">[5]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:5_16-2\" class=\"reference\"><a href=\"#cite_note-:5-16\">[16]<\/a><\/sup> In the meantime, funding bodies in particular have recognized the necessity of effective RDM, making it a prerequisite for the submission of research proposals.<sup id=\"rdp-ebb-cite_ref-:5_16-3\" class=\"reference\"><a href=\"#cite_note-:5-16\">[16]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:6_17-1\" class=\"reference\"><a href=\"#cite_note-:6-17\">[17]<\/a><\/sup>\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Digital_twins\">Digital twins<\/span><\/h3>\n<p>The first pioneering principles for twinning systems can be dated back to training and simulation facilities of the National Aeronautics and Space Administration (NASA). In 1970, these facilities gained particular prominence during the thirteenth mission of the Apollo lunar landing program. Using a full-scale simulation environment of the command and lunar landing capsule, NASA engineers on Earth mirrored the condition of the seriously damaged spacecraft and tested all necessary operations for a successful return of the astronauts. All the possibilities could thus be simulated and validated before executing the real protocol to avoid the potential fatal outcome of a mishandling.<sup id=\"rdp-ebb-cite_ref-19\" class=\"reference\"><a href=\"#cite_note-19\">[19]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-20\" class=\"reference\"><a href=\"#cite_note-20\">[20]<\/a><\/sup> The actual paradigm of a virtual representation of physical entities was initiated later in 2002. After the first introduction, Michael Grieves further developed his product life cycle (PLC) model, which was later given the term \"digital twin\" by NASA engineer John Vickers. The mirrored systems approach was popularized in 2010 when it was incorporated into NASA\u2019s technical road map.<sup id=\"rdp-ebb-cite_ref-21\" class=\"reference\"><a href=\"#cite_note-21\">[21]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-22\" class=\"reference\"><a href=\"#cite_note-22\">[22]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:7_23-0\" class=\"reference\"><a href=\"#cite_note-:7-23\">[23]<\/a><\/sup>\n<\/p><p>The fundamental concept can be divided into the duality of the physical and virtual world. According to Figure 1, the physical world or space contains tangible components, i.e., machines, apparatuses, production assets, measurement devices, or even physical processes, the so-called PTs. In this context, the illustration shows a stylized device of arbitrary complexity on the left-hand side. On the right side, its virtual counterpart is shown in the virtual world or space. The coexistence of both is ensured by the bilateral stream of data and information, which is introduced as a digital thread. All raw data accumulated from the physical world are sent by the PT to its DT, which aggregates them and provides accessibility. Vice versa, by processing these data, the DT provides the PT with refined analytical information. Each PT is allocated to precisely one DT. One of the goals is to transfer work activities from the physical world to the virtual world so that efficiency and resources are preserved.<sup id=\"rdp-ebb-cite_ref-:7_23-1\" class=\"reference\"><a href=\"#cite_note-:7-23\">[23]<\/a><\/sup> Systems with a multitude of devices especially require flexible approaches for orchestration. Processes and devices must be able to be varied, rescheduled, and reconfigured.<sup id=\"rdp-ebb-cite_ref-24\" class=\"reference\"><a href=\"#cite_note-24\">[24]<\/a><\/sup> Twin technologies as enablers for this, providing the greatest possible degree of freedom.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig1_Lehmann_Sensors23_23-1.png\" class=\"image wiki-link\" data-key=\"604f09095d226cadb90bc1d2204c99e0\"><img alt=\"Fig1 Lehmann Sensors23 23-1.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/d\/de\/Fig1_Lehmann_Sensors23_23-1.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 1.<\/b> Concept of digital twins according to Grieves.<sup id=\"rdp-ebb-cite_ref-:8_6-1\" class=\"reference\"><a href=\"#cite_note-:8-6\">[6]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:7_23-2\" class=\"reference\"><a href=\"#cite_note-:7-23\">[23]<\/a><\/sup><\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>In the evolution of DTs, gradations concerning integration depth can be identified. A distinction is made between \"digital model,\" \"digital shadow,\" and the \"digital twin\" itself. Sepasgozar<sup id=\"rdp-ebb-cite_ref-25\" class=\"reference\"><a href=\"#cite_note-25\">[25]<\/a><\/sup> investigates this coherence and elaborates that digital models are created before the actual physical life cycle of a DT, whereas digital shadows have a unidirectional mirroring of a physical entity. To meet the characteristics of a real twin, communication must be in a bivalent way. Van der Valk <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-26\" class=\"reference\"><a href=\"#cite_note-26\">[26]<\/a><\/sup> deduce DT archetypes from characteristics as well as industry interviews. Starting from basic digital twins which, similar to a digital shadow, just represent the state of a physical object, up to increasingly complex twin variations, the following archetypes are further differentiated: enriched digital twin, autonomous control twin, enhanced autonomous control twin, exhaustive twin. Starting with the autonomous control twin, the DT emerges from its passive role and receives autonomous, intelligent features, which reach their completion in the exhaustive twin. While the first three archetypes can already be found in industry, the more advanced approaches are rather domain-limited or limited to research activities. Grieves<sup id=\"rdp-ebb-cite_ref-:7_23-3\" class=\"reference\"><a href=\"#cite_note-:7-23\">[23]<\/a><\/sup> also criticizes this and argues that intelligent DTs must shift from their passive role and become active, online, goal-seeking, and anticipatory. DT technologies still need a long time to reveal their full potential. Just by identifying and focusing on the domain-specific challenges, this lack of utilizing the opportunities can be tackled.<sup id=\"rdp-ebb-cite_ref-:9_8-1\" class=\"reference\"><a href=\"#cite_note-:9-8\">[8]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-27\" class=\"reference\"><a href=\"#cite_note-27\">[27]<\/a><\/sup>\n<\/p><p>Addressing some of these problems, semantic web technologies are inevitably needed labeling the required data streams.<sup id=\"rdp-ebb-cite_ref-28\" class=\"reference\"><a href=\"#cite_note-28\">[28]<\/a><\/sup> Lehmann <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-:11_29-0\" class=\"reference\"><a href=\"#cite_note-:11-29\">[29]<\/a><\/sup> show that a knowledge-based approach for the representation of DTs is indispensable. Only then interaction between intelligent DTs can take place, and they are able to proactively negotiate with others so that, for example, optimal process flows emerge. Sahlab <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-:10_11-1\" class=\"reference\"><a href=\"#cite_note-:10-11\">[11]<\/a><\/sup> use knowledge graphs to refine intelligent DTs. Particularly in industrial applications, these approaches are distinguished by the management of dynamically emerging DTs. Only through reasoning over the knowledge graphs do opportunities for self-adaptation and self-adaptation emerge. G\u00f6ppert <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-30\" class=\"reference\"><a href=\"#cite_note-30\">[30]<\/a><\/sup> develop a reference architecture for the development of DTs based on an end-to-end workflow that addresses definition, modeling, and deployment for the description of a pipeline for ontology-based DT creation. Zhang <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-31\" class=\"reference\"><a href=\"#cite_note-31\">[31]<\/a><\/sup> combine DTs, dynamic knowledge bases, and knowledge-based intelligent skills to realize an autonomous framework for manufacturing cells. Due to the manufacturing context, other ontologies are relevant for the definition phase. Therefore, various other requirements and constraints are needed for different applications.\n<\/p><p>Finally, Segovia and Garcia-Alfaro<sup id=\"rdp-ebb-cite_ref-32\" class=\"reference\"><a href=\"#cite_note-32\">[32]<\/a><\/sup> investigate DTs in terms of design, modeling, and implementation and derive functional specifications. Lober <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-33\" class=\"reference\"><a href=\"#cite_note-33\">[33]<\/a><\/sup> also elaborate general specifications for DTs in their work on improving control systems based on them. Introducing a general framework and use case studies, Onaji <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-34\" class=\"reference\"><a href=\"#cite_note-34\">[34]<\/a><\/sup> show important characteristics that DTs must fulfil. \n<\/p><p>In accordance with the insights outlined above and the specifically developed guidelines, the following requirements have to be considered in the present work to realize proper virtual representations of physical entities:\n<\/p>\n<ol><li><b>Replication, representation, and interoperability<\/b>: The virtual counterpart of a physical entity should be as detailed as possible, but at the same time as less complex as required without violating the fidelity of the replicated device. A representation should not only include the data of a device but also describe the meaning of this data to lay the foundations for autonomous interoperability.<\/li>\n<li><b>Interconnectivity and data acquisition<\/b>: All physical devices must be connected bi-directionally via suitable communication standards. The incoming data must be processed in a time-appropriate manner and reflected in the twin. The data forms to be taken into account can be of a descriptive, static, or dynamic nature and must be considered accordingly during processing. Processed information from the DT must also be reflected back into the PT.<\/li>\n<li><b>Data storage<\/b>: All aggregated data must be stored agnostic of format immediately. For reusability, it is necessary to store the data with reference and labeling in suitable storage forms. Not only time but also version, as well as change management, are useful options regarding this.<\/li>\n<li><b>Synchronization<\/b>: Whenever possible, the bivalent data connection should be carried out in real-time and under adequate latency conditions. Both twins should replicate the condition of their counterparts if possible.<\/li>\n<li><b>Interface and interaction<\/b> In order to enable collaboration and interaction between and with the twins, suitable interfaces are required. On the one hand, it must be possible for data to be exchanged and accessed by machines, and on the other hand, data must be readable and interpretable by humans providing suitable interaction modes.<\/li>\n<li><b>Optimization, analytics, simulation, and decision-making<\/b>: To gain further advantages, additional features should be accessible through the DTs. Thus, real-time analyses and optimizations, as well as independent algorithms for data evaluation, can be applied to the data basis of the DT. It should be possible to use AI technologies, establish decision making, or use far-reaching simulations, for example. The DT is intended to create context awareness and to facilitate collaborative approaches to reliably choreograph the twins.<\/li>\n<li><b>Security<\/b>: Each entity must comply with current security standards, i.e., authorization, policies, and <a href=\"https:\/\/www.limswiki.org\/index.php\/Encryption\" title=\"Encryption\" class=\"wiki-link\" data-key=\"86a503652ed5cc9d8e2b0252a480b5e1\">encryption<\/a>. Both privacy and integrity must be preserved. Optionally, the DT could monitor the current security through \"what-if\" scenarios and initiate countermeasures.<\/li><\/ol>\n<p>After a detailed examination of both concepts (RDM and DTs), it is obvious that symbiosis of both can draw certain advantages. The requirements for reliable and sustainable RDM especially align well with the DT characteristics described above. Both data management and DTs\u2014as disruptive technology\u2014are mutual enablers in terms of their realization.<sup id=\"rdp-ebb-cite_ref-:0_1-4\" class=\"reference\"><a href=\"#cite_note-:0-1\">[1]<\/a><\/sup> The works analyzed in this section reveal a gap in research, which this paper attempts to address. The synthesized architecture built out of these pillars is presented subsequently.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Concept_architecture\">Concept architecture<\/span><\/h2>\n<p>After the relevant requirements for a sustainable RDM leveraged by intelligent DTs have been elaborated, the architectural concept will be introduced and aligned regarding these requirements. Subsequently, a decentralized RDM infrastructure is presented facing general data management problems within a research and development institute. Due to the wide range and interdisciplinarity of the institution, countless data of different origins, formats, and quantities are generated. In the authors\u2019 context, data from mass spectroscopy and spectrometry must be specifically assumed, especially in the field of process analytics and medical technology. However, general infrastructure approaches are to be built up agnostically so that the RDM can also be operated independently of use cases. By enriching it with DT paradigms, the efficiency of a reliable RDM can be further expanded. Thus, not only the physical measuring devices and appliances benefit in the form of flexible reconfiguration facilitated by the possibilities of incorporating their virtual representation with intelligent features, but also directly the wide variety of harmonized data structures and interfaces that are empowered by DTs. Hence, typical data management problems\u2014i.e., many locally, decentrally distributed research data, missing access permissions, and missing experimental references, which is why the results become unusable over long periods of time\u2014are addressed. The proposed approach depicted in Figure 2 tries to overcome these issues.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig2_Lehmann_Sensors23_23-1.png\" class=\"image wiki-link\" data-key=\"fff5dff5ff14e089da30a8b348994cc6\"><img alt=\"Fig2 Lehmann Sensors23 23-1.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/a\/ac\/Fig2_Lehmann_Sensors23_23-1.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 2.<\/b> Conceptual RDM architecture.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>The overall architecture is split into two main areas: the Physical Space in which the physical measuring devices and research equipment are settled, and the RDM infrastructure itself, which is subdivided into three functional layers: the Digital Twin Space, the RDM Core Space, and the Smart Application Space.\n<\/p><p>From the bottom up, Figure 2 shows there are the physical devices generically referred to as <i>PT 1\u2013PT n<\/i>. In accordance with Grieves\u2019 twinning paradigm, these are uniquely linked to their digital counterparts situated within the Digital Twin Space. The data-driven representations must be enriched with semantic information content. In this way, it is possible to derive a machine-readable information model. Due to the asynchronous nature of many measurement procedures, the bivalent data pipeline between the two twins is l-driven so that data synchronicity is preserved. The DTs in the Digital Twin Space consequently aggregate all data and static, structural information from the Physical Space and provide it in a harmonized form to the superimposed infrastructure layers through standard communication interfaces.\n<\/p><p>The middle layer of the infrastructure, the RDM Core Space, contains the main elements required for reliable and sustainable RDM. These include a knowledge graph, a storage environment, and a messaging broker. Special attention should be paid to the DT Orchestration Service (DTOS), which takes over the choreography of the DTs with all the aggregated data, information, and requests from all participants of the RDM infrastructure that arise.\n<\/p><p>Starting with the knowledge graph, it offers itself as an environment for storing all domain-specific knowledge through an ontology. It is intended to organize the entire semantic information of the DT information models. As a sub-discipline of AI, such a knowledge-based approach should bring with it possibilities for reasoning and inferring complex system interrelationships. In this way, DT should be harnessed with intelligence through the knowledge graph.\n<\/p><p>The second pillar of the RDM Core Space involves examining storage approaches for all accruing forms of data. Because of the different measurement methods and data sources, it also needs different concepts for storage to be considered. For example, some devices deliver a continuous data stream, others asynchronous data points or data sets, and still others preprocessed data, i.e., from imaging measurement procedures. This requires, on the one hand, the necessity of archiving time series data and, on the other hand, a conventional repository-based file system approach. To ensure reusability and interoperability, experiment-specific data must be labeled and versioned. This is also done on a semantic basis so that the experiment data can also be located within the knowledge graph in order to create intelligent links at later stages and to be able to put data sets into context to other ones.\n<\/p><p>A messaging broker is also envisaged as a central RDM Core Space element that can be accessed anywhere within the architecture. This constitutes an asynchronous, event-driven communications interface for live data of all intended layers and ensures that everyone has non-discriminatory access to all necessary data.\n<\/p><p>The last major component of the RDM Core Space, the DTOS, has extra intersections with the lower as well as the superordinate layers. The DTOS manages and orchestrates the entire RDM Core Space and thus simultaneously enables intelligent interplay of the DTs. A living part of the DTs is located in the DTOS and gives them functional freedom of action beyond the mostly passive Digital Twin Space. Combining the DTs with the stored knowledge within the knowledge graph results in powerful tools for superposed smart applications. Hence, the responsibility for reading out the DT information models and creating them within the knowledge graph also lies here. For both data and DTs, lifecycle management is established, so that sustainability and reliability in RDM are created. The DTOS should also provide access to the administration of the DTs as well as the versioning and management of the achieved data in storage. The DTOS can also be utilized by the top-level Smart Application Space through interfaces in order to trigger intelligent functionalities between the DT and the accumulated data.\n<\/p><p>The top-level Smart Application Space allows arbitrary services to consume data via the interfaces of the messaging broker or the DTOS and use it for their purposes. It would also be conceivable for smart applications to proactively offer their capabilities as a service to the DTs of the devices, or even as a service twin. Realizing this, they could also be represented in the knowledge graph and the Digital Twin Space and get into contact with other DTs.\n<\/p><p>Due to the nature of the research devices, a partly decentralized (data acquisition and preprocessing are commonly facilitated decentrally), mostly asynchronous event-based architecture is needed. Instead of choosing a monolithic software approach, which makes perfect sense on a central system, independent microservices are utilized here. A microservice-based architecture offers the greatest possible advantages in the context of RDM through separate areas of responsibility, independence, autarky, scalability, and fault tolerance through modularity.<sup id=\"rdp-ebb-cite_ref-35\" class=\"reference\"><a href=\"#cite_note-35\">[35]<\/a><\/sup> In order to make the later implementation approaches more comprehensible, a typical use case of the authors\u2019 research institute is presented subsequently.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Research_landscape_and_use_case_description_of_a_photometrical_measurement_device\">Research landscape and use case description of a photometrical measurement device<\/span><\/h2>\n<p>After the basic concept architecture for RDM has been presented, the subsequent implementation of a PT will take place on the basis of a specific measuring device and its use cases in order to integrate it prototypically within the RDM architecture as a first application example. Therefore, the research and device landscape of the institute will be considered first. The CeMOS conducts interdisciplinary research in the fields of medical biotechnology or medical technology and intelligent sensor technology in order to create synergies between mass spectrometry and optical device development. Based on a variety of covered research areas, several devices from different manufacturers, as well as self-built ones, are used to create a wide-ranging heterogeneous equipment landscape. It includes microscopes, cell imagers, and various mass spectrometers for generating hyperspectral images, as well as other hyperspectral imagers for specific use cases and optical measuring devices. Some of these imagers and measuring devices were developed, built, and are currently operating at the institute itself. \n<\/p><p>Representatives of these self-developed and manufactured measuring devices are the MIR scanner<sup id=\"rdp-ebb-cite_ref-:12_36-0\" class=\"reference\"><a href=\"#cite_note-:12-36\">[36]<\/a><\/sup>, the Multimodal Imaging System<sup id=\"rdp-ebb-cite_ref-:13_37-0\" class=\"reference\"><a href=\"#cite_note-:13-37\">[37]<\/a><\/sup>, and a multipurpose, multichannel photometer. The MIR scanner is used for generating hyperspectral images of tissue sections and consists of a laser unit with four lasers with different wavenumbers, a detector unit, a focusing unit, an agile mirror unit, and a movable object slide. It is used for frozen section analysis in tumor detection for the identification of tissue morphologies or tumor margins.<sup id=\"rdp-ebb-cite_ref-:12_36-1\" class=\"reference\"><a href=\"#cite_note-:12-36\">[36]<\/a><\/sup> The Multimodal Imaging System also generates hyperspectral data for tissue sections with various procedures and consists of a modular upright light microscope combined with a <a href=\"https:\/\/www.limswiki.org\/index.php\/Raman_spectroscopy\" title=\"Raman spectroscopy\" class=\"wiki-link\" data-key=\"60e69470fcd47644f07a6969414597ea\">Raman spectrometer<\/a>, a visible (VIS) \/ <a href=\"https:\/\/www.limswiki.org\/index.php\/Near-infrared_spectroscopy\" title=\"Near-infrared spectroscopy\" class=\"wiki-link\" data-key=\"612e60c5d0558cc6205674759f59d74a\">near-Infrared<\/a> (NIR) reflectance spectrometer, and a detector unit. Its applications are in brightfield, darkfield, and polarization microscopy of normal mouse brain tissue, and an exemplary application provides the ability to make a distinction between white and grey matter.<sup id=\"rdp-ebb-cite_ref-:13_37-1\" class=\"reference\"><a href=\"#cite_note-:13-37\">[37]<\/a><\/sup> The majority of all devices currently do not use a network interface. The resulting measurement data are mostly stored locally and manually collected for analysis purposes. Due to the decentralized processing of the data, no problems arose with regard to security and confidentiality. Likewise, due to the low level of automation, no problems occurred with regard to emerging experimental errors or technical failures. Manual intervention could directly mitigate these errors. In the future, with an increased degree of automation of the RDM infrastructure, issues regarding security, confidentiality, and functional safety have to be taken into account. In addition to the previously mentioned devices, the photometer also serves as an essential component of the institute\u2019s research. For the subsequent implementation of the presented architecture, the feasibility is to be proven on the basis of the photometer.\n<\/p><p>The developed photometer system, schematically shown in Figure 3, essentially consists of three main components: the parts for digitizing the analog sensors, the driver for controlling the light sources, and a powerful microcontroller.<sup id=\"rdp-ebb-cite_ref-38\" class=\"reference\"><a href=\"#cite_note-38\">[38]<\/a><\/sup> These components and their interaction are described in the following. Depending on the application and measuring principle (transmission, reflection), photodiodes with different spectral sensitivities are used. Ideally, these sensors should have high photon sensitivity, fast response time, and low capacitance. Since these criteria are in mutual interaction, an application case-individual consideration is necessary. Exposure of the photodiode causes electrons to be released from the photocathode, resulting in a slight change in the diode\u2019s dark current. A current-to-voltage converter integrated circuit (IC)<sup id=\"rdp-ebb-cite_ref-39\" class=\"reference\"><a href=\"#cite_note-39\">[39]<\/a><\/sup> senses this photocurrent from the diode. The application-specific integrated circuit (ASIC) is a low-noise sensor interface and is suitable for coupling optical sensors with current output. These input currents are quantized into a digital output signal (with up to 16 bits, depending on the integration time). The integration time can be varied between 1 ms and 1024 ms, and the current sensitivity can be varied in steps from 20 fA\/Least Significant Bit (LSB) to 5000 pA\/LSB. Measurements can be continuous or manually triggered. An advantage of the integration of the input signals performed by the device is the resulting significant increase in the dynamic range. Furthermore, high-frequency components are filtered, and periodic disturbances with a multiple of the period duration are suppressed.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig3_Lehmann_Sensors23_23-1.png\" class=\"image wiki-link\" data-key=\"6dc0906ab9f6a414196d6dfe93058381\"><img alt=\"Fig3 Lehmann Sensors23 23-1.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/b\/bd\/Fig3_Lehmann_Sensors23_23-1.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 3.<\/b> Schematic layout of the physical photometer system.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>The microcontroller, connected via an inter-integrated circuit (I\u00b2C) interface, is configured in 400 kHz fast mode to communicate with the sensor interfaces. The parameters for writing and reading the analog-to-digital converters (ADCs) must follow a format specified by the manufacturer. The current state of each ADC (i.e., measurement running, measurement finished) can also be queried by reading special registers. Likewise, dedicated general purpose input\/output (GPIO) pins can be used to signal the status of the ADCs to the controller as an interrupt request (IRQ). This avoids permanent polling of the corresponding register or GPIO pin and saves resources. After a completed measurement, the controller reads the corresponding data packet via the internal interface. Subsequently, a new measurement can be initiated.\n<\/p><p>In addition to communication with the ADCs, the controller has the task of controlling the LEDs. These light sources are controlled via switchable constant current sources. The selection of the light-emitting diodes used is again very much dependent on the selected measuring principle and the detector. In addition to the wavelength and power of the light source, the rise times (\ud835\udc61<sub>\ud835\udc5f\ud835\udc56\ud835\udc60\ud835\udc52<\/sub>) and fall times (\ud835\udc61<sub>\ud835\udc53\ud835\udc4e\ud835\udc59\ud835\udc59<\/sub>) in particular must be taken into account in the LED selection. These times can be stored in the controller as parameters of each light source individually. Before starting and after finishing each measurement, these specific times are taken into account. Especially for complex measurement setups, low detection limits, and\/or short measurement duration, these parameters have a significant influence on the results and the maximum possible scanning speed. Via an integrated USB connection, all parameters can be configured between the photometer and the computer, and the raw measurement data can be sent. By means of the built-in ethernet PHY IC DP83825 from Texas Instruments<sup id=\"rdp-ebb-cite_ref-40\" class=\"reference\"><a href=\"#cite_note-40\">[40]<\/a><\/sup>, 10\/100 MBit communication via Ethernet is also possible. An <a href=\"https:\/\/www.limswiki.org\/index.php\/Internet_of_things\" title=\"Internet of things\" class=\"wiki-link\" data-key=\"13e0b826fa1770fe4bea72e3cb942f0f\">internet of things<\/a> (IoT) interface implemented on the software side, consisting of a Hypertext Transfer Protocol (HTTP) server and Message Queueing Telemetry Transport (MQTT) client, enables the connection to further IT infrastructure or web services.\n<\/p><p>The design of the photometer is highly flexible given its configurability and modularity. According to the selected configuration of the individual components, measurements can be performed in wavelength ranges of ultraviolet (UV), VIS, NIR, and infrared (IR). Furthermore, measurements of, e.g., particle sizes can be performed with special probe designs adapted to the task. It is even possible to conduct Raman measurements with probes that are extended by additional optical components. Some specific examples are listed below:\n<\/p>\n<ol><li><b>Use of a scattered light sensor for monitoring the dispersed surface in crystallization<\/b>: The specific surface area of the dispersed phase in suspensions, emulsions, bubble columns, and aerosols plays a decisive role in the increment of heat and mass transfer processes. This has a direct effect on the space-time yield in large-scale chemical\/process engineering production plants. An easy-to-install optical backscatter sensor outputs the dispersed surface area as a direct primary signal under certain boundary conditions. The sensor works even in highly concentrated suspensions and emulsions, where conventional nephelometry already fails. Several trends and limitations have been found so far for the sensor, which can be used in-line in batch and continuously operated crystallizers, even in harsh production environments, and in potentially explosive zones. The specific dispersed surface is directly detected as the primary measurand.<sup id=\"rdp-ebb-cite_ref-:14_41-0\" class=\"reference\"><a href=\"#cite_note-:14-41\">[41]<\/a><\/sup><\/li>\n<li><b>Development and application of optical sensors and measurement devices for the detection of deposits during reaction fouling<\/b>: In many chemical\/pharmaceutical processes, the technically viable efficiencies and throughputs have not been achieved yet because of the reduction in heat transfer (e.g., in heat transfer units, reactors, etc.) due to the formation of wall deposits. Considerable amounts of energy can be saved by reducing or entirely preventing this problematic area. Therefore, a measurement device and its optical and electronic parts were developed for the detection and measurement of deposits in polymerization reactors, simultaneously aiming the in-line monitoring. The design strategy was carried out systematically via theoretical calculations\u2014such as optical ray tracing and photon flux analysis\u2014via test designs, <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory\" title=\"Laboratory\" class=\"wiki-link\" data-key=\"c57fc5aac9e4abf31dccae81df664c33\">laboratory<\/a> investigations, and then industrial use. The developed sensors are based on fiber-optic technology and thus can be integrated into the smallest and most complex apparatus, even in explosion-hazardous areas. Critical product and process states in the reactant are detected at an early stage by combining several multi-spectral backscattering technologies. Thus, the formation of deposits can be prevented by changing process parameters.<sup id=\"rdp-ebb-cite_ref-:15_42-0\" class=\"reference\"><a href=\"#cite_note-:15-42\">[42]<\/a><\/sup><\/li>\n<li><b>Photometric inline monitoring of the pigment concentration of highly filled coatings<\/b>: This involves inline monitoring of particle concentration in highly filled dispersions and paint systems using fiber-optic backscatter sensors. Due to the miniaturization of the distance between emitter and receiver fiber to <600 \u00b5m, the transmitted light can also penetrate high dispersion phase fractions of up to 60%. Due to the measurement setup, both transmission and scattering influences are found in the resulting signal. In this setup, the photometer is configured with detectors and light sources for the red wavelength range (660 nm). The measurement interval of 128 ms is sufficiently small to allow very close monitoring of the measured values.<sup id=\"rdp-ebb-cite_ref-:16_43-0\" class=\"reference\"><a href=\"#cite_note-:16-43\">[43]<\/a><\/sup><\/li><\/ol>\n<p>As described above, a number of hardware and software settings and modifications have to be made, especially during the pre-test phase, in order to fulfil the intended task. This usually requires a modification of the firmware of the photometer with the corresponding parameters of the installed components. This time-consuming step, which cannot be performed by every end user, can be eliminated by utilizing the DT of the physical device. Hardware-specific settings can be made comfortably via a graphical user interface (GUI). At the same time, a plausibility check of the selected parameters can be realized in a simple way. A misconfiguration of the device can be made more difficult, and the end user has the option to check the settings again.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Implementation\">Implementation<\/span><\/h2>\n<p>Through the proposed architectural approach on the one hand and the described use case on the other hand, the prototypical implementation should be outlined on this foundation subsequently. Thus, the general structure and deployment of the main components and their interrelationship will be presented.\n<\/p><p>Figure 4 illustrates the further developed concept architecture. In each individual layer, the utilized microservice instances are depicted. Instead of just using templates of PTs and their DTs in theory, as shown in the concept, the photometer presented in the use case facilitates a complete integration scenario within the RDM infrastructure. It will proceed from the bottom up beginning with the development of the physical photometers representation to further derive its information model for the corresponding DT. Afterward, the entire integration of the RDM Core Space is executed by the DTOS. Underlining the implementation and the integration scenario of the photometer, a proof of concept will be carried out later.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig4_Lehmann_Sensors23_23-1.png\" class=\"image wiki-link\" data-key=\"820c4a9cd4b87fd71d7e732a9b0c12d6\"><img alt=\"Fig4 Lehmann Sensors23 23-1.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/5\/5e\/Fig4_Lehmann_Sensors23_23-1.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 4.<\/b> Implementation of the conceptual RDM architecture.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>Every implemented component is built up microservice-based. These microservices are deployed within a distributed server environment at the research institute. Depending on required performance and space, such a microservice infrastructure can be deployed as scalable via Docker, a Docker swarm, or even a Kubernetes cluster. Because of the high complexity of such an infrastructure which a reliable and sustainable RDM requires, this section will be further subdivided into several subsections. After prospecting the physical setup of the Physical Space, the Digital Twin Space will be illuminated. Followed by the RDM Core Space, in which the interrelations between main objective functionalities of RDM are laid down, and a brief overview regarding utilized, as well as potentially realizable, smart applications within the Smart Application Space are shown.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Physical_Space\">Physical Space<\/span><\/h3>\n<p>The lower layer of the architecture is the Physical Space, which contains all physical devices. In this case, the implementation of the Physical Space is exemplarily reduced to the photometer introduced in the research landscape. The other pre-presented devices are featured at the proof of concept level, demonstrating and validating the functionality of the RDM infrastructure. Later, this layer should be extended by the heterogeneous research equipment of the institute.\n<\/p><p>The photometer previously described above will be used to demonstrate the seamless integration of a research measuring device. For the connection, the IoT interface of the photometer PT is foreseen. The device logs into the Digital Twin Space on every boot sequence via its integrated Representational State Transfer (REST) interface and transmits its structural configuration to it. This auto-deployment ensures that the state between PT and DT is always up to date. All the configurations are embedded in the form of an information model in a JavaScript Object Notation (JSON) file which is directly readable for the overlaying architecture layers. The JSON-formatted information model is recognized in Appendix A and exactly mirrors the measuring capabilities and functionalities for the setup of experiments, which was outlined before. Important parameters for identification and policy are declared at the beginning of the document. Then the attribute part describes the semantic meta contexts and capabilities of the device. Finally, the setting parameters, actuators, and sensors are described as features. For reasons of clarity and space, <i>LED3\u2013LED6<\/i>, as well as <i>adc2\u2013adc4<\/i>, have been substituted. Their structure is analogous to the ones shown. Based on the physical structure of the PT, it is one of the main components of the later DT representation. Thus, it serves not only as a data basis for all applications infrastructurally settled above it but also as a semantically enriched information model, which the DTOS uses to describe the device within the knowledge graph.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Digital_Twin_Space\">Digital Twin Space<\/span><\/h3>\n<p>At the base of the RDM infrastructure itself, the Digital Twin Space contains the DTs of the physical measuring devices. It is based on the open-source project Eclipse Ditto<sup id=\"rdp-ebb-cite_ref-44\" class=\"reference\"><a href=\"#cite_note-44\">[44]<\/a><\/sup>, which aims to cope with representations of DTs. With a scalable basis, the Ditto project offers the possibility of integrating physical devices and their digital representations at a high abstraction level. Not only the organization but also the entire physical-virtual interaction is thus made possible for further back-end applications in a simplified manner. The PT and its DT can be accessed bi-directionally via the provided <a href=\"https:\/\/www.limswiki.org\/index.php\/Application_programming_interface\" title=\"Application programming interface\" class=\"wiki-link\" data-key=\"36fc319869eba4613cb0854b421b0934\">application programming interface<\/a> (API). As a result, the Physical Space can be influenced by changes within the Digital Twin Space. Eclipse Ditto is, as the rest of the authors\u2019 infrastructure, built on various microservices. Individual scalability, space-saving deployment, and separation of different task areas as a robust, distributed system, let the project become a universal <a href=\"https:\/\/www.limswiki.org\/index.php\/Middleware\" title=\"Middleware\" class=\"wiki-link\" data-key=\"82ee1d9577571b4f9e4d83d6d6124c81\">middleware<\/a> for the provision of DTs. The essential system components of Eclipse Ditto are briefly outlined below.\n<\/p>\n<ul><li><b>Connectivity service<\/b>: Ensuring frictionless communication between physical devices, their virtual counterparts, and data consuming back-end applications, the Connectivity service provides a direct interface for various protocols and communication standards such as HTTP, Websockets, MQTT, or Advanced Message Queuing Protocol (AMQP). A specially developed, unified JSON-based Ditto Protocol as the payload of messages of the listed communication standards opens up numerous interaction possibilities. For example, messages can be also mapped via scripts for preprocessing and post-processing, as well as structuring. Furthermore, by using the Ditto Protocol, the entire Ditto instance can be managed, thereby a complete interface is established to interact efficiently with the DTs and their physical counterparts.<\/li>\n<li><b>Things service<\/b>: The Things service contains the actual structure and telemetry representation of the PTs. This abstract representation consists of a simple JSON file. While the first part of the JSON includes the static describing attributes of a DT, such as a unique identifier, the assigned policy, or other semantically describing properties, the second part contains the dynamic features to which all telemetry data belong. These mirror the constantly changing status of the PTs.<\/li>\n<li><b>Policies service<\/b>: Individual permissions for access and management of the twins, preserving privacy and integrity, are managed by the policy microservice. In order to grant finely graded read and write permissions to certain subjects, Eclipse Ditto offers the Policies service concept that can be easily modified via specific Ditto Protocol communication patterns. In addition to extensible certificate-based security mechanisms which Eclipse Ditto naively offers, this setup forms the foundation for the fulfilment of modern security standards.<\/li><\/ul>\n<p>To substantiate the advantages which are brought by Eclipse Ditto, the photometer DT should be further instantiated at the Things service. Therefore the before introduced representation form is used and aligned to the requirements of the Ditto Protocol. As a result Appendix A with its photometer JSON representation can be reviewed. The part with the key attributes at the beginning of the file contains all static and semantic necessary information to draw later benefits from. The second part with the key features contains the dynamic telemetry data, which are transmitted while operating constantly via MQTT from the PT to the DT and are further consumed by back-end applications or storage purposes. The responsibility for proper connections in direction of the superordinated architectural layers is preserved by the Connectivity service. On top of this middleware-like DT abstraction layer, value-generating features, i.e., the subsequently introduced RDM infrastructure, can be constructed.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"RDM_Core_Space\">RDM Core Space<\/span><\/h3>\n<p>On top of the Digital Twin Space, the RDM Core Space layer is settled. Here, the orchestration of the infrastructure and DTs takes place, the generated data are managed, and the communication service is provided. The RDM Core Space includes the DTOS, Apache Jena Fuseki as the knowledge graph, a combination of InfluxDB and Dataverse for storage, and Eclipse Mosquitto as the message broker. These instances provide various functionalities that are necessary for the microservices of the RDM infrastructure. Starting on the left with Apache Jena Fuseki, the individual infrastructure components are explained in order to subsequently characterize the features of the DTOS and thus fully cover the RDM Core Space later on.\n<\/p><p>Apache Jena Fuseki is a web <a href=\"https:\/\/www.limswiki.org\/index.php\/Ontology_(information_science)\" title=\"Ontology (information science)\" class=\"wiki-link\" data-key=\"52d0664bde4b458e81fbc128b911a4a6\">ontology<\/a> server that stands out from other alternatives such as Neo4j due to its higher performance. Although Jena Fuseki offers less flexibility in the area of integration of multiple sources, performance is the key criterion for this infrastructure.<sup id=\"rdp-ebb-cite_ref-45\" class=\"reference\"><a href=\"#cite_note-45\">[45]<\/a><\/sup> In comparison to JanusGraph, another alternative, Jena Fuseki also predominates in terms of performance.<sup id=\"rdp-ebb-cite_ref-46\" class=\"reference\"><a href=\"#cite_note-46\">[46]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-47\" class=\"reference\"><a href=\"#cite_note-47\">[47]<\/a><\/sup> In addition, Jena Fuseki uses the common query language SPARQL, while Neo4j or JanusGraph use the less common languages Cypher and Gremlin. Based on these mentioned arguments, Jena Fuseki is implemented within the infrastructure; however, a more extensive analysis of the suitability of Apache Jena Fuseki will be conducted in the future.<sup id=\"rdp-ebb-cite_ref-48\" class=\"reference\"><a href=\"#cite_note-48\">[48]<\/a><\/sup> \n<\/p><p>Apache Jena Fuseki is implemented on a dedicated server in a Docker container and offers the ability to receive and answer SPARQL queries. It includes a REST API that is used to create the semantic representation of the DTs and to communicate with the microservices across the infrastructure. In addition, a web interface can be used to submit SPARQL queries directly through an input form.<sup id=\"rdp-ebb-cite_ref-49\" class=\"reference\"><a href=\"#cite_note-49\">[49]<\/a><\/sup> In order to process queries, it is necessary that Apache Jena Fuseki contains a domain-specific ontology in which the entire semantic information of the DTs information model can be captured. The domain-specific ontology was designed with Prot\u00e9g\u00e9 according to the requirements of RDM and the DT representation. Figure 5 depicts the top-level ontology for RDM, allowing for the development of further complex sublevel ontologies in the future due to its modular structure. It enables the DTs to be described semantically with minimal complexity, along with contextualizing the generated data of their PTs. The ontology\u2019s basic structure is inspired by Lehmann <i>et al.<\/i>, who presented an ontology for production resources and products.<sup id=\"rdp-ebb-cite_ref-:11_29-1\" class=\"reference\"><a href=\"#cite_note-:11-29\">[29]<\/a><\/sup>\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig5_Lehmann_Sensors23_23-1.png\" class=\"image wiki-link\" data-key=\"ef5dfdeff3b5ed2f90400fb849d3d71f\"><img alt=\"Fig5 Lehmann Sensors23 23-1.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/0\/0b\/Fig5_Lehmann_Sensors23_23-1.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 5.<\/b> Top-level knowledge graph for RDM.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>In the context of this work, the ontology was adapted and further fitted to the needs of measurement devices and RDM. At the top hierarchy, the ontology is divided into five logical sections, corresponding to the classes Resource (yellow), Service (green), Target (red), Data (blue), and Manufacturer (black). As such, due to the underlying use case, it has been developed starting from the Resource class. The relationship between the individual classes is as follows. Each Resource has a Manufacturer and provides a specific Service for a particular Target and generates specific Data from it. These relations can equally be expressed in an inverse manner on the basis of the generated reasoning and inferences, as Figure 5 shows. The ontology\u2019s four top-level classes of Resource, Service, Target, and Data are further divided into sub-classes as shown by the logical sections. The Resource (yellow) has the SubResource \"Measurement Resource,\" which contains the sub-class \"Sensor\" and enables the Measurement Resource to gather data. In order for the Measurement Resource to collect data, it must provide a specific Service. This Service (green) is provided in the context of the Services sub-class as the \"Measurement Service.\" For a more precise specification of the given device landscape, the top-level class Target (red) is divided into the three sub-classes within the ontology: Tissue Slice, Surface, and Suspension. The classification of the top-level Data (blue) is thereby based on the degree of structuration into structured, semi-structured, and unstructured data. In the development of this domain-specific ontology, great efforts were made to ensure the best possible foundation for representing the semantic characteristics of the DTs and their data. At the same time, due to its modular structure, it offers future connecting points to roll out the ontology to the entire context of the institute and its requirements. Furthermore, the demonstrated ontology, in association with the Apache Jena Fuseki web server, offers the possibility to provide knowledge-based recommendations via use case specifications, as demonstrated in the proof of concept.\n<\/p><p>The next essential part of the implementation is the storage, which is implemented as a combination of InfluxDB and Dataverse. The hereby united different storage concepts cover a maximum number of use cases and fulfil the RDM and DT requirements as effectively as possible. For storing discrete data points, InfluxDB, an open-source time series database, is used.<sup id=\"rdp-ebb-cite_ref-50\" class=\"reference\"><a href=\"#cite_note-50\">[50]<\/a><\/sup> The InfluxDB stands out from other popular time series databases such as Prometheus, Druid, or OpenTSDB due to its query response time.<sup id=\"rdp-ebb-cite_ref-:17_51-0\" class=\"reference\"><a href=\"#cite_note-:17-51\">[51]<\/a><\/sup> Compared to Prometheus, InfluxDB offers an SQL-like query language, the possibility to manage user rights, and in-memory capabilities.<sup id=\"rdp-ebb-cite_ref-52\" class=\"reference\"><a href=\"#cite_note-52\">[52]<\/a><\/sup> In addition, the InfluxDB features a better compression ratio than Druid and OpenTSDB. These advantages make InfluxDB suitable for storing time series data within the infrastructure.<sup id=\"rdp-ebb-cite_ref-:17_51-1\" class=\"reference\"><a href=\"#cite_note-:17-51\">[51]<\/a><\/sup> InfluxDB is able to sign incoming data with a timestamp and classify it into corresponding buckets, which can be assigned to sensors in even more detail with the help of further criteria from the DT. InfluxDB comes with a REST API which allows data to be queried or sent, as well as a comprehensive web interface, which enables buckets to be searched and data to be <a href=\"https:\/\/www.limswiki.org\/index.php\/Data_visualization\" title=\"Data visualization\" class=\"wiki-link\" data-key=\"4a3b86cba74bc7bb7471aa3fc2fcccc3\">visualized<\/a>, further making data more findable and accessible. Moreover, the web interface allows the creation of dashboards to enable live monitoring. \n<\/p><p>Besides the discrete data points in the sense of measurement series, other data, such as hyperspectral images, are generated at the institute. InfluxDB is not suitable for storing this type of data, which is why Dataverse is also implemented and serves other storage concepts. It is an open-source web application that allows publishing, storing, citing, and providing research data in associated repositories.<sup id=\"rdp-ebb-cite_ref-53\" class=\"reference\"><a href=\"#cite_note-53\">[53]<\/a><\/sup> Besides Dataverse, there are other alternatives such as Zenodo for storing various research data in repositories. Dataverse is distinguished from Zenodo by its more advanced authentication options and the superior concept of <a href=\"https:\/\/www.limswiki.org\/index.php\/Version_control\" title=\"Version control\" class=\"wiki-link\" data-key=\"81823f6b21d385f8db9ac0a17b571cc1\">version control<\/a>.<sup id=\"rdp-ebb-cite_ref-54\" class=\"reference\"><a href=\"#cite_note-54\">[54]<\/a><\/sup> As a result, Dataverse is implemented within the infrastructure. The repositories in the Dataverse are called Dataverses and can be subdivided for example by working groups or projects. Within the Dataverses so-called Datasets can be created, in which data, e.g., from measurements, can be saved. The Dataverse Project offers extensive metadata management and makes it possible to describe the individual Datasets more exactly, which facilitates interoperability of data. Furthermore, the Datasets can be directly linked to a publication with a digital object identifier (DOI), allowing the user to extract corresponding citations directly from the web application. Metadata management within Dataverse contextualizes the stored data and provides a high level of reusability for other users.<sup id=\"rdp-ebb-cite_ref-55\" class=\"reference\"><a href=\"#cite_note-55\">[55]<\/a><\/sup> Dataverse also provides a REST API to upload or query data, which simultaneously can be uploaded and searched with different filters for the metadata in a simplified manner via the web application. Thus, requirements of different users are served by it. Analogous to Apache Jena Fuski, InfluxDB and Dataverse are also implemented on a provided server within the distributed environment.\n<\/p><p>The next element of the implementation in the RDM Core Space is the message broker, which is implemented by Eclipse Mosquitto. It is a message broker that supports the MQTT communication protocol and enables interaction within the infrastructure and its microservices.<sup id=\"rdp-ebb-cite_ref-56\" class=\"reference\"><a href=\"#cite_note-56\">[56]<\/a><\/sup> At this stage of development, MQTT is used due to its less required deployment resources. In the future, however, this communication interface will be enhanced by AMQP due to the buffering and the larger range of functions, in order to thus be able to serve a wider scope of requirements.<sup id=\"rdp-ebb-cite_ref-57\" class=\"reference\"><a href=\"#cite_note-57\">[57]<\/a><\/sup> In this context, Eclipse Mosquitto provides the link between the PTs and DTs, the DTOS and DTs, the DTOS and Node-RED, and the DTOS and smart applications in general. By connecting the PTs and the DTs, a bidirectional connection of both units is formed and their interconnectivity is ensured. Eclipse Mosquitto provides different security levels for the message traffic, ranging from \"no security\" to \"encryption\" with Transport Layer Security (TLS) and TLS with a client certificate. The implementation currently provides no encryption for the message traffic, but this will be addressed in the future. The broker is located on the provided server for the infrastructure and operates beside the previously presented elements of the RDM Core Space.\n<\/p><p>In addition to the presented instances within the RDM Core Space, the DTOS represents a central component at this layer of the infrastructure, which performs different orchestration tasks as an event-driven microservice and is currently implemented as a <a href=\"https:\/\/www.limswiki.org\/index.php\/Python_(programming_language)\" title=\"Python (programming language)\" class=\"wiki-link\" data-key=\"ef6905a29cbb75d3c71e6bdf6e2915dd\">Python<\/a>-based microservice. It captures the registration of new DTs, and it creates their semantic representations in Jena Fuseki and a file system in the InfluxDB. Furthermore, the DTOS connects the Digital Twin Space with the Smart Application Space and thereby enables intelligent cooperation between the DTs. The sequence chart shown in Figure 6 illustrates the operations inside the infrastructure and the DTOS, which are outlined in more detail below.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig6_Lehmann_Sensors23_23-1.png\" class=\"image wiki-link\" data-key=\"fa3fb03b2394b932a30e5f730b37097b\"><img alt=\"Fig6 Lehmann Sensors23 23-1.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/d\/d0\/Fig6_Lehmann_Sensors23_23-1.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 6.<\/b> Registration of a digital twin within the RDM Core Space.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>The entire process starts with a PT being activated and registering with its underlying information in the DT information model via a HTTP registration request. Through this registration request, a DT is provided for the PT with the help of Eclipse Ditto. After the DT is successfully instantiated, it notifies the DTOS of its registration via MQTT. This is the event trigger for the DTOS, which now submits a HTTP request for the DT\u2019s information model. The DT then provides its information model to the DTOS. This information model contains, as shown in Appendix A exemplarily, the photometer, attributes, and features besides general properties. The general properties and the structure of the JSON with the keys attributes and features originate from the Eclipse Ditto information model and are necessary for registration, as well as instantiation. All components of the device like sensors or ADCs are included in the key features. Additionally, information for the DTOS is included with the variable <tt>regComplete<\/tt>, which is set to false by default. This variable is altered from false to true with the registration of the DT within the infrastructure and controlled by the DTOS to prevent the duplicate creation of the file system in the InfluxDB and the semantic information on the Apache Jena Fuseki server. The DTOS starts to extract the semantic information about the DT from the information model if the check of the variable <tt>regComplete<\/tt> results in the fact that the DT has not yet been registered. These semantic information are contained in the key attributes and structured as a resource description framework (RDF) triple according to the Web Ontology Language (OWL) to simplify its creation in the knowledge graph. After the positive check of the <tt>regComplete<\/tt> variable, the DTOS parses the content of the key attributes and generates a SPARQL query using the HTTP POST method and thereby transmits it to Apache Jena Fuskei, instantiating the DT in the knowledge graph and all its sensors as new individuals inside the presented ontology. Simultaneously, the DTOS creates a file system for the DT in the InfluxDB via HTTP to store its generated data. Subsequently, the DTOS reports the complete registration process to the DT, and the DT returns the notification of successful registration to its PT via MQTT. Thus, the PT is fully integrated into the infrastructure and can start generating data that now are saved via the DTOS in the associated file system and made accessible to other microservices as well as the smart applications in the Smart Application Space. In addition, the semantic information can now be queried through the knowledge graph and taken into account in queries about specific device properties.\n<\/p><p>Besides the orchestration of the DTs, the creation of file systems and semantic representations, the DTOS takes over the lifecycle management for the generated data as well as the DTs. For this purpose, the DTOS monitors the DTs registered in the infrastructure and enables their deregistration if required. During deregistration, the DTOS removes the semantic representations from the knowledge graph and moves the DT\u2019s data to the long-term archive, which is provided via the Dataverse, as well as deletes them when they reached the end of their lifecycle. Similarly, the DTOS monitors the generated data over its lifecycle from creation to publication to archiving. In doing so, the DTOS links the data to the associated publications and then moves the data to the archive until it deletes it at the end of the life cycle.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Smart_Application_Space\">Smart Application Space<\/span><\/h3>\n<p>Smart applications in the RDM infrastructure are settled at the top of the hierarchy. Applications for the evaluation of experiments and the creation of added value are to be located according to the infrastructure modalities that are as open as possible. Examples of such smart features would be AI algorithms for the evaluation of medical image data, context-based correlation of multi-dimensional parameter fields, optimization procedures for measurement arrangements, and much more. Some of the applications already implemented and those planned for the near future are discussed below.\n<\/p><p>Originally developed by IBM, the open-source software Node-RED is a tool for flow-based programming and is used for connecting hardware components, APIs, and online services. Node-RED provides an editor through the web browser that enables a graphically supported creation of flows with different nodes.<sup id=\"rdp-ebb-cite_ref-58\" class=\"reference\"><a href=\"#cite_note-58\">[58]<\/a><\/sup> With Node-RED further microservices for the RDM infrastructure are implemented, such as the knowledge-based recommendation system for measuring devices. With the knowledge-based recommendation system, a tool is implemented in the Smart Application Space that facilitates the selection of measuring devices for the end user. For this purpose, an input mask is set up in a Node-RED dashboard with which the parameters for a measurement to be performed can be specified. Based on these specifications, the microservice generates a SPARQL query and sends it to Apache Jena Fuseki. The response from the knowledge graph is output in tabular form with the required parameters. With the help of the input mask the required service, the target, and the output data can be defined. In addition, the required wavelength can be specified either as a specific value or as a range. All entries are optional and serve the refinement of the search filter. This recommendation system is featured in the proof of concept for the RDM infrastructures functionality validation.\n<\/p><p>In the future, further smart applications will be integrated into the infrastructure, such as scientific trial management, which provides two essential features for reliable and sustainable RDM. The first aspect of scientific trial management is the standardized creation of Dataverses and Datasets within the Dataverse. This is made possible via a web-based GUI in which using standardized catalogs Dataverses can be created or the metadata for projects Dataset can be specified. During the creation of a Dataset, the affiliation to a corresponding Dataverse can be established. The list of existing Dataverses is continuously updated to avoid duplicates. The use of standardized catalogs prevents different spellings and establishes a joint terminology among the institute\u2019s researchers. This increases the findability of the data in the Dataverse via the metadata search. \n<\/p><p>The second aspect of scientific trial management is the export of timer series data from the InfluxDB into a Dataset within the Dataverse. The export of time series data enables the movement of PT data to the Dataverse after a measurement series and thus increases its findability, accessibility, and reusability. An input mask is used to select the PT for which the data needs to be extracted and specify the Metadata on the basis of the standardized catalogs. The process gets triggered by an integrated button in the input mask. Based on the entered name, the corresponding bucket is determined in the InfluxDB and the data for the specified period is retrieved. The microservice then creates a Dataset in the Dataverse according to the specifications and metadata of the input mask and saves the data in a structured, neutral tabular format to improve interoperability and reusability.\n<\/p><p>In addition to the scientific trial management efforts, prospective activities include the implementation of an application for the user-friendly creation of data management plans in the Smart Application Space in order to fully meet the requirements for a reliable RDM, including the FAIR criteria to fully and sustainably document the project\u2019s own data lifecycle. For this purpose, the established software RDMO will be integrated into the infrastructure, which will be connected to the existing microservices using its native API, offering the researchers a centralized and standardized tool for the creation of data management plans.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Proof_of_concept_and_evaluation\">Proof of concept and evaluation<\/span><\/h2>\n<p>Following the implementation of the presented RDM architecture, a two-stage proof of concept with a respective concluding evaluation will demonstrate the advantages and the practicability of the authors\u2019 architectural approach. First, a knowledge-based recommendation system is outlined to illustrate one use case and the benefits of a knowledge graph. Subsequently, the practical implementation of a DT is presented using the photometer. Afterward, it is demonstratively visualized by a real measurement series. Although the demonstration is based on a measurement series, the concluding evaluation is only qualitative. The focus of this work is the introduction of a novel infrastructure for RDM, not the investigation of a specific experimental context.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Knowledge-based_recommendation_of_measuring_devices\">Knowledge-based recommendation of measuring devices<\/span><\/h3>\n<p>The first stage of the proof of concept is performed with a knowledge-based recommendation system to demonstrate the simplified identification of suitable measurement devices for generating research data enabled by the developed RDM infrastructure. For this purpose, three measuring devices have been selected and equipped with the necessary control system for their integration into the infrastructure and registration ability within Eclipse Ditto.\n<\/p><p>In addition to the photometer, the MIR scanner and the Multimodal Imaging System have been chosen because of their representative character. To enable this proof of concept, the individuals <i>OpticalMeasurementService<\/i>, <i>MFG_1<\/i> (corresponding to the photometer), <i>MFG_2<\/i> (corresponding to the Multimodal Imaging System), <i>MFG_3<\/i> (corresponding to the MIR scanner), <i>DiscreteDataPoint<\/i>, <i>HyperspectralImage<\/i>, <i>SkinLikeLiquid<\/i>, and <i>MouseBrain<\/i> have been inserted into the ontology. The <i>OpticalMeasurementService<\/i> describes the type of offered service which is the same for all three devices. The individuals <i>MFG_1<\/i>, <i>MFG_2<\/i>, and <i>MFG_3<\/i> represent the device\u2019s manufacturer, and the individuals <i>DiscreteDataPoint<\/i> and <i>HyperspectralImage<\/i> describe the generated data. The last two individuals <i>SkinLikeLiquid<\/i> and <i>MouseBrain<\/i> describe the target of the offered measurements, whereby they are only exemplary.\n<\/p><p>With the prepared ontology, the three devices are started, which triggers the process described in the implementation. After their boot, the devices register themselves in Ditto and are provided with a DT. Afterward, the DTs notify the DTOS of their registration, and the semantic representations are created automatically, which can now be queried. Figure 7 shows the GUI. It is divided into the three areas: Demand, Query, and Result. In the Demand area, various filters can be used to specify the demands on a measurement or a measuring device. The Query area displays the SPARQL query generated from the set filters by the microservice, which is sent to the knowledge graph. The Result area lists the device recommendations based on the submitted query. The fewer filters are set, the more comprehensive the results are. Figure 7 therefore features all devices presented in the research landscape, as only the service is defined as an <i>OpticalMeasurementService<\/i>.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig7_Lehmann_Sensors23_23-1.png\" class=\"image wiki-link\" data-key=\"c01f0ec4567277a30c84f4b2dba7c8a0\"><img alt=\"Fig7 Lehmann Sensors23 23-1.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/b\/b4\/Fig7_Lehmann_Sensors23_23-1.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 7.<\/b> Node-RED dashboard, showing query functionality for specification-driven device recommendation.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>Subsequently, the Node-RED dashboard is used for a device recommendation based on the present use case. This use case requires an optical measurement service for a skin-like liquid to provide discrete data points as measured values. The liquid needs to be measured with a wavelength range of 300 to 450 nanometers. Figure 8 illustrates the result of the query with the specified filters. In the Result section, the recommended devices for the defined requirements are shown. For this use case, the photometer\u2014which is able to cover the required wavelength range with its four sensors, provides an optical measurement service, and delivers discrete data points\u2014is recommended.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig8_Lehmann_Sensors23_23-1.png\" class=\"image wiki-link\" data-key=\"6818111e88f0e9219ad597a174cef8d1\"><img alt=\"Fig8 Lehmann Sensors23 23-1.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/b\/b0\/Fig8_Lehmann_Sensors23_23-1.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 8.<\/b> Node-RED dashboard, showing the detailed results of the query functionality for specification-driven device recommendation.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h3><span class=\"mw-headline\" id=\"Interfacing_the_digital_twin_of_a_photometrical_measurement_device\">Interfacing the digital twin of a photometrical measurement device<\/span><\/h3>\n<p>In the second step, the continuous integration of the photometer and its DT is proven. In this case, Node-RED, which is located in the Smart Application Space, was utilized again to provide a GUI to the DT. This ensures an interface to the DT via the DTOS and Eclipse Ditto for parameterizing, operating, and monitoring the physical measurement experiment. The practical procedures of the demonstration experiment are described below.\n<\/p><p>A dilution series was performed on a skin-like liquid suspension with variable concentration of New Coccine (E124 Sigma Aldrich, St. Louis, MO, United States) (Figure 9). On the detector side, two EPD-660-1-0.9<sup id=\"rdp-ebb-cite_ref-59\" class=\"reference\"><a href=\"#cite_note-59\">[59]<\/a><\/sup> from Roithner Lasertechnik were used. An LED, type ELD-650-523<sup id=\"rdp-ebb-cite_ref-60\" class=\"reference\"><a href=\"#cite_note-60\">[60]<\/a><\/sup> from Roithner Lasertechnik, was utilized as the light source. The connection between the photometer and the probe is realized by optical fibers. <i>Adc1<\/i> measures the reflected light signal caused by different concentrations of New Coccine. <i>Adc2<\/i> measures the relevant ambient interfering light. The obtained digital values can be converted into a corresponding current value based on the set parameters. The resulting measurement signal, cleaned of interfering signals, can then be determined by subtracting these two values.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig9_Lehmann_Sensors23_23-1.png\" class=\"image wiki-link\" data-key=\"d3c25696225da2b1355470ddf5940dff\"><img alt=\"Fig9 Lehmann Sensors23 23-1.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/a\/ae\/Fig9_Lehmann_Sensors23_23-1.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 9.<\/b> Node-RED dashboard, showing the visualization results and GUI of the photometer digital twin recording an actual measurement of a skin-like liquid suspension (<i>adc#1<\/i>) and reference ambient interference measurement (<i>adc#2<\/i>).<\/p><\/blockquote>\n<p><br \/> \n<\/p>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>For a brief qualitative evaluation, the comparison with the previously applied procedure for data processing and storage can be referred to. Since the photometer did not have a network interface, data could only be read out via the serial interface of the controller. Other researchers<sup id=\"rdp-ebb-cite_ref-:14_41-1\" class=\"reference\"><a href=\"#cite_note-:14-41\">[41]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:15_42-1\" class=\"reference\"><a href=\"#cite_note-:15-42\">[42]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:16_43-1\" class=\"reference\"><a href=\"#cite_note-:16-43\">[43]<\/a><\/sup> have employed manual methods to read out the measurement series using tools like Matlab and Labview or directly writing down the data arriving via the serial interface within a text file. Afterward, they used tools like Excel or Matlab to evaluate the measurement results manually. As a consequence, the typical problems of data management arise again, i.e., many locally, decentrally distributed research data, missing access authorizations, and missing experimental references. The new concept of holistic engagement within the RDM infrastructure overcomes these problems and provides an integration platform. All data are labeled and referenced in the database with relevant experimental information and thus made accessible for future investigations. The parameterization of the photometer no longer has to be done by reprogramming the controller, but can now be conveniently adjusted via its GUI. This approach prevents misoperation and saves expert knowledge, workload, and, consequently, time. Henceforth, measurement apparatuses can be developed independently of use cases and used for experiments without any subsequent effort.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Discussion\">Discussion<\/span><\/h2>\n<p>This work elaborates an architecture for reliable as well as sustainable RDM tailored for the increasing amount of gathered data at a large interdisciplinary research institute. Facing the problems of multiple measuring sources, experimental devices, and the associated mass data that must be processed, the need for modern RDM arises. Guiding away from the conservative treatment of research data in a decentralized and passive archival manner, accompanied by issues in data loss, accessibility, interpretability, etc., a holonic infrastructure for managing devices, experiments, and their resulting data, entirely new opportunities arise to exploit the extensive potential of digitized RDM in research institutions. Addressing the obstacles of transposing RDM in a complete digitized form, the envisioned concept is leveraged by the DT paradigm. DTs act use-case independently as a disruptive enabler technology in digital transformation. Therefore, the state of the art was examined and several requirements were identified in terms of RDM (<i>R-RDM1<\/i>\u2013<i>R-RDM4<\/i>) and DTs (<i>R-DT1<\/i>\u2013<i>R-DT6<\/i>). Aligning the architecture to a sustainable modern RDM practice, the broadly accepted FAIR principles were utilized. Additionally, the derived requirements for DTs naturally fit well with the aforementioned FAIR principles and jointly build the foundation on the herewithin outlined RDM infrastructure. Subsequently, all the requirements are set into context, followed by a brief overview of how they have been satisfied in the previous sections. In particular:\n<\/p>\n<ul><li><i>R-DT1: Replication, representation, and interoperability<\/i> meets perfectly with <i>R-RDM1: Findable<\/i>, <i>R-RDM3: Interoperable<\/i>, and <i>R-RDM4: Reusable<\/i>. Every PT is precisely described as DT within the Digital Twin Space facilitated by Eclipse Ditto and its JSON-based twin representation examined by the exemplary Photometer implementation. Herein embedded are all necessary structural and semantic information, which are further consumed by the DTOS, which instantiates this information into Apache Jena Fuseki\u2019s knowledge graph and thus establishes the pillar of later interaction and querying of all metadata within the RDM Core Space. To do so, the twins and their knowledge representation are unambiguously connected with each other.<\/li>\n<li><i>R-DT2: Interconnectivity and data acquisition<\/i> fits well with <i>R-RDM2: Accessible<\/i>. The DTs settled within Eclipse Ditto are connected bi-directionally via various standard IoT interfaces (e.g., MQTT, HTTP, etc.) to their physical pendants. The different types of the DT\u2019s data, including its metadata, are all covered by the dynamically updated JSON representation.<\/li>\n<li><i>R-DT3: Data storage<\/i> can be aligned with <i>R-RDM1: Findable<\/i>, <i>R-RDM2: Accessible<\/i>, <i>R-RDM3: Interoperable<\/i>, and <i>R-RDM4: Reusable<\/i>. All types of data are managed, homogeneously stored, and labeled by the DTOS within both applied storage approaches. While InfluxDB is serving a time-series technique, the Dataverse offers a repository-based approach. The labeling relates to the metadata managed by the DTOS and the instantiated individuals within Apache Jena Fuseki\u2019s knowledge graph. Data access can be achieved by calling the DTOS API or in a two-staged manner by querying the knowledge graph and afterward pulling the data from the resulting storage locations.<\/li>\n<li><i>R-DT4: Synchronization<\/i> could be satisfied in the demo implementation by using MQTT realized by Eclipse Mosquitto for the connection between the twins. Every change of state actualizes the DT and superordinated components or vice versa the PT.<\/li>\n<li><i>R-DT5: Interface and interaction<\/i> meets with <i>R-RDM2: Accessible<\/i> and <i>R-RDM3: Interoperable<\/i>. Eclipse Ditto, as well as the entire RDM Core Space components, offer open APIs to interact and request data. Even Node-RED, located in the Smart Application Space, embodies basic GUI and interaction schemes of the DTs as a demonstrative implementation.<\/li>\n<li><i>R-DT6: Optimization, analytics, simulation, and decision-making<\/i> addresses several value-adding features on top of DTs in accordance with every FAIR principle. The Smart Application Space is intended to be the habitat of these value-adding features and applications, which is founded on the subordinated three spaces. So far, just Node-RED represents one demonstrative approach to highlight potential future functionalities. With the introduced RDM infrastructure in place, there are no restrictions and obstacles in the potential magnitude of later developable smart applications or tools.<\/li>\n<li>The last requirement <i>R-DT6: Security<\/i> is suitable regarding <i>R-RDM2: Accessible<\/i>. Eclipse Ditto supports state-of-the-art security standards, including encryption, policy, and tenant-based DT management to gain proper access to required entities.<\/li><\/ul>\n<p>The contribution of this paper demonstrates that DTs are a perfectly suitable enabling technology for the central management of RDM entities and a reliable RDM itself. Further advantages can be drawn in the generation of knowledge and the reuse of data from other experiments or devices. Diversified datasets can be correlated with each other to gain entirely new insights. Especially for non-trivial human-readable data structures, i.e., multidimensional parameter arrays, this brings tremendous benefits. By processing research data in the way shown, inconsistencies, accessibility problems, data loss, etc., are no longer issues, also paving the way for more sustainability in research. Even the reusability of experimental knowledge can dissolve the need of reproducing difficult and energy-consuming experiment setups if still examined and well-labeled data are available. This also contributes to the minimization of the environmental footprint in research.\n<\/p><p>A self-developed photometer from the authors\u2019 research institute is used as the first demo implementation of a PT and its DT. Subsequently, it is shown that the maximum depth of integration allows access to all the functionalities of the RDM infrastructure. Especially the reconfigurability of already manufactured physical devices through their DT offers great modification opportunities. i.e., measuring devices can be built use-case-agnostic and later parameterized by their DT as proven before.\n<\/p><p>The introduced knowledge graph is dedicated as the heart of the RDM infrastructure. Paving the way for intelligent interaction behavior between the DTs, it was initially proven that a measuring equipment recommendation can be established based on the DT\u2019s knowledge representation.\n<\/p><p>However, the presented implementation covers mandatory sub-parts of the overall RDM infrastructure, and the individual parts will need to be investigated at a much more fine-grained level in future proceedings. The authors are aware of the fact that this kind of infrastructure is only reasonable in large research institutions, where large amounts of diverse data accumulate. The optimum potential of such twinning architectures comes with a critical count of DTs. Improved scenarios for collaboration and interaction can then be explored. Another challenge arises from more complex measuring devices like non-customizable mass spectrometers. The ability of all research devices to communicate via the network and automatically aggregate data for the RDM involves an increased security risk and vulnerability to functional errors. Thus, future work has to cope with the analysis and integration of extremely heterogeneous research equipment into the RDM infrastructure. Furthermore, the development of experiment-specific representative metrics to ease the correlation between various previously examined results must be tackled. Additionally, the knowledge graph as a foundation for intelligent behavior must be further developed to realize a higher degree of action within the Smart Application Space. Henceforth, with a higher level of automation of the RDM infrastructure, matters of security, confidentiality, and functional safety will be considered in the authors\u2019 future work.\n<\/p><p>Summarizing with the overall rationales, the RDM infrastructure is intended to be introduced and used at the authors\u2019 research institute within the upcoming three years, empowered by a research project. All the researchers should be sensitized in terms of FAIR-compliant RDM. Thus, better research-domain-independent cooperation between people should take place tackling problems together. To get them all on board and involved, it is absolutely necessary to pay the highest attention to usability and user-friendliness to avoid acceptance problems later on. This can be achieved by involving researchers from every domain in the development process of the RDM infrastructure.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Conclusions_and_future_work\">Conclusions and future work<\/span><\/h2>\n<p>In order to cope with the increasing amount of generated data at research institutions, this paper introduced an infrastructure for handling research data produced by manifold heterogeneous measurement devices and experimental setups. Facing rapidly growing requirements on RDM, the DT paradigm is utilized and highlighted as suitable enabler technology.\n<\/p><p>According to the analysis of relevant literature, the combination of both approaches results in a research gap, which this paper attempts to address. Requirements on reliable RDM, especially the FAIR principles, preserve value to researchers through the entire data lifecycle. In symbiosis with DT requirements, these principles could be afterward conceptually derived. Underlining the subsequent implementation, some of the typical measuring devices and apparatuses of the authors\u2019 institute, the CeMOS, have been highlighted as well as the specific use case of a photometer which was implemented afterward. Built upon four hierarchical key pillars, the architecture splits from the bottom up in the Physical Space, the Digital Twin Space, the RDM Core Space, and the Smart Application Space. Through the example of the photometer, a complete integration scenario was shown, including every mandatory part of the RDM infrastructure. Further, a proof of concept showed the feasibility and advantages of the utilization of knowledge graphs as well as the beneficial functionalities of DTs. In the subsequent discussion, the individual requirements were put into context with each other along with the implemented architecture. The discussion revealed that DTs are the perfect companion for the realization of a reliable and sustainable RDM to gain added value.\n<\/p><p>As its main contributions, this paper (1) introduced a novel approach for handling large amounts of research data according to the FAIR principles managing them in a centralized, structured manner; (2) obtained the necessity of utilizing DTs to overcome obstacles of the digital transformation within research institutes gearing them for Research 4.0; (3) developed a top-level knowledge graph addressing upcoming issues of interoperability and meta representation of experimental data and associated devices, paving the way for correlation of complex experimental data; (4) elaborated an implementation variant for reactive DTs as the base for later proactive realizations going beyond DTs as pure, passive state representations; and (5) outlined a highly reconfigurable design approach shown by a photometer opening up entirely new possibilities with less development efforts in hardware and software engineering by utilizing the DT, not the physical device itself.\n<\/p><p>However, several limitations force future research rationales. To fully exploit the potential of the architecture, much research data needs to be collected, which is only feasible for large research institutes. A lot of work regarding the integration of measuring devices and experimental setups (e.g., non-customizable mass spectrometers) needs to be done. Interaction schemes based on the knowledge graph must be elaborated to rise DTs to proactive behavior. Experiment-specific representative metrics must be envisioned to facilitate the correlation of the resulting data. In addition to that, smart applications have to be integrated into the Smart Application Space to gain the genuine added value of such infrastructures. This work covers mandatory sub-parts of the overall RDM infrastructure, but the single parts have to be examined in a much more fine-grained manner in the authors\u2019 future work. Likewise, issues of security, confidentiality, and functional security will be considered forthcoming. In order to substantiate the practicability, future publications with experimental use-case-specific data are planned.\n<\/p>\n<h2><span id=\"rdp-ebb-Abbreviations,_acronyms,_and_initialisms\"><\/span><span class=\"mw-headline\" id=\"Abbreviations.2C_acronyms.2C_and_initialisms\">Abbreviations, acronyms, and initialisms<\/span><\/h2>\n<ul><li><b>ADC<\/b>: analog-to-digital converter<\/li>\n<li><b>AI<\/b>: artificial intelligence<\/li>\n<li><b>AMQP<\/b>: Advanced Message Queuing Protocol<\/li>\n<li><b>API<\/b>: application programming interface<\/li>\n<li><b>ASIC<\/b>: application-specific integrated circuit<\/li>\n<li><b>CeMOS<\/b>: Center for Mass Spectrometry and Optical Spectroscopy<\/li>\n<li><b>DMP<\/b>: data management plan<\/li>\n<li><b>DOI<\/b>: digital object identifier<\/li>\n<li><b>DT<\/b>: digital twin<\/li>\n<li><b>DTOS<\/b>: DT Orchestration Service<\/li>\n<li><b>FAIR<\/b>: findable, accessible, interoperable, and reusable<\/li>\n<li><b>GPIO<\/b>: general purpose input\/output<\/li>\n<li><b>GUI<\/b>: graphical user interface<\/li>\n<li><b>HTTP<\/b>: Hypertext Transfer Protocol<\/li>\n<li><b>I\u00b2C<\/b>: inter-integrated circuit<\/li>\n<li><b>IC<\/b>: integrated circuit<\/li>\n<li><b>IoT<\/b>: internet of things<\/li>\n<li><b>IR<\/b>: infrared<\/li>\n<li><b>IRQ<\/b>: interrupt request<\/li>\n<li><b>JSON<\/b>: JavaScript Object Notation<\/li>\n<li><b>LSB<\/b>: Least Significant Bit<\/li>\n<li><b>MIR<\/b>: middle infrared<\/li>\n<li><b>MQTT<\/b>: Message Queueing Telemetry Transport<\/li>\n<li><b>NIR<\/b>: near infrared<\/li>\n<li><b>NASA<\/b>: National Aeronautics and Space Administration<\/li>\n<li><b>PT<\/b>: physical twin<\/li>\n<li><b>REST<\/b>: Representational State Transfer<\/li>\n<li><b>RDM<\/b>: research data management<\/li>\n<li><b>RDMO<\/b>: Research Data Management Organizer<\/li>\n<li><b>UV<\/b>: ultraviolet<\/li>\n<li><b>VIS<\/b>: visible<\/li><\/ul>\n<h2><span class=\"mw-headline\" id=\"Appendix_A\">Appendix A<\/span><\/h2>\n<p>he JSON-formatted information model of Eclipse Ditto is shown in Figure A1. For reasons of clarity and space, <i>LED3<\/i>\u2013<i>LED6<\/i>, as well as <i>adc2<\/i>\u2013<i>adc4<\/i>, have been substituted. Their structure is analogous to the ones shown. Based on the physical structure of a PT, it is one of the main components of the DT representation. Thus, it serves not only as a data basis for all applications infrastructurally settled above it but also as a semantically enriched information model, which the DTOS uses to describe the device within the knowledge graph.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:FigA1_Lehmann_Sensors23_23-1.png\" class=\"image wiki-link\" data-key=\"ba7bb595681e2d8a468fb212d1bfd6b9\"><img alt=\"FigA1 Lehmann Sensors23 23-1.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/0\/02\/FigA1_Lehmann_Sensors23_23-1.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure A1.<\/b> Eclipse Ditto information model.<\/p><\/blockquote>\n<p><br \/> \n<\/p>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h2><span class=\"mw-headline\" id=\"Acknowledgements\">Acknowledgements<\/span><\/h2>\n<p>Parts of this work presented in this paper were supported by a grant from the German Ministry of Education and Research (BMBF), grant number 16FDFH125.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Author_contributions\">Author contributions<\/span><\/h3>\n<p>Conceptualization, J.L., S.S., A.R. and T.H.; methodology, J.L., S.S., A.R. and T.H.; software, J.L., S.S. and T.H.; validation, J.L., S.S., A.R. and T.H.; investigation, J.L., A.R. and T.H.; writing\u2014original draft preparation, J.L., S.S., A.R. and T.H.; supervision, M.R. and J.R.; project administration, J.L. All authors have read and agreed to the published version of the manuscript.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Conflicts_of_interest\">Conflicts of interest<\/span><\/h3>\n<p>The authors declare no conflict of interest.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"References\">References<\/span><\/h2>\n<div class=\"reflist references-column-width\" style=\"-moz-column-width: 30em; -webkit-column-width: 30em; column-width: 30em; list-style-type: decimal;\">\n<div class=\"mw-references-wrap mw-references-columns\"><ol class=\"references\">\n<li id=\"cite_note-:0-1\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:0_1-0\">1.0<\/a><\/sup> <sup><a href=\"#cite_ref-:0_1-1\">1.1<\/a><\/sup> <sup><a href=\"#cite_ref-:0_1-2\">1.2<\/a><\/sup> <sup><a href=\"#cite_ref-:0_1-3\">1.3<\/a><\/sup> <sup><a href=\"#cite_ref-:0_1-4\">1.4<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Raptis, Theofanis P.; Passarella, Andrea; Conti, Marco (2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/8764545\/\" target=\"_blank\">\"Data Management in Industry 4.0: State of the Art and Open Challenges\"<\/a>. <i>IEEE Access<\/i> <b>7<\/b>: 97052\u201397093. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FACCESS.2019.2929296\" target=\"_blank\">10.1109\/ACCESS.2019.2929296<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2169-3536\" target=\"_blank\">2169-3536<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/8764545\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/8764545\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Data+Management+in+Industry+4.0%3A+State+of+the+Art+and+Open+Challenges&rft.jtitle=IEEE+Access&rft.aulast=Raptis&rft.aufirst=Theofanis+P.&rft.au=Raptis%2C%26%2332%3BTheofanis+P.&rft.au=Passarella%2C%26%2332%3BAndrea&rft.au=Conti%2C%26%2332%3BMarco&rft.date=2019&rft.volume=7&rft.pages=97052%E2%80%9397093&rft_id=info:doi\/10.1109%2FACCESS.2019.2929296&rft.issn=2169-3536&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F8764545%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-2\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-2\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Jones, E.; Kalantery, N.; Glover, B. (1 October 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/apo.org.au\/node\/262636\" target=\"_blank\">\"Research 4.0: Interim Report\"<\/a>. Analysis & Policy Observatory<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/apo.org.au\/node\/262636\" target=\"_blank\">https:\/\/apo.org.au\/node\/262636<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 06 December 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Research+4.0%3A+Interim+Report&rft.atitle=&rft.aulast=Jones%2C+E.%3B+Kalantery%2C+N.%3B+Glover%2C+B.&rft.au=Jones%2C+E.%3B+Kalantery%2C+N.%3B+Glover%2C+B.&rft.date=1+October+2019&rft.pub=Analysis+%26+Policy+Observatory&rft_id=https%3A%2F%2Fapo.org.au%2Fnode%2F262636&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:1-3\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:1_3-0\">3.0<\/a><\/sup> <sup><a href=\"#cite_ref-:1_3-1\">3.1<\/a><\/sup> <sup><a href=\"#cite_ref-:1_3-2\">3.2<\/a><\/sup> <sup><a href=\"#cite_ref-:1_3-3\">3.3<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Wilkinson, Mark D.; Dumontier, Michel; Aalbersberg, IJsbrand Jan; Appleton, Gabrielle; Axton, Myles; Baak, Arie; Blomberg, Niklas; Boiten, Jan-Willem <i>et al.<\/i> (15 March 2016). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/sdata201618\" target=\"_blank\">\"The FAIR Guiding Principles for scientific data management and stewardship\"<\/a> (in en). <i>Scientific Data<\/i> <b>3<\/b> (1): 160018. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fsdata.2016.18\" target=\"_blank\">10.1038\/sdata.2016.18<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2052-4463\" target=\"_blank\">2052-4463<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC4792175\/\" target=\"_blank\">PMC4792175<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/26978244\" target=\"_blank\">26978244<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/sdata201618\" target=\"_blank\">https:\/\/www.nature.com\/articles\/sdata201618<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+FAIR+Guiding+Principles+for+scientific+data+management+and+stewardship&rft.jtitle=Scientific+Data&rft.aulast=Wilkinson&rft.aufirst=Mark+D.&rft.au=Wilkinson%2C%26%2332%3BMark+D.&rft.au=Dumontier%2C%26%2332%3BMichel&rft.au=Aalbersberg%2C%26%2332%3BIJsbrand+Jan&rft.au=Appleton%2C%26%2332%3BGabrielle&rft.au=Axton%2C%26%2332%3BMyles&rft.au=Baak%2C%26%2332%3BArie&rft.au=Blomberg%2C%26%2332%3BNiklas&rft.au=Boiten%2C%26%2332%3BJan-Willem&rft.au=da+Silva+Santos%2C%26%2332%3BLuiz+Bonino&rft.date=15+March+2016&rft.volume=3&rft.issue=1&rft.pages=160018&rft_id=info:doi\/10.1038%2Fsdata.2016.18&rft.issn=2052-4463&rft_id=info:pmc\/PMC4792175&rft_id=info:pmid\/26978244&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fsdata201618&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:2-4\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:2_4-0\">4.0<\/a><\/sup> <sup><a href=\"#cite_ref-:2_4-1\">4.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation book\">Mons, Barend (9 March 2018) (in en). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.taylorfrancis.com\/books\/9781498753180\" target=\"_blank\"><i>Data Stewardship for Open Science: Implementing FAIR Principles<\/i><\/a> (1 ed.). Chapman and Hall\/CRC. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1201%2F9781315380711\" target=\"_blank\">10.1201\/9781315380711<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-315-38071-1<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.taylorfrancis.com\/books\/9781498753180\" target=\"_blank\">https:\/\/www.taylorfrancis.com\/books\/9781498753180<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=Data+Stewardship+for+Open+Science%3A+Implementing+FAIR+Principles&rft.aulast=Mons&rft.aufirst=Barend&rft.au=Mons%2C%26%2332%3BBarend&rft.date=9+March+2018&rft.edition=1&rft.pub=Chapman+and+Hall%2FCRC&rft_id=info:doi\/10.1201%2F9781315380711&rft.isbn=978-1-315-38071-1&rft_id=https%3A%2F%2Fwww.taylorfrancis.com%2Fbooks%2F9781498753180&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:3-5\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:3_5-0\">5.0<\/a><\/sup> <sup><a href=\"#cite_ref-:3_5-1\">5.1<\/a><\/sup> <sup><a href=\"#cite_ref-:3_5-2\">5.2<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation book\">Diepenbroek, M.; Gl\u00f6ckner, F.O.; Grobe, P. et al. (2014). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/dl.gi.de\/items\/618c1a92-fff2-423e-8c60-a0d6a65fe04f\" target=\"_blank\">\"Towards an integrated biodiversity and ecological research data management and archiving platform: the German federation for the curation of biological data (GFBio)\"<\/a>. <i>Informatik 2014<\/i>. Gesellschaft f\u00fcr Informatik e.V. pp. 1711\u201321. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-3-88579-626-8<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/dl.gi.de\/items\/618c1a92-fff2-423e-8c60-a0d6a65fe04f\" target=\"_blank\">https:\/\/dl.gi.de\/items\/618c1a92-fff2-423e-8c60-a0d6a65fe04f<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Towards+an+integrated+biodiversity+and+ecological+research+data+management+and+archiving+platform%3A+the+German+federation+for+the+curation+of+biological+data+%28GFBio%29&rft.atitle=Informatik+2014&rft.aulast=Diepenbroek%2C+M.%3B+Gl%C3%B6ckner%2C+F.O.%3B+Grobe%2C+P.+et+al.&rft.au=Diepenbroek%2C+M.%3B+Gl%C3%B6ckner%2C+F.O.%3B+Grobe%2C+P.+et+al.&rft.date=2014&rft.pages=pp.%26nbsp%3B1711%E2%80%9321&rft.pub=Gesellschaft+f%C3%BCr+Informatik+e.V&rft.isbn=978-3-88579-626-8&rft_id=https%3A%2F%2Fdl.gi.de%2Fitems%2F618c1a92-fff2-423e-8c60-a0d6a65fe04f&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:8-6\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:8_6-0\">6.0<\/a><\/sup> <sup><a href=\"#cite_ref-:8_6-1\">6.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Grieves, Michael (2016). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/rgdoi.net\/10.13140\/RG.2.2.26367.61609\" target=\"_blank\"><i>Origins of the Digital Twin Concept<\/i><\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.13140%2FRG.2.2.26367.61609\" target=\"_blank\">10.13140\/RG.2.2.26367.61609<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/rgdoi.net\/10.13140\/RG.2.2.26367.61609\" target=\"_blank\">http:\/\/rgdoi.net\/10.13140\/RG.2.2.26367.61609<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=Origins+of+the+Digital+Twin+Concept&rft.aulast=Grieves&rft.aufirst=Michael&rft.au=Grieves%2C%26%2332%3BMichael&rft.date=2016&rft_id=info:doi\/10.13140%2FRG.2.2.26367.61609&rft_id=http%3A%2F%2Frgdoi.net%2F10.13140%2FRG.2.2.26367.61609&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-7\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-7\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Mihai, Stefan; Yaqoob, Mahnoor; Hung, Dang V.; Davis, William; Towakel, Praveer; Raza, Mohsin; Karamanoglu, Mehmet; Barn, Balbir <i>et al.<\/i> (24\/2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9899718\/\" target=\"_blank\">\"Digital Twins: A Survey on Enabling Technologies, Challenges, Trends and Future Prospects\"<\/a>. <i>IEEE Communications Surveys & Tutorials<\/i> <b>24<\/b> (4): 2255\u20132291. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FCOMST.2022.3208773\" target=\"_blank\">10.1109\/COMST.2022.3208773<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1553-877X\" target=\"_blank\">1553-877X<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9899718\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/9899718\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Digital+Twins%3A+A+Survey+on+Enabling+Technologies%2C+Challenges%2C+Trends+and+Future+Prospects&rft.jtitle=IEEE+Communications+Surveys+%26+Tutorials&rft.aulast=Mihai&rft.aufirst=Stefan&rft.au=Mihai%2C%26%2332%3BStefan&rft.au=Yaqoob%2C%26%2332%3BMahnoor&rft.au=Hung%2C%26%2332%3BDang+V.&rft.au=Davis%2C%26%2332%3BWilliam&rft.au=Towakel%2C%26%2332%3BPraveer&rft.au=Raza%2C%26%2332%3BMohsin&rft.au=Karamanoglu%2C%26%2332%3BMehmet&rft.au=Barn%2C%26%2332%3BBalbir&rft.au=Shetve%2C%26%2332%3BDattaprasad&rft.date=24%2F2022&rft.volume=24&rft.issue=4&rft.pages=2255%E2%80%932291&rft_id=info:doi\/10.1109%2FCOMST.2022.3208773&rft.issn=1553-877X&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F9899718%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:9-8\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:9_8-0\">8.0<\/a><\/sup> <sup><a href=\"#cite_ref-:9_8-1\">8.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Massonet, Angela; Kiesel, Raphael; Schmitt, Robert H. (7 April 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.degruyter.com\/document\/doi\/10.3139\/104.112324\/html\" target=\"_blank\">\"Der Digitale Zwilling \u00fcber den Produktlebenszyklus: Das Konzept des Digitalen Zwillings verstehen und gewinnbringend einsetzen\"<\/a> (in en). <i>Zeitschrift f\u00fcr wirtschaftlichen Fabrikbetrieb<\/i> <b>115<\/b> (s1): 97\u2013100. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3139%2F104.112324\" target=\"_blank\">10.3139\/104.112324<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0947-0085\" target=\"_blank\">0947-0085<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.degruyter.com\/document\/doi\/10.3139\/104.112324\/html\" target=\"_blank\">https:\/\/www.degruyter.com\/document\/doi\/10.3139\/104.112324\/html<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Der+Digitale+Zwilling+%C3%BCber+den+Produktlebenszyklus%3A+Das+Konzept+des+Digitalen+Zwillings+verstehen+und+gewinnbringend+einsetzen&rft.jtitle=Zeitschrift+f%C3%BCr+wirtschaftlichen+Fabrikbetrieb&rft.aulast=Massonet&rft.aufirst=Angela&rft.au=Massonet%2C%26%2332%3BAngela&rft.au=Kiesel%2C%26%2332%3BRaphael&rft.au=Schmitt%2C%26%2332%3BRobert+H.&rft.date=7+April+2020&rft.volume=115&rft.issue=s1&rft.pages=97%E2%80%93100&rft_id=info:doi\/10.3139%2F104.112324&rft.issn=0947-0085&rft_id=https%3A%2F%2Fwww.degruyter.com%2Fdocument%2Fdoi%2F10.3139%2F104.112324%2Fhtml&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-9\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-9\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Semeraro, Concetta; Lezoche, Mario; Panetto, Herv\u00e9; Dassisti, Michele (1 September 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0166361521000762\" target=\"_blank\">\"Digital twin paradigm: A systematic literature review\"<\/a> (in en). <i>Computers in Industry<\/i> <b>130<\/b>: 103469. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.compind.2021.103469\" target=\"_blank\">10.1016\/j.compind.2021.103469<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0166361521000762\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0166361521000762<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Digital+twin+paradigm%3A+A+systematic+literature+review&rft.jtitle=Computers+in+Industry&rft.aulast=Semeraro&rft.aufirst=Concetta&rft.au=Semeraro%2C%26%2332%3BConcetta&rft.au=Lezoche%2C%26%2332%3BMario&rft.au=Panetto%2C%26%2332%3BHerv%C3%A9&rft.au=Dassisti%2C%26%2332%3BMichele&rft.date=1+September+2021&rft.volume=130&rft.pages=103469&rft_id=info:doi\/10.1016%2Fj.compind.2021.103469&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0166361521000762&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-10\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-10\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Bao, Qiangwei; Zhao, Gang; Yu, Yong; Dai, Sheng; Wang, Wei (1 November 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/link.springer.com\/10.1007\/s00170-021-07773-1\" target=\"_blank\">\"The ontology-based modeling and evolution of digital twin for assembly workshop\"<\/a> (in en). <i>The International Journal of Advanced Manufacturing Technology<\/i> <b>117<\/b> (1-2): 395\u2013411. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2Fs00170-021-07773-1\" target=\"_blank\">10.1007\/s00170-021-07773-1<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0268-3768\" target=\"_blank\">0268-3768<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/link.springer.com\/10.1007\/s00170-021-07773-1\" target=\"_blank\">https:\/\/link.springer.com\/10.1007\/s00170-021-07773-1<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+ontology-based+modeling+and+evolution+of+digital+twin+for+assembly+workshop&rft.jtitle=The+International+Journal+of+Advanced+Manufacturing+Technology&rft.aulast=Bao&rft.aufirst=Qiangwei&rft.au=Bao%2C%26%2332%3BQiangwei&rft.au=Zhao%2C%26%2332%3BGang&rft.au=Yu%2C%26%2332%3BYong&rft.au=Dai%2C%26%2332%3BSheng&rft.au=Wang%2C%26%2332%3BWei&rft.date=1+November+2021&rft.volume=117&rft.issue=1-2&rft.pages=395%E2%80%93411&rft_id=info:doi\/10.1007%2Fs00170-021-07773-1&rft.issn=0268-3768&rft_id=https%3A%2F%2Flink.springer.com%2F10.1007%2Fs00170-021-07773-1&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:10-11\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:10_11-0\">11.0<\/a><\/sup> <sup><a href=\"#cite_ref-:10_11-1\">11.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Sahlab, Nada; Kamm, Simon; Muller, Timo; Jazdi, Nasser; Weyrich, Michael (10 May 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9468219\/\" target=\"_blank\">\"Knowledge Graphs as Enhancers of Intelligent Digital Twins\"<\/a>. <i>2021 4th IEEE International Conference on Industrial Cyber-Physical Systems (ICPS)<\/i> (Victoria, BC, Canada: IEEE): 19\u201324. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FICPS49255.2021.9468219\" target=\"_blank\">10.1109\/ICPS49255.2021.9468219<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-7281-6207-2<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9468219\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/9468219\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Knowledge+Graphs+as+Enhancers+of+Intelligent+Digital+Twins&rft.jtitle=2021+4th+IEEE+International+Conference+on+Industrial+Cyber-Physical+Systems+%28ICPS%29&rft.aulast=Sahlab&rft.aufirst=Nada&rft.au=Sahlab%2C%26%2332%3BNada&rft.au=Kamm%2C%26%2332%3BSimon&rft.au=Muller%2C%26%2332%3BTimo&rft.au=Jazdi%2C%26%2332%3BNasser&rft.au=Weyrich%2C%26%2332%3BMichael&rft.date=10+May+2021&rft.pages=19%E2%80%9324&rft.place=Victoria%2C+BC%2C+Canada&rft.pub=IEEE&rft_id=info:doi\/10.1109%2FICPS49255.2021.9468219&rft.isbn=978-1-7281-6207-2&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F9468219%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-12\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-12\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation\" id=\"rdp-ebb-CITEREFPangSzafron2014\">Pang, Candy; Szafron, Duane (2014), Franch, Xavier; Ghose, Aditya K.; Lewis, Grace A. <i>et al.<\/i>., eds., <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/978-3-662-45391-9_50\" target=\"_blank\">\"Single Source of Truth (SSOT) for Service Oriented Architecture (SOA)\"<\/a> (in en), <i>Service-Oriented Computing<\/i> (Berlin, Heidelberg: Springer Berlin Heidelberg) <b>8831<\/b>: 575\u2013589, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2F978-3-662-45391-9_50\" target=\"_blank\">10.1007\/978-3-662-45391-9_50<\/a>, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-3-662-45390-2<span class=\"printonly\">, <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/978-3-662-45391-9_50\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/978-3-662-45391-9_50<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 2023-09-12<\/span><\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Single+Source+of+Truth+%28SSOT%29+for+Service+Oriented+Architecture+%28SOA%29&rft.jtitle=Service-Oriented+Computing&rft.aulast=Pang&rft.aufirst=Candy&rft.au=Pang%2C%26%2332%3BCandy&rft.au=Szafron%2C%26%2332%3BDuane&rft.date=2014&rft.volume=8831&rft.pages=575%E2%80%93589&rft.place=Berlin%2C+Heidelberg&rft.pub=Springer+Berlin+Heidelberg&rft_id=info:doi\/10.1007%2F978-3-662-45391-9_50&rft.isbn=978-3-662-45390-2&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2F978-3-662-45391-9_50&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-13\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-13\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Miksa, Tomasz; Cardoso, Joao; Borbinha, Jose (1 December 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/8622618\/\" target=\"_blank\">\"Framing the scope of the common data model for machine-actionable Data Management Plans\"<\/a>. <i>2018 IEEE International Conference on Big Data (Big Data)<\/i> (Seattle, WA, USA: IEEE): 2733\u20132742. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FBigData.2018.8622618\" target=\"_blank\">10.1109\/BigData.2018.8622618<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-5386-5035-6<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/8622618\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/8622618\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Framing+the+scope+of+the+common+data+model+for+machine-actionable+Data+Management+Plans&rft.jtitle=2018+IEEE+International+Conference+on+Big+Data+%28Big+Data%29&rft.aulast=Miksa&rft.aufirst=Tomasz&rft.au=Miksa%2C%26%2332%3BTomasz&rft.au=Cardoso%2C%26%2332%3BJoao&rft.au=Borbinha%2C%26%2332%3BJose&rft.date=1+December+2018&rft.pages=2733%E2%80%932742&rft.place=Seattle%2C+WA%2C+USA&rft.pub=IEEE&rft_id=info:doi\/10.1109%2FBigData.2018.8622618&rft.isbn=978-1-5386-5035-6&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F8622618%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:4-14\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:4_14-0\">14.0<\/a><\/sup> <sup><a href=\"#cite_ref-:4_14-1\">14.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Gray, Jim; Liu, David T.; Nieto-Santisteban, Maria; Szalay, Alex; DeWitt, David J.; Heber, Gerd (1 December 2005). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/dl.acm.org\/doi\/10.1145\/1107499.1107503\" target=\"_blank\">\"Scientific data management in the coming decade\"<\/a> (in en). <i>ACM SIGMOD Record<\/i> <b>34<\/b> (4): 34\u201341. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1145%2F1107499.1107503\" target=\"_blank\">10.1145\/1107499.1107503<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0163-5808\" target=\"_blank\">0163-5808<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/dl.acm.org\/doi\/10.1145\/1107499.1107503\" target=\"_blank\">https:\/\/dl.acm.org\/doi\/10.1145\/1107499.1107503<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Scientific+data+management+in+the+coming+decade&rft.jtitle=ACM+SIGMOD+Record&rft.aulast=Gray&rft.aufirst=Jim&rft.au=Gray%2C%26%2332%3BJim&rft.au=Liu%2C%26%2332%3BDavid+T.&rft.au=Nieto-Santisteban%2C%26%2332%3BMaria&rft.au=Szalay%2C%26%2332%3BAlex&rft.au=DeWitt%2C%26%2332%3BDavid+J.&rft.au=Heber%2C%26%2332%3BGerd&rft.date=1+December+2005&rft.volume=34&rft.issue=4&rft.pages=34%E2%80%9341&rft_id=info:doi\/10.1145%2F1107499.1107503&rft.issn=0163-5808&rft_id=https%3A%2F%2Fdl.acm.org%2Fdoi%2F10.1145%2F1107499.1107503&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-15\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-15\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Schadt, Eric E.; Linderman, Michael D.; Sorenson, Jon; Lee, Lawrence; Nolan, Garry P. (1 September 2010). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/nrg2857\" target=\"_blank\">\"Computational solutions to large-scale data management and analysis\"<\/a> (in en). <i>Nature Reviews Genetics<\/i> <b>11<\/b> (9): 647\u2013657. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fnrg2857\" target=\"_blank\">10.1038\/nrg2857<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1471-0056\" target=\"_blank\">1471-0056<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC3124937\/\" target=\"_blank\">PMC3124937<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/20717155\" target=\"_blank\">20717155<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/nrg2857\" target=\"_blank\">https:\/\/www.nature.com\/articles\/nrg2857<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Computational+solutions+to+large-scale+data+management+and+analysis&rft.jtitle=Nature+Reviews+Genetics&rft.aulast=Schadt&rft.aufirst=Eric+E.&rft.au=Schadt%2C%26%2332%3BEric+E.&rft.au=Linderman%2C%26%2332%3BMichael+D.&rft.au=Sorenson%2C%26%2332%3BJon&rft.au=Lee%2C%26%2332%3BLawrence&rft.au=Nolan%2C%26%2332%3BGarry+P.&rft.date=1+September+2010&rft.volume=11&rft.issue=9&rft.pages=647%E2%80%93657&rft_id=info:doi\/10.1038%2Fnrg2857&rft.issn=1471-0056&rft_id=info:pmc\/PMC3124937&rft_id=info:pmid\/20717155&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fnrg2857&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:5-16\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:5_16-0\">16.0<\/a><\/sup> <sup><a href=\"#cite_ref-:5_16-1\">16.1<\/a><\/sup> <sup><a href=\"#cite_ref-:5_16-2\">16.2<\/a><\/sup> <sup><a href=\"#cite_ref-:5_16-3\">16.3<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Andreas, F\u00fcrholz; Martin, Jaekel (2021) (in en). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/digitalcollection.zhaw.ch\/handle\/11475\/23073\" target=\"_blank\"><i>Data life cycle management pilot projects and implications for research data management at universities of applied sciences<\/i><\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.21256%2FZHAW-23073\" target=\"_blank\">10.21256\/ZHAW-23073<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/digitalcollection.zhaw.ch\/handle\/11475\/23073\" target=\"_blank\">https:\/\/digitalcollection.zhaw.ch\/handle\/11475\/23073<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=Data+life+cycle+management+pilot+projects+and+implications+for+research+data+management+at+universities+of+applied+sciences&rft.aulast=Andreas&rft.aufirst=F%C3%BCrholz&rft.au=Andreas%2C%26%2332%3BF%C3%BCrholz&rft.au=Martin%2C%26%2332%3BJaekel&rft.date=2021&rft_id=info:doi\/10.21256%2FZHAW-23073&rft_id=https%3A%2F%2Fdigitalcollection.zhaw.ch%2Fhandle%2F11475%2F23073&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:6-17\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:6_17-0\">17.0<\/a><\/sup> <sup><a href=\"#cite_ref-:6_17-1\">17.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Redkina, N. S. (1 April 2019). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.3103\/S0147688219020035\" target=\"_blank\">\"Current Trends in Research Data Management\"<\/a> (in en). <i>Scientific and Technical Information Processing<\/i> <b>46<\/b> (2): 53\u201358. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3103%2FS0147688219020035\" target=\"_blank\">10.3103\/S0147688219020035<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0147-6882\" target=\"_blank\">0147-6882<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.3103\/S0147688219020035\" target=\"_blank\">http:\/\/link.springer.com\/10.3103\/S0147688219020035<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Current+Trends+in+Research+Data+Management&rft.jtitle=Scientific+and+Technical+Information+Processing&rft.aulast=Redkina&rft.aufirst=N.+S.&rft.au=Redkina%2C%26%2332%3BN.+S.&rft.date=1+April+2019&rft.volume=46&rft.issue=2&rft.pages=53%E2%80%9358&rft_id=info:doi\/10.3103%2FS0147688219020035&rft.issn=0147-6882&rft_id=http%3A%2F%2Flink.springer.com%2F10.3103%2FS0147688219020035&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-18\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-18\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Engelhardt, C.; Enke, H.; Klar, J. et al. (2017). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www-archive.cseas.kyoto-u.ac.jp\/ipres2017.jp\/wp-content\/uploads\/27Claudia-Engelhardt.pdf\" target=\"_blank\">\"Research Data Management Organiser\"<\/a> (PDF). <i>Proceedings of the 14th International Conference on Digital Preservation<\/i>: 25\u201329<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www-archive.cseas.kyoto-u.ac.jp\/ipres2017.jp\/wp-content\/uploads\/27Claudia-Engelhardt.pdf\" target=\"_blank\">http:\/\/www-archive.cseas.kyoto-u.ac.jp\/ipres2017.jp\/wp-content\/uploads\/27Claudia-Engelhardt.pdf<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Research+Data+Management+Organiser&rft.jtitle=Proceedings+of+the+14th+International+Conference+on+Digital+Preservation&rft.aulast=Engelhardt%2C+C.%3B+Enke%2C+H.%3B+Klar%2C+J.+et+al.&rft.au=Engelhardt%2C+C.%3B+Enke%2C+H.%3B+Klar%2C+J.+et+al.&rft.date=2017&rft.pages=25%E2%80%9329&rft_id=http%3A%2F%2Fwww-archive.cseas.kyoto-u.ac.jp%2Fipres2017.jp%2Fwp-content%2Fuploads%2F27Claudia-Engelhardt.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-19\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-19\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Rosen, Roland; von Wichert, Georg; Lo, George; Bettenhausen, Kurt D. (2015). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2405896315003808\" target=\"_blank\">\"About The Importance of Autonomy and Digital Twins for the Future of Manufacturing\"<\/a> (in en). <i>IFAC-PapersOnLine<\/i> <b>48<\/b> (3): 567\u2013572. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.ifacol.2015.06.141\" target=\"_blank\">10.1016\/j.ifacol.2015.06.141<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2405896315003808\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2405896315003808<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=About+The+Importance+of+Autonomy+and+Digital+Twins+for+the+Future+of+Manufacturing&rft.jtitle=IFAC-PapersOnLine&rft.aulast=Rosen&rft.aufirst=Roland&rft.au=Rosen%2C%26%2332%3BRoland&rft.au=von+Wichert%2C%26%2332%3BGeorg&rft.au=Lo%2C%26%2332%3BGeorge&rft.au=Bettenhausen%2C%26%2332%3BKurt+D.&rft.date=2015&rft.volume=48&rft.issue=3&rft.pages=567%E2%80%93572&rft_id=info:doi\/10.1016%2Fj.ifacol.2015.06.141&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2405896315003808&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-20\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-20\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation\" id=\"rdp-ebb-CITEREFWang2020\">Wang, Zongyan (18 March 2020), B\u00e1nyai, Tam\u00e1s; Petrilloand Fabio De Felice, Antonella, eds., <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.intechopen.com\/books\/industry-4-0-impact-on-intelligent-logistics-and-manufacturing\/digital-twin-technology\" target=\"_blank\">\"Digital Twin Technology\"<\/a> (in en), <i>Industry 4.0 - Impact on Intelligent Logistics and Manufacturing<\/i> (IntechOpen), <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.5772%2Fintechopen.80974\" target=\"_blank\">10.5772\/intechopen.80974<\/a>, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-953-51-6996-3<span class=\"printonly\">, <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.intechopen.com\/books\/industry-4-0-impact-on-intelligent-logistics-and-manufacturing\/digital-twin-technology\" target=\"_blank\">https:\/\/www.intechopen.com\/books\/industry-4-0-impact-on-intelligent-logistics-and-manufacturing\/digital-twin-technology<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 2023-09-12<\/span><\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Digital+Twin+Technology&rft.jtitle=Industry+4.0+-+Impact+on+Intelligent+Logistics+and+Manufacturing&rft.aulast=Wang&rft.aufirst=Zongyan&rft.au=Wang%2C%26%2332%3BZongyan&rft.date=18+March+2020&rft.pub=IntechOpen&rft_id=info:doi\/10.5772%2Fintechopen.80974&rft.isbn=978-953-51-6996-3&rft_id=https%3A%2F%2Fwww.intechopen.com%2Fbooks%2Findustry-4-0-impact-on-intelligent-logistics-and-manufacturing%2Fdigital-twin-technology&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-21\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-21\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Barricelli, Barbara Rita; Casiraghi, Elena; Fogli, Daniela (2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/8901113\/\" target=\"_blank\">\"A Survey on Digital Twin: Definitions, Characteristics, Applications, and Design Implications\"<\/a>. <i>IEEE Access<\/i> <b>7<\/b>: 167653\u2013167671. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FACCESS.2019.2953499\" target=\"_blank\">10.1109\/ACCESS.2019.2953499<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2169-3536\" target=\"_blank\">2169-3536<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/8901113\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/8901113\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Survey+on+Digital+Twin%3A+Definitions%2C+Characteristics%2C+Applications%2C+and+Design+Implications&rft.jtitle=IEEE+Access&rft.aulast=Barricelli&rft.aufirst=Barbara+Rita&rft.au=Barricelli%2C%26%2332%3BBarbara+Rita&rft.au=Casiraghi%2C%26%2332%3BElena&rft.au=Fogli%2C%26%2332%3BDaniela&rft.date=2019&rft.volume=7&rft.pages=167653%E2%80%93167671&rft_id=info:doi\/10.1109%2FACCESS.2019.2953499&rft.issn=2169-3536&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F8901113%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-22\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-22\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Shafto, M.; Conroy, M.; Doyle, R. et al. (April 2012). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nasa.gov\/sites\/default\/files\/501321main_TA11-ID_rev4_NRC-wTASR.pdf\" target=\"_blank\">\"Modeling, Simulation, Information Technology & Processing Roadmap\"<\/a> (PDF). National Aeronautics and Space Administration<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nasa.gov\/sites\/default\/files\/501321main_TA11-ID_rev4_NRC-wTASR.pdf\" target=\"_blank\">https:\/\/www.nasa.gov\/sites\/default\/files\/501321main_TA11-ID_rev4_NRC-wTASR.pdf<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Modeling%2C+Simulation%2C+Information+Technology+%26+Processing+Roadmap&rft.atitle=&rft.aulast=Shafto%2C+M.%3B+Conroy%2C+M.%3B+Doyle%2C+R.+et+al.&rft.au=Shafto%2C+M.%3B+Conroy%2C+M.%3B+Doyle%2C+R.+et+al.&rft.date=April+2012&rft.pub=National+Aeronautics+and+Space+Administration&rft_id=https%3A%2F%2Fwww.nasa.gov%2Fsites%2Fdefault%2Ffiles%2F501321main_TA11-ID_rev4_NRC-wTASR.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:7-23\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:7_23-0\">23.0<\/a><\/sup> <sup><a href=\"#cite_ref-:7_23-1\">23.1<\/a><\/sup> <sup><a href=\"#cite_ref-:7_23-2\">23.2<\/a><\/sup> <sup><a href=\"#cite_ref-:7_23-3\">23.3<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Grieves, Michael (25 May 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/digitaltwin1.org\/articles\/2-8\/v1\" target=\"_blank\">\"Intelligent digital twins and the development and management of complex systems\"<\/a> (in en). <i>Digital Twin<\/i> <b>2<\/b>: 8. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.12688%2Fdigitaltwin.17574.1\" target=\"_blank\">10.12688\/digitaltwin.17574.1<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2752-5783\" target=\"_blank\">2752-5783<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/digitaltwin1.org\/articles\/2-8\/v1\" target=\"_blank\">https:\/\/digitaltwin1.org\/articles\/2-8\/v1<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Intelligent+digital+twins+and+the+development+and+management+of+complex+systems&rft.jtitle=Digital+Twin&rft.aulast=Grieves&rft.aufirst=Michael&rft.au=Grieves%2C%26%2332%3BMichael&rft.date=25+May+2022&rft.volume=2&rft.pages=8&rft_id=info:doi\/10.12688%2Fdigitaltwin.17574.1&rft.issn=2752-5783&rft_id=https%3A%2F%2Fdigitaltwin1.org%2Farticles%2F2-8%2Fv1&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-24\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-24\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Dorofeev, Kirill; Zoitl, Alois (1 July 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/8471978\/\" target=\"_blank\">\"Skill-based Engineering Approach using OPC UA Programs\"<\/a>. <i>2018 IEEE 16th International Conference on Industrial Informatics (INDIN)<\/i> (Porto: IEEE): 1098\u20131103. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FINDIN.2018.8471978\" target=\"_blank\">10.1109\/INDIN.2018.8471978<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-5386-4829-2<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/8471978\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/8471978\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Skill-based+Engineering+Approach+using+OPC+UA+Programs&rft.jtitle=2018+IEEE+16th+International+Conference+on+Industrial+Informatics+%28INDIN%29&rft.aulast=Dorofeev&rft.aufirst=Kirill&rft.au=Dorofeev%2C%26%2332%3BKirill&rft.au=Zoitl%2C%26%2332%3BAlois&rft.date=1+July+2018&rft.pages=1098%E2%80%931103&rft.place=Porto&rft.pub=IEEE&rft_id=info:doi\/10.1109%2FINDIN.2018.8471978&rft.isbn=978-1-5386-4829-2&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F8471978%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-25\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-25\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Sepasgozar, Samad M. E. (2 April 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.mdpi.com\/2075-5309\/11\/4\/151\" target=\"_blank\">\"Differentiating Digital Twin from Digital Shadow: Elucidating a Paradigm Shift to Expedite a Smart, Sustainable Built Environment\"<\/a> (in en). <i>Buildings<\/i> <b>11<\/b> (4): 151. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3390%2Fbuildings11040151\" target=\"_blank\">10.3390\/buildings11040151<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2075-5309\" target=\"_blank\">2075-5309<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.mdpi.com\/2075-5309\/11\/4\/151\" target=\"_blank\">https:\/\/www.mdpi.com\/2075-5309\/11\/4\/151<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Differentiating+Digital+Twin+from+Digital+Shadow%3A+Elucidating+a+Paradigm+Shift+to+Expedite+a+Smart%2C+Sustainable+Built+Environment&rft.jtitle=Buildings&rft.aulast=Sepasgozar&rft.aufirst=Samad+M.+E.&rft.au=Sepasgozar%2C%26%2332%3BSamad+M.+E.&rft.date=2+April+2021&rft.volume=11&rft.issue=4&rft.pages=151&rft_id=info:doi\/10.3390%2Fbuildings11040151&rft.issn=2075-5309&rft_id=https%3A%2F%2Fwww.mdpi.com%2F2075-5309%2F11%2F4%2F151&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-26\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-26\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">van der Valk, Hendrik; Ha\u00dfe, Hendrik; M\u00f6ller, Frederik; Otto, Boris (1 June 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/link.springer.com\/10.1007\/s12599-021-00727-7\" target=\"_blank\">\"Archetypes of Digital Twins\"<\/a> (in en). <i>Business & Information Systems Engineering<\/i> <b>64<\/b> (3): 375\u2013391. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2Fs12599-021-00727-7\" target=\"_blank\">10.1007\/s12599-021-00727-7<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2363-7005\" target=\"_blank\">2363-7005<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/link.springer.com\/10.1007\/s12599-021-00727-7\" target=\"_blank\">https:\/\/link.springer.com\/10.1007\/s12599-021-00727-7<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Archetypes+of+Digital+Twins&rft.jtitle=Business+%26+Information+Systems+Engineering&rft.aulast=van+der+Valk&rft.aufirst=Hendrik&rft.au=van+der+Valk%2C%26%2332%3BHendrik&rft.au=Ha%C3%9Fe%2C%26%2332%3BHendrik&rft.au=M%C3%B6ller%2C%26%2332%3BFrederik&rft.au=Otto%2C%26%2332%3BBoris&rft.date=1+June+2022&rft.volume=64&rft.issue=3&rft.pages=375%E2%80%93391&rft_id=info:doi\/10.1007%2Fs12599-021-00727-7&rft.issn=2363-7005&rft_id=https%3A%2F%2Flink.springer.com%2F10.1007%2Fs12599-021-00727-7&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-27\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-27\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Singh, Maulshree; Fuenmayor, Evert; Hinchy, Eoin; Qiao, Yuansong; Murray, Niall; Devine, Declan (24 May 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.mdpi.com\/2571-5577\/4\/2\/36\" target=\"_blank\">\"Digital Twin: Origin to Future\"<\/a> (in en). <i>Applied System Innovation<\/i> <b>4<\/b> (2): 36. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3390%2Fasi4020036\" target=\"_blank\">10.3390\/asi4020036<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2571-5577\" target=\"_blank\">2571-5577<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.mdpi.com\/2571-5577\/4\/2\/36\" target=\"_blank\">https:\/\/www.mdpi.com\/2571-5577\/4\/2\/36<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Digital+Twin%3A+Origin+to+Future&rft.jtitle=Applied+System+Innovation&rft.aulast=Singh&rft.aufirst=Maulshree&rft.au=Singh%2C%26%2332%3BMaulshree&rft.au=Fuenmayor%2C%26%2332%3BEvert&rft.au=Hinchy%2C%26%2332%3BEoin&rft.au=Qiao%2C%26%2332%3BYuansong&rft.au=Murray%2C%26%2332%3BNiall&rft.au=Devine%2C%26%2332%3BDeclan&rft.date=24+May+2021&rft.volume=4&rft.issue=2&rft.pages=36&rft_id=info:doi\/10.3390%2Fasi4020036&rft.issn=2571-5577&rft_id=https%3A%2F%2Fwww.mdpi.com%2F2571-5577%2F4%2F2%2F36&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-28\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-28\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Zehnder, Philipp; Riemer, Dominik (1 December 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/8622400\/\" target=\"_blank\">\"Representing Industrial Data Streams in Digital Twins using Semantic Labeling\"<\/a>. <i>2018 IEEE International Conference on Big Data (Big Data)<\/i> (Seattle, WA, USA: IEEE): 4223\u20134226. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FBigData.2018.8622400\" target=\"_blank\">10.1109\/BigData.2018.8622400<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-5386-5035-6<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/8622400\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/8622400\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Representing+Industrial+Data+Streams+in+Digital+Twins+using+Semantic+Labeling&rft.jtitle=2018+IEEE+International+Conference+on+Big+Data+%28Big+Data%29&rft.aulast=Zehnder&rft.aufirst=Philipp&rft.au=Zehnder%2C%26%2332%3BPhilipp&rft.au=Riemer%2C%26%2332%3BDominik&rft.date=1+December+2018&rft.pages=4223%E2%80%934226&rft.place=Seattle%2C+WA%2C+USA&rft.pub=IEEE&rft_id=info:doi\/10.1109%2FBigData.2018.8622400&rft.isbn=978-1-5386-5035-6&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F8622400%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:11-29\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:11_29-0\">29.0<\/a><\/sup> <sup><a href=\"#cite_ref-:11_29-1\">29.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Lehmann, Joel; Lober, Andreas; Rache, Alessa; Baumg\u00e4rtel, Hartwig; Reichwald, Julian (2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.scitepress.org\/DigitalLibrary\/Link.aspx?doi=10.5220\/0011141200003286\" target=\"_blank\">\"Collaboration of Semantically Enriched Digital Twins based on a Marketplace Approach:\"<\/a>. <i>Proceedings of the 19th International Conference on Wireless Networks and Mobile Systems<\/i> (Lisbon, Portugal: SCITEPRESS - Science and Technology Publications): 35\u201345. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.5220%2F0011141200003286\" target=\"_blank\">10.5220\/0011141200003286<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-989-758-592-0<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.scitepress.org\/DigitalLibrary\/Link.aspx?doi=10.5220\/0011141200003286\" target=\"_blank\">https:\/\/www.scitepress.org\/DigitalLibrary\/Link.aspx?doi=10.5220\/0011141200003286<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Collaboration+of+Semantically+Enriched+Digital+Twins+based+on+a+Marketplace+Approach%3A&rft.jtitle=Proceedings+of+the+19th+International+Conference+on+Wireless+Networks+and+Mobile+Systems&rft.aulast=Lehmann&rft.aufirst=Joel&rft.au=Lehmann%2C%26%2332%3BJoel&rft.au=Lober%2C%26%2332%3BAndreas&rft.au=Rache%2C%26%2332%3BAlessa&rft.au=Baumg%C3%A4rtel%2C%26%2332%3BHartwig&rft.au=Reichwald%2C%26%2332%3BJulian&rft.date=2022&rft.pages=35%E2%80%9345&rft.place=Lisbon%2C+Portugal&rft.pub=SCITEPRESS+-+Science+and+Technology+Publications&rft_id=info:doi\/10.5220%2F0011141200003286&rft.isbn=978-989-758-592-0&rft_id=https%3A%2F%2Fwww.scitepress.org%2FDigitalLibrary%2FLink.aspx%3Fdoi%3D10.5220%2F0011141200003286&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-30\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-30\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">G\u00f6ppert, Amon; Grahn, Lea; Rachner, Jonas; Grunert, Dennis; Hort, Simon; Schmitt, Robert H. (1 June 2023). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/link.springer.com\/10.1007\/s10845-021-01860-6\" target=\"_blank\">\"Pipeline for ontology-based modeling and automated deployment of digital twins for planning and control of manufacturing systems\"<\/a> (in en). <i>Journal of Intelligent Manufacturing<\/i> <b>34<\/b> (5): 2133\u20132152. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2Fs10845-021-01860-6\" target=\"_blank\">10.1007\/s10845-021-01860-6<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0956-5515\" target=\"_blank\">0956-5515<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/link.springer.com\/10.1007\/s10845-021-01860-6\" target=\"_blank\">https:\/\/link.springer.com\/10.1007\/s10845-021-01860-6<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Pipeline+for+ontology-based+modeling+and+automated+deployment+of+digital+twins+for+planning+and+control+of+manufacturing+systems&rft.jtitle=Journal+of+Intelligent+Manufacturing&rft.aulast=G%C3%B6ppert&rft.aufirst=Amon&rft.au=G%C3%B6ppert%2C%26%2332%3BAmon&rft.au=Grahn%2C%26%2332%3BLea&rft.au=Rachner%2C%26%2332%3BJonas&rft.au=Grunert%2C%26%2332%3BDennis&rft.au=Hort%2C%26%2332%3BSimon&rft.au=Schmitt%2C%26%2332%3BRobert+H.&rft.date=1+June+2023&rft.volume=34&rft.issue=5&rft.pages=2133%E2%80%932152&rft_id=info:doi\/10.1007%2Fs10845-021-01860-6&rft.issn=0956-5515&rft_id=https%3A%2F%2Flink.springer.com%2F10.1007%2Fs10845-021-01860-6&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-31\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-31\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Zhang, Chao; Zhou, Guanghui; He, Jun; Li, Zhi; Cheng, Wei (2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2212827119306985\" target=\"_blank\">\"A data- and knowledge-driven framework for digital twin manufacturing cell\"<\/a> (in en). <i>Procedia CIRP<\/i> <b>83<\/b>: 345\u2013350. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.procir.2019.04.084\" target=\"_blank\">10.1016\/j.procir.2019.04.084<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2212827119306985\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2212827119306985<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+data-+and+knowledge-driven+framework+for+digital+twin+manufacturing+cell&rft.jtitle=Procedia+CIRP&rft.aulast=Zhang&rft.aufirst=Chao&rft.au=Zhang%2C%26%2332%3BChao&rft.au=Zhou%2C%26%2332%3BGuanghui&rft.au=He%2C%26%2332%3BJun&rft.au=Li%2C%26%2332%3BZhi&rft.au=Cheng%2C%26%2332%3BWei&rft.date=2019&rft.volume=83&rft.pages=345%E2%80%93350&rft_id=info:doi\/10.1016%2Fj.procir.2019.04.084&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2212827119306985&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-32\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-32\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Segovia, Mariana; Garcia-Alfaro, Joaquin (20 July 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.mdpi.com\/1424-8220\/22\/14\/5396\" target=\"_blank\">\"Design, Modeling and Implementation of Digital Twins\"<\/a> (in en). <i>Sensors<\/i> <b>22<\/b> (14): 5396. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3390%2Fs22145396\" target=\"_blank\">10.3390\/s22145396<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1424-8220\" target=\"_blank\">1424-8220<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC9318241\/\" target=\"_blank\">PMC9318241<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/35891076\" target=\"_blank\">35891076<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.mdpi.com\/1424-8220\/22\/14\/5396\" target=\"_blank\">https:\/\/www.mdpi.com\/1424-8220\/22\/14\/5396<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Design%2C+Modeling+and+Implementation+of+Digital+Twins&rft.jtitle=Sensors&rft.aulast=Segovia&rft.aufirst=Mariana&rft.au=Segovia%2C%26%2332%3BMariana&rft.au=Garcia-Alfaro%2C%26%2332%3BJoaquin&rft.date=20+July+2022&rft.volume=22&rft.issue=14&rft.pages=5396&rft_id=info:doi\/10.3390%2Fs22145396&rft.issn=1424-8220&rft_id=info:pmc\/PMC9318241&rft_id=info:pmid\/35891076&rft_id=https%3A%2F%2Fwww.mdpi.com%2F1424-8220%2F22%2F14%2F5396&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-33\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-33\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Lober, Andreas; Lehmann, Joel; Hausermann, Tim; Reichwald, Julian; Baumgartel, Hartwig (22 November 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9965259\/\" target=\"_blank\">\"Improving the Engineering Process of Control Systems Based on Digital Twin Specifications\"<\/a>. <i>2022 4th International Conference on Emerging Trends in Electrical, Electronic and Communications Engineering (ELECOM)<\/i> (Mauritius: IEEE): 1\u20136. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FELECOM54934.2022.9965259\" target=\"_blank\">10.1109\/ELECOM54934.2022.9965259<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-6654-6697-4<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9965259\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/9965259\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Improving+the+Engineering+Process+of+Control+Systems+Based+on+Digital+Twin+Specifications&rft.jtitle=2022+4th+International+Conference+on+Emerging+Trends+in+Electrical%2C+Electronic+and+Communications+Engineering+%28ELECOM%29&rft.aulast=Lober&rft.aufirst=Andreas&rft.au=Lober%2C%26%2332%3BAndreas&rft.au=Lehmann%2C%26%2332%3BJoel&rft.au=Hausermann%2C%26%2332%3BTim&rft.au=Reichwald%2C%26%2332%3BJulian&rft.au=Baumgartel%2C%26%2332%3BHartwig&rft.date=22+November+2022&rft.pages=1%E2%80%936&rft.place=Mauritius&rft.pub=IEEE&rft_id=info:doi\/10.1109%2FELECOM54934.2022.9965259&rft.isbn=978-1-6654-6697-4&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F9965259%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-34\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-34\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Onaji, Igiri; Tiwari, Divya; Soulatiantork, Payam; Song, Boyang; Tiwari, Ashutosh (3 August 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/0951192X.2022.2027014\" target=\"_blank\">\"Digital twin in manufacturing: conceptual framework and case studies\"<\/a> (in en). <i>International Journal of Computer Integrated Manufacturing<\/i> <b>35<\/b> (8): 831\u2013858. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1080%2F0951192X.2022.2027014\" target=\"_blank\">10.1080\/0951192X.2022.2027014<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0951-192X\" target=\"_blank\">0951-192X<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/0951192X.2022.2027014\" target=\"_blank\">https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/0951192X.2022.2027014<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Digital+twin+in+manufacturing%3A+conceptual+framework+and+case+studies&rft.jtitle=International+Journal+of+Computer+Integrated+Manufacturing&rft.aulast=Onaji&rft.aufirst=Igiri&rft.au=Onaji%2C%26%2332%3BIgiri&rft.au=Tiwari%2C%26%2332%3BDivya&rft.au=Soulatiantork%2C%26%2332%3BPayam&rft.au=Song%2C%26%2332%3BBoyang&rft.au=Tiwari%2C%26%2332%3BAshutosh&rft.date=3+August+2022&rft.volume=35&rft.issue=8&rft.pages=831%E2%80%93858&rft_id=info:doi\/10.1080%2F0951192X.2022.2027014&rft.issn=0951-192X&rft_id=https%3A%2F%2Fwww.tandfonline.com%2Fdoi%2Ffull%2F10.1080%2F0951192X.2022.2027014&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-35\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-35\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Blinowski, Grzegorz; Ojdowska, Anna; Przybylek, Adam (2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9717259\/\" target=\"_blank\">\"Monolithic vs. Microservice Architecture: A Performance and Scalability Evaluation\"<\/a>. <i>IEEE Access<\/i> <b>10<\/b>: 20357\u201320374. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FACCESS.2022.3152803\" target=\"_blank\">10.1109\/ACCESS.2022.3152803<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2169-3536\" target=\"_blank\">2169-3536<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9717259\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/9717259\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Monolithic+vs.+Microservice+Architecture%3A+A+Performance+and+Scalability+Evaluation&rft.jtitle=IEEE+Access&rft.aulast=Blinowski&rft.aufirst=Grzegorz&rft.au=Blinowski%2C%26%2332%3BGrzegorz&rft.au=Ojdowska%2C%26%2332%3BAnna&rft.au=Przybylek%2C%26%2332%3BAdam&rft.date=2022&rft.volume=10&rft.pages=20357%E2%80%9320374&rft_id=info:doi\/10.1109%2FACCESS.2022.3152803&rft.issn=2169-3536&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F9717259%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:12-36\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:12_36-0\">36.0<\/a><\/sup> <sup><a href=\"#cite_ref-:12_36-1\">36.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">K\u00fcmmel, Tim; van Marwick, Bj\u00f6rn; Rittel, Miriam; Ramallo Guevara, Carina; W\u00fchler, Felix; Teumer, Tobias; W\u00e4ngler, Bj\u00f6rn; Hopf, Carsten <i>et al.<\/i> (28 May 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41598-021-90777-4\" target=\"_blank\">\"Rapid brain structure and tumour margin detection on whole frozen tissue sections by fast multiphotometric mid-infrared scanning\"<\/a> (in en). <i>Scientific Reports<\/i> <b>11<\/b> (1): 11307. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41598-021-90777-4\" target=\"_blank\">10.1038\/s41598-021-90777-4<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2045-2322\" target=\"_blank\">2045-2322<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC8163866\/\" target=\"_blank\">PMC8163866<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/34050224\" target=\"_blank\">34050224<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41598-021-90777-4\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41598-021-90777-4<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Rapid+brain+structure+and+tumour+margin+detection+on+whole+frozen+tissue+sections+by+fast+multiphotometric+mid-infrared+scanning&rft.jtitle=Scientific+Reports&rft.aulast=K%C3%BCmmel&rft.aufirst=Tim&rft.au=K%C3%BCmmel%2C%26%2332%3BTim&rft.au=van+Marwick%2C%26%2332%3BBj%C3%B6rn&rft.au=Rittel%2C%26%2332%3BMiriam&rft.au=Ramallo+Guevara%2C%26%2332%3BCarina&rft.au=W%C3%BChler%2C%26%2332%3BFelix&rft.au=Teumer%2C%26%2332%3BTobias&rft.au=W%C3%A4ngler%2C%26%2332%3BBj%C3%B6rn&rft.au=Hopf%2C%26%2332%3BCarsten&rft.au=R%C3%A4dle%2C%26%2332%3BMatthias&rft.date=28+May+2021&rft.volume=11&rft.issue=1&rft.pages=11307&rft_id=info:doi\/10.1038%2Fs41598-021-90777-4&rft.issn=2045-2322&rft_id=info:pmc\/PMC8163866&rft_id=info:pmid\/34050224&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41598-021-90777-4&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:13-37\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:13_37-0\">37.0<\/a><\/sup> <sup><a href=\"#cite_ref-:13_37-1\">37.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Heintz, Annabell; Sold, Sebastian; W\u00fchler, Felix; Dyckow, Julia; Schirmer, Lucas; Beuermann, Thomas; R\u00e4dle, Matthias (23 May 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.mdpi.com\/2076-3417\/11\/11\/4777\" target=\"_blank\">\"Design of a Multimodal Imaging System and Its First Application to Distinguish Grey and White Matter of Brain Tissue. A Proof-of-Concept-Study\"<\/a> (in en). <i>Applied Sciences<\/i> <b>11<\/b> (11): 4777. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3390%2Fapp11114777\" target=\"_blank\">10.3390\/app11114777<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2076-3417\" target=\"_blank\">2076-3417<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.mdpi.com\/2076-3417\/11\/11\/4777\" target=\"_blank\">https:\/\/www.mdpi.com\/2076-3417\/11\/11\/4777<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Design+of+a+Multimodal+Imaging+System+and+Its+First+Application+to+Distinguish+Grey+and+White+Matter+of+Brain+Tissue.+A+Proof-of-Concept-Study&rft.jtitle=Applied+Sciences&rft.aulast=Heintz&rft.aufirst=Annabell&rft.au=Heintz%2C%26%2332%3BAnnabell&rft.au=Sold%2C%26%2332%3BSebastian&rft.au=W%C3%BChler%2C%26%2332%3BFelix&rft.au=Dyckow%2C%26%2332%3BJulia&rft.au=Schirmer%2C%26%2332%3BLucas&rft.au=Beuermann%2C%26%2332%3BThomas&rft.au=R%C3%A4dle%2C%26%2332%3BMatthias&rft.date=23+May+2021&rft.volume=11&rft.issue=11&rft.pages=4777&rft_id=info:doi\/10.3390%2Fapp11114777&rft.issn=2076-3417&rft_id=https%3A%2F%2Fwww.mdpi.com%2F2076-3417%2F11%2F11%2F4777&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-38\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-38\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.pjrc.com\/teensy\/IMXRT1060CEC_rev0_1.pdf\" target=\"_blank\">\"i.MX RT1060 Crossover Processors for Consumer Products\"<\/a> (PDF). NXP B.V. April 2019<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.pjrc.com\/teensy\/IMXRT1060CEC_rev0_1.pdf\" target=\"_blank\">https:\/\/www.pjrc.com\/teensy\/IMXRT1060CEC_rev0_1.pdf<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 08 December 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=i.MX+RT1060+Crossover+Processors+for+Consumer+Products&rft.atitle=&rft.date=April+2019&rft.pub=NXP+B.V&rft_id=https%3A%2F%2Fwww.pjrc.com%2Fteensy%2FIMXRT1060CEC_rev0_1.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-39\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-39\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ams.com\/as89010\" target=\"_blank\">\"AS89010 Current-to-Digital Converter\"<\/a>. ams-OSRAM AG. 2017<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ams.com\/as89010\" target=\"_blank\">https:\/\/ams.com\/as89010<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 06 December 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=AS89010+Current-to-Digital+Converter&rft.atitle=&rft.date=2017&rft.pub=ams-OSRAM+AG&rft_id=https%3A%2F%2Fams.com%2Fas89010&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-40\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-40\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.ti.com\/product\/DP83825I\" target=\"_blank\">\"DP83825I: Smallest form factor (3-mm by 3-mm), low-power 10\/100-Mbps Ethernet PHY transceiver with 50-MHz c\"<\/a>. Texas Instruments. August 2019<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.ti.com\/product\/DP83825I\" target=\"_blank\">https:\/\/www.ti.com\/product\/DP83825I<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 30 November 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=DP83825I%3A+Smallest+form+factor+%283-mm+by+3-mm%29%2C+low-power+10%2F100-Mbps+Ethernet+PHY+transceiver+with+50-MHz+c&rft.atitle=&rft.date=August+2019&rft.pub=Texas+Instruments&rft_id=https%3A%2F%2Fwww.ti.com%2Fproduct%2FDP83825I&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:14-41\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:14_41-0\">41.0<\/a><\/sup> <sup><a href=\"#cite_ref-:14_41-1\">41.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Schmitt, Lukas; Meyer, Conrad; Schorz, Stefan; Manser, Steffen; Scholl, Stephan; R\u00e4dle, and Matthias (1 August 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/cite.202200076\" target=\"_blank\">\"Use of a Scattered Light Sensor for Monitoring the Dispersed Surface in Crystallization\"<\/a> (in de). <i>Chemie Ingenieur Technik<\/i> <b>94<\/b> (8): 1177\u20131184. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1002%2Fcite.202200076\" target=\"_blank\">10.1002\/cite.202200076<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0009-286X\" target=\"_blank\">0009-286X<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/cite.202200076\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/cite.202200076<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Use+of+a+Scattered+Light+Sensor+for+Monitoring+the+Dispersed+Surface+in+Crystallization&rft.jtitle=Chemie+Ingenieur+Technik&rft.aulast=Schmitt&rft.aufirst=Lukas&rft.au=Schmitt%2C%26%2332%3BLukas&rft.au=Meyer%2C%26%2332%3BConrad&rft.au=Schorz%2C%26%2332%3BStefan&rft.au=Manser%2C%26%2332%3BSteffen&rft.au=Scholl%2C%26%2332%3BStephan&rft.au=R%C3%A4dle%2C%26%2332%3Band+Matthias&rft.date=1+August+2022&rft.volume=94&rft.issue=8&rft.pages=1177%E2%80%931184&rft_id=info:doi\/10.1002%2Fcite.202200076&rft.issn=0009-286X&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1002%2Fcite.202200076&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:15-42\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:15_42-0\">42.0<\/a><\/sup> <sup><a href=\"#cite_ref-:15_42-1\">42.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Teumer, Tobias; Medina, Isabel; Strischakov, Johann; Schorz, Stefan; Kumari, Pooja; Hohlen, Annika; Scholl, Stephan; Schwede, Christian <i>et al.<\/i> (2021) (in en). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/rgdoi.net\/10.13140\/RG.2.2.25284.55681\" target=\"_blank\"><i>Development and application of optical sensors and measurement devices for the detection of deposits during reaction fouling 13th ECCE and 6th ECAB -post 31762<\/i><\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.13140%2FRG.2.2.25284.55681\" target=\"_blank\">10.13140\/RG.2.2.25284.55681<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/rgdoi.net\/10.13140\/RG.2.2.25284.55681\" target=\"_blank\">http:\/\/rgdoi.net\/10.13140\/RG.2.2.25284.55681<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=Development+and+application+of+optical+sensors+and+measurement+devices+for+the+detection+of+deposits+during+reaction+fouling+13th+ECCE+and+6th+ECAB+-post+31762&rft.aulast=Teumer&rft.aufirst=Tobias&rft.au=Teumer%2C%26%2332%3BTobias&rft.au=Medina%2C%26%2332%3BIsabel&rft.au=Strischakov%2C%26%2332%3BJohann&rft.au=Schorz%2C%26%2332%3BStefan&rft.au=Kumari%2C%26%2332%3BPooja&rft.au=Hohlen%2C%26%2332%3BAnnika&rft.au=Scholl%2C%26%2332%3BStephan&rft.au=Schwede%2C%26%2332%3BChristian&rft.au=Melchin%2C%26%2332%3BTimo&rft.date=2021&rft_id=info:doi\/10.13140%2FRG.2.2.25284.55681&rft_id=http%3A%2F%2Frgdoi.net%2F10.13140%2FRG.2.2.25284.55681&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:16-43\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:16_43-0\">43.0<\/a><\/sup> <sup><a href=\"#cite_ref-:16_43-1\">43.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Guffart, Julia; Bus, Yannick; Nachtmann, Marcel; Lettau, Markus; Schorz, Stefan; Nieder, Helmut; Repke, Jens\u2010Uwe; R\u00e4dle, Matthias (1 June 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/cite.201900186\" target=\"_blank\">\"Photometric Inline Monitoring of Pigment Concentration in Highly Filled Lacquers\"<\/a> (in en). <i>Chemie Ingenieur Technik<\/i> <b>92<\/b> (6): 729\u2013735. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1002%2Fcite.201900186\" target=\"_blank\">10.1002\/cite.201900186<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0009-286X\" target=\"_blank\">0009-286X<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/cite.201900186\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/cite.201900186<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Photometric+Inline+Monitoring+of+Pigment+Concentration+in+Highly+Filled+Lacquers&rft.jtitle=Chemie+Ingenieur+Technik&rft.aulast=Guffart&rft.aufirst=Julia&rft.au=Guffart%2C%26%2332%3BJulia&rft.au=Bus%2C%26%2332%3BYannick&rft.au=Nachtmann%2C%26%2332%3BMarcel&rft.au=Lettau%2C%26%2332%3BMarkus&rft.au=Schorz%2C%26%2332%3BStefan&rft.au=Nieder%2C%26%2332%3BHelmut&rft.au=Repke%2C%26%2332%3BJens%E2%80%90Uwe&rft.au=R%C3%A4dle%2C%26%2332%3BMatthias&rft.date=1+June+2020&rft.volume=92&rft.issue=6&rft.pages=729%E2%80%93735&rft_id=info:doi\/10.1002%2Fcite.201900186&rft.issn=0009-286X&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1002%2Fcite.201900186&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-44\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-44\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/eclipse.dev\/ditto\/intro-overview.html\" target=\"_blank\">\"Eclipse Ditto documentation overview\"<\/a>. <i>Eclipse Ditto Documentation<\/i>. Eclipse Foundation<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/eclipse.dev\/ditto\/intro-overview.html\" target=\"_blank\">https:\/\/eclipse.dev\/ditto\/intro-overview.html<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 09 November 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Eclipse+Ditto+documentation+overview&rft.atitle=Eclipse+Ditto+Documentation&rft.pub=Eclipse+Foundation&rft_id=https%3A%2F%2Feclipse.dev%2Fditto%2Fintro-overview.html&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-45\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-45\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation\" id=\"rdp-ebb-CITEREFNguyenDo2021\">Nguyen, Dat Tien; Do, Hao Duc (2021), Tran, Duc-Tan; Jeon, Gwanggil; Nguyen, Thi Dieu Linh <i>et al.<\/i>., eds., <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/link.springer.com\/10.1007\/978-981-16-2094-2_11\" target=\"_blank\">\"Research on Large-Scale Knowledge Base Management Frameworks for Open-Domain Question Answering Systems\"<\/a> (in en), <i>Intelligent Systems and Networks<\/i> (Singapore: Springer Singapore) <b>243<\/b>: 87\u201392, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2F978-981-16-2094-2_11\" target=\"_blank\">10.1007\/978-981-16-2094-2_11<\/a>, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-981-16-2093-5<span class=\"printonly\">, <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/link.springer.com\/10.1007\/978-981-16-2094-2_11\" target=\"_blank\">https:\/\/link.springer.com\/10.1007\/978-981-16-2094-2_11<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 2023-09-12<\/span><\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Research+on+Large-Scale+Knowledge+Base+Management+Frameworks+for+Open-Domain+Question+Answering+Systems&rft.jtitle=Intelligent+Systems+and+Networks&rft.aulast=Nguyen&rft.aufirst=Dat+Tien&rft.au=Nguyen%2C%26%2332%3BDat+Tien&rft.au=Do%2C%26%2332%3BHao+Duc&rft.date=2021&rft.volume=243&rft.pages=87%E2%80%9392&rft.place=Singapore&rft.pub=Springer+Singapore&rft_id=info:doi\/10.1007%2F978-981-16-2094-2_11&rft.isbn=978-981-16-2093-5&rft_id=https%3A%2F%2Flink.springer.com%2F10.1007%2F978-981-16-2094-2_11&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-46\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-46\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Hong, Seokyong; Lee, Sangkeun; Lim, Seung-Hwan; Sukumar, Sreenivas R.; Vatsavai, Ranga Raju (31 May 2016). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/dl.acm.org\/doi\/10.1145\/2907294.2907305\" target=\"_blank\">\"Evaluation of Pattern Matching Workloads in Graph Analysis Systems\"<\/a> (in en). <i>Proceedings of the 25th ACM International Symposium on High-Performance Parallel and Distributed Computing<\/i> (Kyoto Japan: ACM): 263\u2013266. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1145%2F2907294.2907305\" target=\"_blank\">10.1145\/2907294.2907305<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-4503-4314-5<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/dl.acm.org\/doi\/10.1145\/2907294.2907305\" target=\"_blank\">https:\/\/dl.acm.org\/doi\/10.1145\/2907294.2907305<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Evaluation+of+Pattern+Matching+Workloads+in+Graph+Analysis+Systems&rft.jtitle=Proceedings+of+the+25th+ACM+International+Symposium+on+High-Performance+Parallel+and+Distributed+Computing&rft.aulast=Hong&rft.aufirst=Seokyong&rft.au=Hong%2C%26%2332%3BSeokyong&rft.au=Lee%2C%26%2332%3BSangkeun&rft.au=Lim%2C%26%2332%3BSeung-Hwan&rft.au=Sukumar%2C%26%2332%3BSreenivas+R.&rft.au=Vatsavai%2C%26%2332%3BRanga+Raju&rft.date=31+May+2016&rft.pages=263%E2%80%93266&rft.place=Kyoto+Japan&rft.pub=ACM&rft_id=info:doi\/10.1145%2F2907294.2907305&rft.isbn=978-1-4503-4314-5&rft_id=https%3A%2F%2Fdl.acm.org%2Fdoi%2F10.1145%2F2907294.2907305&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-47\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-47\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Lissandrini, Matteo; Brugnara, Martin; Velegrakis, Yannis (1 December 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/dl.acm.org\/doi\/10.14778\/3297753.3297759\" target=\"_blank\">\"Beyond macrobenchmarks: microbenchmark-based graph database evaluation\"<\/a> (in en). <i>Proceedings of the VLDB Endowment<\/i> <b>12<\/b> (4): 390\u2013403. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.14778%2F3297753.3297759\" target=\"_blank\">10.14778\/3297753.3297759<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2150-8097\" target=\"_blank\">2150-8097<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/dl.acm.org\/doi\/10.14778\/3297753.3297759\" target=\"_blank\">https:\/\/dl.acm.org\/doi\/10.14778\/3297753.3297759<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Beyond+macrobenchmarks%3A+microbenchmark-based+graph+database+evaluation&rft.jtitle=Proceedings+of+the+VLDB+Endowment&rft.aulast=Lissandrini&rft.aufirst=Matteo&rft.au=Lissandrini%2C%26%2332%3BMatteo&rft.au=Brugnara%2C%26%2332%3BMartin&rft.au=Velegrakis%2C%26%2332%3BYannis&rft.date=1+December+2018&rft.volume=12&rft.issue=4&rft.pages=390%E2%80%93403&rft_id=info:doi\/10.14778%2F3297753.3297759&rft.issn=2150-8097&rft_id=https%3A%2F%2Fdl.acm.org%2Fdoi%2F10.14778%2F3297753.3297759&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-48\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-48\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Vrgoc, Domagoj; Rojas, Carlos; Angles, Renzo; Arenas, Marcelo; Arroyuelo, Diego; Aranda, Carlos Buil; Hogan, Aidan; Navarro, Gonzalo <i>et al.<\/i> (2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/arxiv.org\/abs\/2111.01540\" target=\"_blank\"><i>MillenniumDB: A Persistent, Open-Source, Graph Database<\/i><\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.48550%2FARXIV.2111.01540\" target=\"_blank\">10.48550\/ARXIV.2111.01540<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/arxiv.org\/abs\/2111.01540\" target=\"_blank\">https:\/\/arxiv.org\/abs\/2111.01540<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=MillenniumDB%3A+A+Persistent%2C+Open-Source%2C+Graph+Database&rft.aulast=Vrgoc&rft.aufirst=Domagoj&rft.au=Vrgoc%2C%26%2332%3BDomagoj&rft.au=Rojas%2C%26%2332%3BCarlos&rft.au=Angles%2C%26%2332%3BRenzo&rft.au=Arenas%2C%26%2332%3BMarcelo&rft.au=Arroyuelo%2C%26%2332%3BDiego&rft.au=Aranda%2C%26%2332%3BCarlos+Buil&rft.au=Hogan%2C%26%2332%3BAidan&rft.au=Navarro%2C%26%2332%3BGonzalo&rft.au=Riveros%2C%26%2332%3BCristian&rft.date=2021&rft_id=info:doi\/10.48550%2FARXIV.2111.01540&rft_id=https%3A%2F%2Farxiv.org%2Fabs%2F2111.01540&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-49\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-49\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/jena.apache.org\/\" target=\"_blank\">\"Apache Jena\"<\/a>. The Apache Software Foundation<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/jena.apache.org\/\" target=\"_blank\">https:\/\/jena.apache.org\/<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 22 November 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Apache+Jena&rft.atitle=&rft.pub=The+Apache+Software+Foundation&rft_id=https%3A%2F%2Fjena.apache.org%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-50\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-50\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/docs.influxdata.com\/influxdb\/v2.5\/\" target=\"_blank\">\"Get started with InfluxDB OSS 2.5\"<\/a>. <i>InfluxData Documentation<\/i>. InfluxData. 2023<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/docs.influxdata.com\/influxdb\/v2.5\/\" target=\"_blank\">https:\/\/docs.influxdata.com\/influxdb\/v2.5\/<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 22 November 2023<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Get+started+with+InfluxDB+OSS+2.5&rft.atitle=InfluxData+Documentation&rft.date=2023&rft.pub=InfluxData&rft_id=https%3A%2F%2Fdocs.influxdata.com%2Finfluxdb%2Fv2.5%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:17-51\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:17_51-0\">51.0<\/a><\/sup> <sup><a href=\"#cite_ref-:17_51-1\">51.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Hao, Yuanzhe; Qin, Xiongpai; Chen, Yueguo; Li, Yaru; Sun, Xiaoguang; Tao, Yu; Zhang, Xiao; Du, Xiaoyong (1 April 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9458659\/\" target=\"_blank\">\"TS-Benchmark: A Benchmark for Time Series Databases\"<\/a>. <i>2021 IEEE 37th International Conference on Data Engineering (ICDE)<\/i> (Chania, Greece: IEEE): 588\u2013599. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FICDE51399.2021.00057\" target=\"_blank\">10.1109\/ICDE51399.2021.00057<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-7281-9184-3<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9458659\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/9458659\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=TS-Benchmark%3A+A+Benchmark+for+Time+Series+Databases&rft.jtitle=2021+IEEE+37th+International+Conference+on+Data+Engineering+%28ICDE%29&rft.aulast=Hao&rft.aufirst=Yuanzhe&rft.au=Hao%2C%26%2332%3BYuanzhe&rft.au=Qin%2C%26%2332%3BXiongpai&rft.au=Chen%2C%26%2332%3BYueguo&rft.au=Li%2C%26%2332%3BYaru&rft.au=Sun%2C%26%2332%3BXiaoguang&rft.au=Tao%2C%26%2332%3BYu&rft.au=Zhang%2C%26%2332%3BXiao&rft.au=Du%2C%26%2332%3BXiaoyong&rft.date=1+April+2021&rft.pages=588%E2%80%93599&rft.place=Chania%2C+Greece&rft.pub=IEEE&rft_id=info:doi\/10.1109%2FICDE51399.2021.00057&rft.isbn=978-1-7281-9184-3&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F9458659%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-52\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-52\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Department of Computing and Informatics, Mazoon College, Muscat, Sultanate of Oman.; Nasar, Mohammad; Kausar, Mohammad Abu; Department of Information Systems, University of Nizwa, Nizwa, Sultanate of Oman. (30 August 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.ijitee.org\/portfolio-item\/J92250881019\/\" target=\"_blank\">\"Suitability Of Influxdb Database For Iot Applications\"<\/a>. <i>International Journal of Innovative Technology and Exploring Engineering<\/i> <b>8<\/b> (10): 1850\u20131857. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.35940%2Fijitee.J9225.0881019\" target=\"_blank\">10.35940\/ijitee.J9225.0881019<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.ijitee.org\/portfolio-item\/J92250881019\/\" target=\"_blank\">https:\/\/www.ijitee.org\/portfolio-item\/J92250881019\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Suitability+Of+Influxdb+Database+For+Iot+Applications&rft.jtitle=International+Journal+of+Innovative+Technology+and+Exploring+Engineering&rft.aulast=Department+of+Computing+and+Informatics%2C+Mazoon+College%2C+Muscat%2C+Sultanate+of+Oman.&rft.au=Department+of+Computing+and+Informatics%2C+Mazoon+College%2C+Muscat%2C+Sultanate+of+Oman.&rft.au=Nasar%2C%26%2332%3BMohammad&rft.au=Kausar%2C%26%2332%3BMohammad+Abu&rft.au=Department+of+Information+Systems%2C+University+of+Nizwa%2C+Nizwa%2C+Sultanate+of+Oman.&rft.date=30+August+2019&rft.volume=8&rft.issue=10&rft.pages=1850%E2%80%931857&rft_id=info:doi\/10.35940%2Fijitee.J9225.0881019&rft_id=https%3A%2F%2Fwww.ijitee.org%2Fportfolio-item%2FJ92250881019%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-53\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-53\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/guides.dataverse.org\/en\/latest\/\" target=\"_blank\">\"Dataverse Documentation v. 6.0\"<\/a>. <i>Dataverse Guides<\/i>. Dataverse Project. 2022<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/guides.dataverse.org\/en\/latest\/\" target=\"_blank\">https:\/\/guides.dataverse.org\/en\/latest\/<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 22 November 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Dataverse+Documentation+v.+6.0&rft.atitle=Dataverse+Guides&rft.date=2022&rft.pub=Dataverse+Project&rft_id=https%3A%2F%2Fguides.dataverse.org%2Fen%2Flatest%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-54\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-54\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Stall, Shelley; Martone, Maryann E.; Chandramouliswaran, Ishwar; Crosas, Merc\u00e8; Federer, Lisa; Gautier, Julian; Hahnel, Mark; Larkin, Jennie <i>et al.<\/i> (15 July 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/zenodo.org\/record\/3946720\" target=\"_blank\"><i>Generalist Repository Comparison Chart<\/i><\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.5281%2FZENODO.3946720\" target=\"_blank\">10.5281\/ZENODO.3946720<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/zenodo.org\/record\/3946720\" target=\"_blank\">https:\/\/zenodo.org\/record\/3946720<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=Generalist+Repository+Comparison+Chart&rft.aulast=Stall%2C+Shelley&rft.au=Stall%2C+Shelley&rft.au=Martone%2C+Maryann+E.&rft.au=Chandramouliswaran%2C+Ishwar&rft.au=Crosas%2C+Merc%C3%A8&rft.au=Federer%2C+Lisa&rft.au=Gautier%2C+Julian&rft.au=Hahnel%2C+Mark&rft.au=Larkin%2C+Jennie&rft.au=Lowenberg%2C+Daniella&rft.date=15+July+2020&rft_id=info:doi\/10.5281%2FZENODO.3946720&rft_id=https%3A%2F%2Fzenodo.org%2Frecord%2F3946720&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-55\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-55\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Wittenberg, Marion; Tykhonov, Vyacheslav; Indarto, Eko; Steinhoff, Wilko; Veld, Laura Huis In 'T; Kasberger, Stefan; Conzett, Philipp; Concordia, Cesare <i>et al.<\/i> (31 March 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/zenodo.org\/record\/6676391\" target=\"_blank\"><i>D5.5 'Archive in a Box' repository software and proof of concept of centralised installation in the cloud<\/i><\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.5281%2FZENODO.6676391\" target=\"_blank\">10.5281\/ZENODO.6676391<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/zenodo.org\/record\/6676391\" target=\"_blank\">https:\/\/zenodo.org\/record\/6676391<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=D5.5+%27Archive+in+a+Box%27+repository+software+and+proof+of+concept+of+centralised+installation+in+the+cloud&rft.aulast=Wittenberg&rft.aufirst=Marion&rft.au=Wittenberg%2C%26%2332%3BMarion&rft.au=Tykhonov%2C%26%2332%3BVyacheslav&rft.au=Indarto%2C%26%2332%3BEko&rft.au=Steinhoff%2C%26%2332%3BWilko&rft.au=Veld%2C%26%2332%3BLaura+Huis+In+%27T&rft.au=Kasberger%2C%26%2332%3BStefan&rft.au=Conzett%2C%26%2332%3BPhilipp&rft.au=Concordia%2C%26%2332%3BCesare&rft.au=Kiraly%2C%26%2332%3BPeter&rft.date=31+March+2022&rft_id=info:doi\/10.5281%2FZENODO.6676391&rft_id=https%3A%2F%2Fzenodo.org%2Frecord%2F6676391&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-56\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-56\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Light, Roger A (26 May 2017). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/joss.theoj.org\/papers\/10.21105\/joss.00265\" target=\"_blank\">\"Mosquitto: server and client implementation of the MQTT protocol\"<\/a>. <i>The Journal of Open Source Software<\/i> <b>2<\/b> (13): 265. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.21105%2Fjoss.00265\" target=\"_blank\">10.21105\/joss.00265<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2475-9066\" target=\"_blank\">2475-9066<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/joss.theoj.org\/papers\/10.21105\/joss.00265\" target=\"_blank\">http:\/\/joss.theoj.org\/papers\/10.21105\/joss.00265<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Mosquitto%3A+server+and+client+implementation+of+the+MQTT+protocol&rft.jtitle=The+Journal+of+Open+Source+Software&rft.aulast=Light&rft.aufirst=Roger+A&rft.au=Light%2C%26%2332%3BRoger+A&rft.date=26+May+2017&rft.volume=2&rft.issue=13&rft.pages=265&rft_id=info:doi\/10.21105%2Fjoss.00265&rft.issn=2475-9066&rft_id=http%3A%2F%2Fjoss.theoj.org%2Fpapers%2F10.21105%2Fjoss.00265&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-57\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-57\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Uy, Nguyen Quoc; Nam, Vu Hoai (1 December 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9023812\/\" target=\"_blank\">\"A comparison of AMQP and MQTT protocols for Internet of Things\"<\/a>. <i>2019 6th NAFOSTED Conference on Information and Computer Science (NICS)<\/i> (Hanoi, Vietnam: IEEE): 292\u2013297. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FNICS48868.2019.9023812\" target=\"_blank\">10.1109\/NICS48868.2019.9023812<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-7281-5163-2<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9023812\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/9023812\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+comparison+of+AMQP+and+MQTT+protocols+for+Internet+of+Things&rft.jtitle=2019+6th+NAFOSTED+Conference+on+Information+and+Computer+Science+%28NICS%29&rft.aulast=Uy&rft.aufirst=Nguyen+Quoc&rft.au=Uy%2C%26%2332%3BNguyen+Quoc&rft.au=Nam%2C%26%2332%3BVu+Hoai&rft.date=1+December+2019&rft.pages=292%E2%80%93297&rft.place=Hanoi%2C+Vietnam&rft.pub=IEEE&rft_id=info:doi\/10.1109%2FNICS48868.2019.9023812&rft.isbn=978-1-7281-5163-2&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F9023812%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-58\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-58\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/nodered.org\/docs\/\" target=\"_blank\">\"Node-RED Documentation\"<\/a>. <i>Node-RED<\/i>. OpenJS Foundation<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/nodered.org\/docs\/\" target=\"_blank\">https:\/\/nodered.org\/docs\/<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 23 November 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Node-RED+Documentation&rft.atitle=Node-RED&rft.pub=OpenJS+Foundation&rft_id=https%3A%2F%2Fnodered.org%2Fdocs%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-59\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-59\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pdf1.alldatasheet.com\/datasheet-pdf\/view\/332642\/EPIGAP\/EPD-660-1-0.9.html\" target=\"_blank\">\"EPD-660-1-0.9 Datasheet (PDF) - EPIGAP optoelectronic GmbH\"<\/a>. Optoelektronik GmbH. 16 May 2022<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pdf1.alldatasheet.com\/datasheet-pdf\/view\/332642\/EPIGAP\/EPD-660-1-0.9.html\" target=\"_blank\">https:\/\/pdf1.alldatasheet.com\/datasheet-pdf\/view\/332642\/EPIGAP\/EPD-660-1-0.9.html<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 08 December 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=EPD-660-1-0.9+Datasheet+%28PDF%29+-+EPIGAP+optoelectronic+GmbH&rft.atitle=&rft.date=16+May+2022&rft.pub=Optoelektronik+GmbH&rft_id=https%3A%2F%2Fpdf1.alldatasheet.com%2Fdatasheet-pdf%2Fview%2F332642%2FEPIGAP%2FEPD-660-1-0.9.html&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-60\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-60\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.roithner-laser.com\/datasheets\/led_div\/eld_650_523.pdf\" target=\"_blank\">\"ELD-650-523\"<\/a> (PDF). Roithner LaserTechnik<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.roithner-laser.com\/datasheets\/led_div\/eld_650_523.pdf\" target=\"_blank\">http:\/\/www.roithner-laser.com\/datasheets\/led_div\/eld_650_523.pdf<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 08 December 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=ELD-650-523&rft.atitle=&rft.pub=Roithner+LaserTechnik&rft_id=http%3A%2F%2Fwww.roithner-laser.com%2Fdatasheets%2Fled_div%2Feld_650_523.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<\/ol><\/div><\/div>\n<h2><span class=\"mw-headline\" id=\"Notes\">Notes<\/span><\/h2>\n<p>This presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added.\n<\/p>\n<!-- \nNewPP limit report\nCached time: 20231215184823\nCache expiry: 86400\nDynamic content: false\nComplications: []\nCPU time usage: 1.200 seconds\nReal time usage: 1.536 seconds\nPreprocessor visited node count: 57622\/1000000\nPost\u2010expand include size: 479251\/2097152 bytes\nTemplate argument size: 144750\/2097152 bytes\nHighest expansion depth: 25\/40\nExpensive parser function count: 0\/100\nUnstrip recursion depth: 0\/20\nUnstrip post\u2010expand size: 137382\/5000000 bytes\n-->\n<!--\nTransclusion expansion time report (%,ms,calls,template)\n100.00% 943.373 1 -total\n 90.80% 856.600 1 Template:Reflist\n 73.06% 689.222 60 Template:Citation\/core\n 61.05% 575.928 43 Template:Cite_journal\n 12.08% 113.925 54 Template:Date\n 10.80% 101.920 12 Template:Cite_web\n 7.34% 69.277 91 Template:Citation\/identifier\n 5.05% 47.601 3 Template:Citation\n 3.51% 33.117 1 Template:Infobox_journal_article\n 3.07% 28.946 1 Template:Infobox\n-->\n\n<!-- Saved in parser cache with key limswiki:pcache:idhash:14377-0!canonical and timestamp 20231215184821 and revision id 52988. Serialized with JSON.\n -->\n<\/div><\/div><div class=\"printfooter\">Source: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.limswiki.org\/index.php\/Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins\">https:\/\/www.limswiki.org\/index.php\/Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins<\/a><\/div>\n<!-- end content --><div class=\"visualClear\"><\/div><\/div><\/div><div class=\"visualClear\"><\/div><\/div><!-- end of the left (by default at least) column --><div class=\"visualClear\"><\/div><\/div>\n\n\n\n<\/body>","9ace6d7c38d417b5bea5133d24ffe1a9_images":["https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/d\/de\/Fig1_Lehmann_Sensors23_23-1.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/a\/ac\/Fig2_Lehmann_Sensors23_23-1.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/b\/bd\/Fig3_Lehmann_Sensors23_23-1.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/5\/5e\/Fig4_Lehmann_Sensors23_23-1.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/0\/0b\/Fig5_Lehmann_Sensors23_23-1.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/d\/d0\/Fig6_Lehmann_Sensors23_23-1.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/b\/b4\/Fig7_Lehmann_Sensors23_23-1.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/b\/b0\/Fig8_Lehmann_Sensors23_23-1.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/a\/ae\/Fig9_Lehmann_Sensors23_23-1.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/0\/02\/FigA1_Lehmann_Sensors23_23-1.png"],"9ace6d7c38d417b5bea5133d24ffe1a9_timestamp":1702682172,"537370f6a0e7e5345701b0b5a2fb2d41_type":"article","537370f6a0e7e5345701b0b5a2fb2d41_title":"Autonomous experimental systems in materials science (Ishizuki et al. 2023)","537370f6a0e7e5345701b0b5a2fb2d41_url":"https:\/\/www.limswiki.org\/index.php\/Journal:Autonomous_experimental_systems_in_materials_science","537370f6a0e7e5345701b0b5a2fb2d41_plaintext":"\n\nJournal:Autonomous experimental systems in materials scienceFrom LIMSWikiJump to navigationJump to searchFull article title\n \nAutonomous experimental systems in materials scienceJournal\n \nScience and Technology of Advanced Materials: MethodsAuthor(s)\n \nIshizuki, Naoya; Shimizu, Ryota; Hitosugi, TaroAuthor affiliation(s)\n \nTokyo Institute of Technology, The University of TokyoPrimary contact\n \nEmail: hitosugi at g dot ecc dot u dash tokyo dot ac dot jpYear published\n \n2023Volume and issue\n \n3(1)Article #\n \n2197519DOI\n \n10.1080\/27660400.2023.2197519ISSN\n \n2766-0400Distribution license\n \nCreative Commons Attribution 4.0 InternationalWebsite\n \nhttps:\/\/www.tandfonline.com\/doi\/full\/10.1080\/27660400.2023.2197519Download\n \nhttps:\/\/www.tandfonline.com\/doi\/pdf\/10.1080\/27660400.2023.2197519 (PDF)\n\nContents \n\n1 Abstract \n2 Introduction \n3 Present status of autonomous experiments \n\n3.1 A brief description of our own autonomous experiments \n3.2 AES in the world \n\n3.2.1 Stage 1: Optimization of the yield of target substances or reaction conditions \n3.2.2 Stage 2: Finding new materials with desired properties \n3.2.3 Future of Stages 1 and 2: Autonomously search within the preset search space \n\n\n\n\n4 Stage 3: Finding materials that no one has thought of before \n\n4.1 Increasing the amount of reliable materials data (materials checkup system and measurement) \n4.2 Constructing scientific theories from a bird\u2019s-eye view \n\n\n5 Important points for the actual operation of an autonomous system \n\n5.1 Setting the right research topic \n5.2 Cost issues \n5.3 Change in the experimental sequences \n5.4 Steps for implementation and the rise of the lab-system integrator \n\n\n6 The autonomous system and intellectual property \n\n6.1 Inventorship \u2013 Who is an inventor? \n6.2 Inventive step \u2013 Is an invention easily invented? \n6.3 Authorship \u2013 Who is an author of academic papers? \n\n\n7 Summary \n8 Acknowledgements \n\n8.1 Funding \n8.2 Conflict of interest \n\n\n9 References \n10 Notes \n\n\n\nAbstract \nThe emergence of autonomous experimental systems (AESs) integrating machine learning (ML) and robots is ushering in a paradigm shift in materials science. Using computer algorithms and robots to decide and perform all experimental steps, these systems require no human intervention. A current direction focuses on discovering unexpected materials and theories with unconventional research approaches. This article reviews the latest achievements and discusses the impact of AESs, which will fundamentally change the way we understand research. Moreover, as AESs continue to develop, the need to think about the role of human researchers becomes more pressing. While ML and robotics can free us from the repetitive aspects of research, we need to understand the strengths and limitations of ML and robots and focus on how humans can perform higher creativity. In addition, we also discuss inventorship and authorship in the era of autonomous systems.\nKeywords: autonomous experimental system, closed-loop, machine learning, robots, materials science, inventorship, authorship, human researcher, human\u2019s role\nGraphic abstract: \n\n\nIntroduction \nThe total number of all possible small organic molecules is estimated to be at least 1060[1][2], and from that one can imagine a similarly large number of possible materials being derived using those molecules. This number helps illustrate the vastness of the materials search space, which must contain many materials that can help address current societal problems. In a way, the world of materials is a frontier for exploration, much like space or the deep sea.\nHow can we quickly and systematically find unexpected materials within this enormous search space? To this end, materials science needs a tool that can transcend the limits of human capabilities to serve as a materials explorer (Figure 1), akin to a spaceship or a deep-sea exploration vessel.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 1. The vision of the materials explorer. The exploration involves an autonomous experimental system (AES), materials informatics, and human researchers. The heart of the materials explorer is an AES based on machine learning and robots (green, orange, and blue). This system is imbued with the skills of experts and generates large amounts of experimental data that could not have been generated by human researchers (data-production factory). The data generated by the AES is then processed by machine learning (ML) and simulations to predict new materials (i.e., materials informatics). In addition, the system organizes the data and generates \"materials maps\" and models, facilitating knowledge creation by providing researchers a sharable big-picture view of unexpected materials, thereby accelerating materials development.\n\n\n\nThe core of such a materials explorer is the autonomous experimental system (AES) based on machine learning (ML) and robots (green, orange, and purple in Figure 1). Here, the term \"autonomous\" means that a computer algorithm decides the next experimental steps while robots perform all experimental steps. This approach, which involves no human intervention, is called the closed-loop experiment (Figure 2).\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 2. The concept of an autonomous experimental system (AES). The system autonomously synthesizes materials with optimal physical characteristics without human intervention. Autonomy leads to significant improvements: 1) fully digitalized experiments transforms all experimental parameters, including process conditions, into data; 2) removal of human error makes reproducibility reliable; and 3) implicit knowledge can be digitalized and embedded.\n\n\n\nIn general, new materials are sought in a multi-dimensional space by optimizing many relevant experimental parameters. Because of the vastness of the search space, the manual optimization of these parameters by individual researchers only produces incremental results that do not show the big picture. However, this problem is ideally suited for the AES to address. Figure 2 illustrates one such example. Here, based on the initial instructions, ML decides which compound to synthesize and feeds the corresponding directions to the robots; the robots synthesize, test, and report the results back to the algorithm, repeating the cycle until the desired result is obtained. This autonomous experimental approach drastically speeds up the materials exploration processes.\nThe concept of the materials explorer fundamentally changes the way we understand and conduct materials research, across three stages:\nStage 1: Optimization of the yield of target substances - Here, the target compound is known, but the optimum synthesis conditions are unknown. The target compound is decided by the human researchers before the experiment, and the AES quickly optimizes the synthesis conditions of the target compound within the search space specified by the human researchers.[3]\nStage 2: Finding new materials with desired properties - Here, the target physical properties are decided, but the compound possessing these properties is unknown. The AES quickly finds the best material within the search space specified by the human researchers. In contrast to Stage 1, it is the composition of the materials that is changed to find new compounds to meet the required physical properties. Materials with a variety of crystal structures and hierarchical structures are also explored.\nStage 3: Finding new materials or principles that no one has thought of before - Here, new materials, theories, and principles that are unexpected to researchers are discovered by combining the results from autonomous experiments, materials informatics, and human researchers (Figure 1).[4]\nAt present, the proofs of concepts of Stages 1 and 2 have been demonstrated in a variety of fields.[5][6][7][8][9][10][11][12][13][14][15][16][17] Expanding the application of AESs to a variety of experiments has become the next objective. Meanwhile, Stage 3 is rapidly advancing owing to the development of ML, robotics, and materials informatics. In this stage, it is critical to embed the researchers\u2019 intuition and experience into the construction of theories. This inclusion of the human role is often referred to as having a \"human-in-the-loop\" or \"researcher-in-the-loop.\"\nAn important point to note is that the transformation resulting from the three stages will change the way researchers think, giving completely new perspectives that researchers cannot obtain using conventional research methods. In addition, the transformation is not limited to a single laboratory; instead, it will change how we conduct research through a digital transformation of materials science. For example, experimentalists can remotely fabricate materials via the internet. In the same way, theorists can fabricate materials to test their predictions.\nThe data generated by such a system would fall into the domain of \"big data.\" Researchers use this big data to extract human-readable information (i.e., via materials informatics). Because ML and robotics alone can neither find insights nor discover concepts in physics and chemistry, human researchers will always remain central to the research. The key point is that materials scientists must understand what ML and robotics can solve, and must set the right problem to be solved. The strength of human researchers lies in concept creation or problem identification in the larger context. Combining these strengths with ML and robotics is critical to accelerating research (i.e., the researcher-in-the-loop).\nThe process of autonomous materials exploration, however, raises one fundamental question: when experiments are performed autonomously and new materials are found without human intervention, who is the discoverer and inventor? This question not only relates to the authorship of papers and the inventorship of patents, but it also concerns researchers\u2019 motivations. It is important to address this discussion as the autonomous experimental approach develops.\nIn this article, we review the trends and the prospects of AESs in materials science, specifically through the following aspects:\n\nthe overall status of autonomous experiments in materials science;\nthe history and the latest topics of autonomous materials synthesis;\nthe future of research using the AES;\nsome important take-home lessons we have learned when developing and using such a technology; and\nthe future of authorship and inventorship.\nThroughout this review, we address what human researchers should focus on in the era of autonomous research. The year 2020 was momentous for this era; there were significant advancements in the field of autonomous research. This review aims to further contribute to the expansion of the era of autonomous materials research.\n\nPresent status of autonomous experiments \nA brief description of our own autonomous experiments \nIn the closed-loop cycle shown in Figure 2, researchers only need to choose the material properties to optimize and provide the system with the necessary raw materials; the automatic system then takes control, repeatedly synthesizing and measuring the properties of new compounds until the best one is found. The ML algorithm uses previous knowledge to decide how the synthesis conditions should be changed to approach the desired outcome with each cycle.\nRecently, we have developed an autonomous synthesis of inorganic thin films.[3] In this proof-of-concept study, we demonstrated the autonomous fabrication of TiO2 thin films with low resistance and showed that this system accelerates experiments by tenfold.\nTo obtain these results, we have used robotic modules of a sputter deposition apparatus and a robotic device for measuring resistance. Other modules with robotic synthesis and measurement equipment can be connected to this system to adapt to the desired research. The robotic arm transfers the samples from module to module as needed, and the Bayesian optimization algorithm predicts the synthesis parameters for the next iteration.\n\nAES in the world \nRecent years have witnessed the rapid progress of autonomous experiments, including a) development of ML technology, b) improvement of robot technology and expansion of its range of applications, and c) realization of autonomous experiments using ML and robots. As discussed in the introduction, examples of autonomous materials syntheses can be found to demonstrate Stages 1 and 2.[5][6][7][8][9][10][11][12][13][14][15][16][17] In this section, we review the history and the latest advances demonstrating these two stages (Table 1).\n\n\n\n\n\n\n\nTable 1. Examples of autonomous materials synthesis using robots.\n\n\nResearch group (year)\n\nRobot mechanism\n\nOptimization objective\/Synthesized material\n\nVariables\/Algorithm\n\n\nMatsuda et al. (1988)[18]\n\nRobotic arm to manipulate the test tube\n\nMaximizing color-developing reactions for analysis (Stage 1)\n\nOptimizing the amount of reagent, reaction time, etc. using the simplex method\n\n\nNikolaev et al. (2016)[19]\n\nLaser to perform both heating and spectroscopy\n\nCarbon nanotubes with maximized growth rate (Stage 1)\n\nOptimizing temperature, pressure, and gas composition using a genetic algorithm\n\n\nChristensen et al. (2021)[20]\n\nLiquid handling robot\n\nMaximizing the yield of Suzuki-Miyaura coupling reaction (Stage 1)\n\nOptimizing continuous variables (amount of ingredients, etc.) and categorical variables (catalyst type) using Bayesian optimization\n\n\nMcMullen et al (2010)[21]\n\nFlow reactor (synthesis in liquid phase)\n\nMaximizing the yield of Heck reaction (Stage 1)\n\nOptimizing temperature and raw materials ratio using simplex method\n\n\nSans et al. (2015)[22]\n\nFlow reactor\n\nMaximizing the yield of imine synthesis (Stage 1)\n\nOptimizing temperature and raw materials ratio using simplex method\n\n\nKing et al. (2009)[23] and King (2011)[24]\n\nLiquid handling robot, a robotic arm, etc.\n\nDiscovering yeast genes (Stage 2)\n\nGenerating hypotheses and experimentally testing these hypotheses\n\n\nBurger et al. (2020)[25]\n\nFree-roaming robot\n\nPhotocatalyst mixtures with maximized photocatalytic activity (Stage 2)\n\nOptimizing the concentration of photocatalyst and additives using Bayesian optimization\n\n\nMacLeod et al. (2020)[26]\n\nA gripper, a pipette mount, and a robotic arm\n\nOrganic hole transport materials with maximized hole mobility (Stage 2)\n\nOptimizing annealing times and dopant concentrations using Bayesian optimization\n\n\nKrishnadasan et al. (2007)[27]\n\nFlow reactor\n\nCdSe nanoparticles with targeted spectroscopic characteristics (Stage 2)\n\nOptimizing temperature and precursor concentrations using SNOBFIT method\n\n\nTao et al. (2021)[28]\n\nFlow reactor\n\nAu nanoparticles with targeted spectroscopic characteristics (Stage 2)\n\nOptimizing reaction time and precursor concentrations using Bayesian optimization\n\n\nDesai et al. (2013)[29]\n\nFlow reactor\n\nAbl kinase inhibitors with maximum activity (Stage 2)\n\nSynthesizing from 27\u2009\u00d7\u200910 row materials using random forest\n\n\nDave et al. (2020)[30]\n\nFlow reactor\n\nAqueous electrolytes with maximum electrochemical window (Stage 2)\n\nOptimizing each precursor solution volume using Bayesian optimization\n\n\nShimizu et al. (2020)[3]\n\nA robot arm, a sputtering system, a resistance meter\n\nNb-doped TiO2 thin film with minimized resistance (Stage 2)\n\nOptimizing oxygen partial pressure using Bayesian optimization\n\n\n\nStage 1: Optimization of the yield of target substances or reaction conditions \nThe idea of autonomous materials synthesis using robots has a long history. The first proposal of a fully automated closed-loop robot aiming for the optimization of chemical reaction parameters was published in 1978.[31] However, no experimental results were reported.\nIn the 1980s, Matsuda et al. reported the optimization of reaction conditions using an autonomous system.[18] The repetition of reagent adjustment, reaction, measurement, and prediction of the next experimental conditions based on the measured results were automatically performed. Here, test tubes were manipulated by a robot. The color-developing reactions for chemical analysis were optimized using the simplex method with three parameters: the amount of two reagents and the reaction time. The exhaustive grid search required 130 experiments, but the robotic system optimized the reaction in less than 28 experiments. While the purpose of this experiment was not to find a new compound but to optimize the color-developing reaction, it is a seminal pioneering work of autonomous experiments.\nIn 2016, Nikolaev et al. demonstrated an autonomous synthesis of inorganic materials. The authors synthesized carbon nanotubes using chemical vapor deposition.[19] They fabricated multiple columns containing a catalyst layer on a substrate in advance. Then, they heated the individual columns one by one with a laser while repeatedly moving the substrate to synthesize carbon nanotubes with different growth conditions. This heating laser was also used as an excitation source for Raman spectroscopy to observe the growth rate in situ. A genetic algorithm maximized the growth rate; the system optimized the temperature, pressure, and gas composition. Later the group utilized a Bayesian optimization as an optimization algorithm.[32]\nFor organics, Christensen et al. maximized the yield of the Suzuki-Miyaura coupling reaction using Bayesian optimization.[20] They used a commercially available liquid handling robot and ChemOS[33][34][35] for autonomous experiments. In the Bayesian optimization, Phoenics[36] and Gryffin[37] algorithms were used to optimize categorical variables (catalyst type) in addition to continuous variables (amount of catalyst, amount of feedstock, reaction temperature). Parallel autonomous process optimization experiments in batches were performed to shorten the time to complete the optimization.\nThe autonomous synthesis of organics using flow reactors was reported by McMullen et al.[21] The authors first reported on a Heck reaction, where the yield was maximized by optimizing the raw materials ratio and reaction time as independent variables, using the simplex method (Nelder-Mead method). Subsequently, the authors worked on other autonomous syntheses using flow reactors[38][39]. For example, they applied SNOBFIT (Stable Noisy Optimization by Branch and Fit) to a variety of reactions using a modular system.[40] In 2015, Sans et al. optimized the yield of an imine synthesis.[22] The authors also used the simplex method to tune the raw materials ratio and reaction time. The system was equipped with in-line nuclear magnetic resonance spectroscopy. The group also developed an organic synthesis robot that navigates chemical reaction spaces.[41][42][43]\n\nStage 2: Finding new materials with desired properties \nIn 2009, King et al. reported a seminal work[23][24], where a robot \"Adam\" autonomously generated functional genomics hypotheses and experimentally tested the hypotheses using robots. The system measured the growth curves of selected microbial strains growing in defined media, \"discovering\" three novel yeast genes. The equipment comprised a liquid handling robot, a robot arm, and an incubator, all fixed in one place.\nIn 2020, Burger et al. demonstrated a free-roaming robot that moved around the laboratory to perform autonomous experiments using the same equipment as those used by its human counterparts.[25] The robot aimed to maximize photocatalytic activity by optimizing the concentrations of photocatalyst and additives. Based on Bayesian optimization, the robot identified the photocatalyst mixtures that were six times more active than the initial formulation. The robot completed 688 experiments in eight\u2009days. This number of experiments would take a human researcher several months. For reagent weighing, the robot handled both powder and liquid materials.\nIn the same year, MacLeod et al. reported the autonomous fabrication of organic thin films.[26] A robot \"Ada,\" equipped with a gripper, a pipette mount, and a robotic arm for liquid injection and substrate transfer, maximized the hole mobility of organic hole transport materials used in perovskite solar cells. The dopant concentrations and annealing time were optimized using Bayesian optimization. Ada finished the experiment in five days instead of nine months. Later, the group used Ada to define a Pareto front of conductivities and processing temperatures for palladium films formed by combustion synthesis.[44] There are other liquid handling robots developed for closed-loop, e.g., Wu et al. reported an automated platform for organic laser discovery, including not only synthesis and compound identification but also integrated target property characterization.[45]\nThere are several autonomous experiments based on flow reactors (synthesis in the liquid phase). For inorganic nanoparticles, Krishnadasan et al. autonomously synthesized CdSe nanoparticles with optimized intensity for a chosen emission wavelength in 2007 (SNOBFIT method).[27] Later, other groups reported various autonomous experiments on inorganic nanoparticles.[46][47][48][49][50] For example, Tao et al. synthesized Au nanoparticles[28] with targeted spectroscopic characteristics using the above-mentioned Gryffin[37] in 2021.\nFor organic compounds, Desai et al. synthesized Abl kinase inhibitors with maximum activity in 2013.[29] Out of 270 possible combinations (10 types\u2009\u00d7\u200927 types) from the raw materials, the system found the novel Abl kinase inhibitor after synthesizing only 21 compounds. The prediction model used random forest regression to handle the types of raw materials with different structures.\nDave et al. reported isolating aqueous electrolytes with the maximum electrochemical window in 2020.[30] The authors used Bayesian optimization to optimize the solution volume of three aqueous Li salts or four aqueous Na salts. The results indicated that non-smooth chemical responses are observed along the axis of the amount of NaBr. The authors pointed out that log-scaling, which varies rapidly with quantity, such as the amount of NaBr, gives the response surface both smaller gradients along this axis and a much-improved performance because Gaussian process regression requires an assumption of smoothness on the response surface. They also pointed out that the selection and presentation of the design space for materials search are of utmost importance to autonomous task design.\n\nFuture of Stages 1 and 2: Autonomously search within the preset search space \nCurrently, Stages 1 and 2 are not widespread because the range of applicable experiments is still limited. As more sophisticated robots are developed, the application of autonomous research will expand in turn. The problem lies in that conventional robots are rigidly designed for precision, aiming for zero error. Because they are programmed based on if-then statements, these robots cannot respond flexibly to changing situations. Therefore, it is difficult to implement the tacit knowledge of skilled human researchers in robot systems. With soft control, soft structure, multiple sensing devices, and ML, a more flexible robot can absorb errors and disruptions, and can operate without stoppage or breakdowns.[51][52] Thus, developing robot technology that can respond flexibly until the goal is achieved is important. To this end, it is essential to develop new techniques for robots to learn the minute details of any required motions.\nWhen versatile robots capable of performing a variety of experiments become affordable, the introduction of robots will rapidly expand into many research fields. At the time of this writing, the cost of robots is reducing rapidly to the point where labs can consider using such robot-experiment automation. However, more effort is needed to help materials scientists implement robots into their experiments.\nWe note that not all researchers will adopt this style of research. The conventional research approach\u2014which relies on researchers\u2019 experience, knowledge, and intuition\u2014will remain essential for conceiving new synthetic methods and analytical techniques. We expect there will be a hybrid between conventional and autonomous approaches.\nAESs are not only useful for performing high-throughput experiments; the most critical aim is to help human researchers deepen their research. Human researchers need to think about how to analyze the collected data, how the new materials can be used, and what theories can be deduced from unexpected results. AESs free human researchers from repetitive tasks and enable researchers to conduct innovative research. Furthermore, combining the system with advanced materials prediction based on computer simulation is the step that enables human researchers to perform more creative work (Figure 3).\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 3. The human researchers will always play the leading role. The autonomous laboratory is for researchers. This technology makes it possible to narrow down candidate materials with high predictive performance and to pinpoint materials synthesis experiments. The researchers then work on highly creative tasks and acquire \"tacit knowledge\" of an even higher level. In turn, the new knowledge is digitized again. It is essential to repeat this cycle.\n\n\n\nWith the implementation of AESs, theorists can test their original ideas. Once theorists predict a promising material, they can log on to the AES via the internet and synthesize the new material. Currently, we are developing a system capable of automatically synthesizing materials from simple instructions, such as \"synthesize AxByCz with a crystal structure of D.\" The system automatically measures X-ray diffraction patterns and feeds the results back to the algorithm to predict the next synthesis condition.\n\nStage 3: Finding materials that no one has thought of before \nBecause the AES will generate increasingly larger amounts of data, it is essential to combine this type of system with materials informatics and computer simulations to extend human thinking. In so doing, the generated data will promote the emergence of unexpected results that are beyond conventional theories and experiences. It is also important to embed the researchers\u2019 intuition and experience, which are tacit knowledge, into the autonomous system and share them with other researchers (Figure 4). \n\r\n\n\n\n\n\n\n\n\n\n\nFigure 4. A vision of how research will be conducted. In the conventional way of research, there has been a disruption between researchers and robots\/artificial intelligence (AI). In the near future, researchers will develop materials based on an enormous amount of experimental and simulation results. In this \"researcher-in-the-loop\" process, human researchers will obtain new insights, a bird\u2019s eye view of materials through multifaceted materials characterization and resulting materials big data.\n\n\n\nThe following are some of the remaining challenges.\n\n Increasing the amount of reliable materials data (materials checkup system and measurement) \nTo date, materials informatics is still limited by the lack of a reliable materials experiment database. Such a database cannot be manually acquired. To this end, AESs enable the storage of all data, ranging from the fabrication processes (synthesis conditions and experimental environment) to the structural and composition information, to the measured materials properties. Even the negative data (data that do not show the desired values) are stored for later use. This inclusive approach to data storage drastically increases the amount of data available for accurate prediction based on ML. In addition, all data are highly reproducible because they are not acquired by humans (i.e., devoid of human errors). Therefore, it is possible to analyze in detail which process parameters are correlated with physical properties, leading to technology for predicting synthetic processes (i.e., process informatics).\nThe sheer increase in experimental speed directly translates to an increase in the amount of materials data. The speed of autonomous experiments is estimated to be at least one order of magnitude faster than conventional experiments; it is possible to accelerate the acquisition of measurement data by another order of magnitude using the materials checkup system (Figure 5).[3]\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 5. Concept of the materials checkup system for finding unexpected results. All the material-property measurements are done automatically. Original source Shimizu et al. 2020, reprinted with permission under a Creative Commons Attribution (CC BY) license.[3]\n\n\n\nIn conventional research, human researchers often need to transport their samples to multiple analysis systems to evaluate the physical properties. To avoid errors associated with sample transport and shorten the time for analysis, all analytical modules should be connected to each other. Connecting multiple materials property measurement devices and automatically measuring one after another enable multifaceted materials characterization (Figure 5). With the recent development of measurement and analysis informatics[53], it is possible to perform measurements with even higher accuracy, precision, and sensitivity measurements in a short period of time.[54]\nThe key to the multifaceted evaluation is to increase the probability of finding unexpected results. We have already experienced finding an unexpected electrode material using multifaceted evaluation while targeting the synthesis of a solid electrolyte.\n\n Constructing scientific theories from a bird\u2019s-eye view \nThe AES does not search aimlessly in the search space. Instead, it creates a materials map[55] based on the large amount of data described in the previous section to give a bird\u2019s-eye view of the materials world (Figure 6).\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 6. Creation of a \"materials map\" to identify unexplored materials. Autonomous experiments enable the creation of materials maps using experimental data. We put particular emphasis on unexpected experimental results and explore their surroundings.\n\n\n\nSpecifically, materials informatics methods help organize the enormous amount of multi-dimensional data (e.g., actual experimental data and high-throughput first-principles simulations) into human-readable forms. This data is overlaid with previously reported experimental and simulated data to create a materials map.\nThe materials map depicts the untapped and promising areas that should be experimentally verified. This depiction stimulates the researchers\u2019 thinking in ways conventional research methods cannot. The new ideas can then be immediately tested using the AES (Figure 1).\nThe unveiling of the hidden correlations and relationships between materials properties and processes will promote the construction of new theories. Until now, completely unexpected and never-before-observed results are often ignored because it has been difficult to distinguish whether they are real or whether they are caused by experimental error. However, these results can be reproduced when using the AES; unexpected results (i.e., \"outliers\") will not be ignored, making it possible to create new scientific theories that explain them (Figure 7).\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 7. Importance of outliers. Suppose there is a number hidden in the tiles in (a). With no data, nothing can be predicted. (b) With some amount of data, hypotheses can be proposed. Typical researchers could not predict that the number \"1\" is hidden in this figure. Only an exceptional researcher can recognize the hidden \"1.\" Materials science to date has been \"local\" (i.e., belonging to particular researchers) in this way. (c) When big data is available, researchers can easily recognize \"1,\" without relying on such exceptional researchers. Furthermore, by considering the \"outliers\" (shown in red), we can see the true situation: a \"4\" rather than a \"1.\" We can thus construct new theories (i.e., big-picture materials science).\n\n\n\nLastly, to overview a massive amount of materials data, it is necessary to have the technology capable of converting this data for humans to understand. In particular, the development of technologies for explainable ML and visualization of information in a multi-dimensional space through dimensionality reduction is mandatory.\n\nImportant points for the actual operation of an autonomous system \nThis section discusses some important take-home lessons we have learned from our constructing and operating the AES.\n\nSetting the right research topic \nIt is not practical to leave all experimental operations to ML and robots; instead, it is crucial to choose the right tasks. The current autonomous experimental technology works best when applied to processes with well-developed instructions. Techniques with broad and general applications that many researchers use are better suited for applying to autonomous systems. For example, we chose to automate sputtering thin-film deposition because it is widely used in the field, including the semiconductor industry.\nAnother point concerns the human operator\u2019s psychological aspect. In our experiments, the human operators perform two tasks: setting up the raw materials and cleaning up the pieces used in the experiments. Because the operator performs these tasks according to the robot\u2019s schedule, in some cases, the operator may feel as if being controlled by robots. It is critical that robots perform these tasks in the future.\n\nCost issues \nIt is difficult to introduce an autonomous system to every laboratory because of its cost. There are three points to consider. The first point is that robots are becoming less expensive, and their range of applications is expanding. The price depends on positioning accuracy and repeatability. Because robots are generally capable of working 24\u2009hours a day, 7\u2009days a week, the operational cost depends on the tasks and the labor cost. The software cost has also decreased, as robots can be run using LabVIEW or Python, two widely used programming environments. In this way, the total system cost is becoming affordable, and we are now in an era when robots can be introduced to laboratories. We note that it is important to provide open labs for researchers unaware of the possibilities of autonomous systems, to get acquainted with the actual setups and operations of such systems.\nThe second point is to promote sharing the autonomous system via the internet, providing access to the system from anywhere in the world (via cloud computing for the laboratory, sharing experiments over the cloud; Figure 8). Currently, the equipment in the laboratory is only used during the daytime on weekdays and therefore is not fully utilized. Full utilization around the clock through sharing will enhance cost-effectiveness; it will also enable materials fabrication while researchers are at home, making a remote style of laboratory work possible.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 8. Experiments in the cloud space. Sharing the autonomous system via the internet has become possible.\n\n\n\nThe last point is the perspective of multifaceted characterization. As shown in Figure 5, multiple analytic instruments automatically characterize a range of properties. The time previously spent by researchers on measurements can now be allocated to higher value-added tasks, further enhancing cost-effectiveness.\n\nChange in the experimental sequences \nConventionally, the materials to be synthesized are decided before the synthesis. After the synthesis, compositions and structures are analyzed to confirm that the aimed materials have been synthesized. After identifying the compounds, the properties are evaluated.\nAutonomous experiments reverse this sequence. Because the goal of the research is to determine the material with superior physical properties, there is no need to decide what materials to synthesize in advance. The compositional and structural analyses to identify the materials are only performed once the robot has synthesized materials with the optimum properties.\nAutonomous experiments applied to device fabrications show a similar sequence reversal. The system automatically fabricates a device, evaluates its performance, and tunes the next materials based on the results. Again, there is no need to know the details of the materials prior to device fabrications.\nFrom a broader perspective, autonomous systems change our attitude toward research. New ideas\u2014which previously required careful experimental planning to be tested with a small number of trials\u2014can now be tested immediately using an autonomous system. With automation, we can comfortably conduct experiments even with a naive idea because the autonomous system performs experiments, and the number of experiments can be increased at will.\n\nSteps for implementation and the rise of the lab-system integrator \nHow should a materials research lab start out with automation and autonomous experiments? The following points should be considered when integrating materials synthesis and evaluation, robotics, ML, and overall system control.\n\nClarify the workflow and choose the appropriate tasks for automation (see the prior discussion about setting the right research topic).\nSet up a team, including an engineer who understands informatics. In addition, include an engineer familiar with equipment control and systemization.\nThe introduction of robots can be expensive. It may be difficult to introduce the whole system at once. Therefore, it is necessary to create a chain of successes to maintain continuous funding. The critical point is to draw an overall plan and accumulate small successes.\nStandardizing the lab equipment is essential to reduce the time to set up and the cost. Shortly, lab-system integrators will emerge. There is a movement in the world towards aligning the format of all mechanical interfaces and communication protocol; each piece of equipment will be integrated to form an automation system.[15] Standardization and de facto standards are key for lab-system integrators, and the technology must be launched quickly.\n\nThe autonomous system and intellectual property \nCurrently, the intellectual property of newly discovered or invented materials is effectuated through scientific publication or patent application. However, when the discovery\/invention is accomplished without human intervention, can we say that a human discovered or invented the materials? Can the AES be an inventor under patent law?[25] Can the system be an author of a paper? Furthermore, patents are designed to protect inventions that are not only new, but also involve an inventive step; will the autonomous system change how inventive steps are determined? In other words, will the autonomous system change the standard of what constitutes an easy invention? In this section, we discuss inventorship, inventive step, and authorship.\n\n Inventorship \u2013 Who is an inventor? \nWhether AI systems, including autonomous systems, should be treated as an inventor under patent law is a hot topic.[56][57][58] A person who makes an invention is called the inventor (we note that the patent owner is often not the inventor but the company that employs the inventor). However, if AI makes the invention, can it be considered the inventor under patent law?\nAbbott et al. applied for patents naming the AI \"DABUS\" as the inventor in several countries and regions to raise this issue.[59] They proposed that the inventor is DABUS and the patent owner would be the person who owns DABUS. For this case, the South African patent authorities have accepted the patent applications with DABUS as the sole inventor[60]; however, we note that the decision can still be overturned by the respective superior courts.\nSince the wording of patent acts and their interpretations differ from one jurisdiction to another, whether AI can be treated by patent acts as an inventor also differs. In contrast to the above examples, the patent offices and courts in the U.S., U.K., Australia, the European Patent Office, and the Japan Patent Office have all ruled that only human beings can be the inventor under the patent acts, rejecting the notion of DABUS being treated as an inventor.[58][59][61][62][63] In such countries\/regions, even when a new material is discovered by an AES, only human researchers who have operated the system and contributed creatively may be treated as inventors. As mentioned prior, the current level of technology still needs significant human contribution to conduct the research.\nHowever, there is an argument that if the AES becomes more autonomous and the level of human contribution continues to decrease, the human researcher may no longer be treated as the inventor. For example, in Japan, the inventor is recognized as a person who conceives an invention or creates the specific materials based on the conceived idea.[64][65] From this perspective, if the AES both conceives and creates a specific material, the human would no longer be considered the inventor. As the autonomous system evolves, in order to be recognized as the inventor, future human researchers will have to either exercise creativity independently from the AES or demonstrate that the AES is used as a tool.\n\n Inventive step \u2013 Is an invention easily invented? \nTo be protected by a patent, an invention must satisfy the definition of inventive step (non-obviousness in the US or Canada). The question here is in what way the AES will change how the inventive step is determined. In the following, we discuss the impact of the system on the inventive step in the future, according to the Japanese patent system.\nBy definition, an inventive step requires that an ordinary skilled person in the field not be able to make the invention easily at the time of filing the patent application. In Japan[66], patent examiners determine the inventive step by assessing (1) factors indicating the absence of inventive step, and (2) factors supporting the existence of inventive step. Specifically, factor (1) questions whether a skilled person in the field could have easily made the invention, and factor (2) questions, for example, the existence of advantageous effects exceeding predictability based on the state of the art.\nAt present, because AI in materials exploration is still in its infancy and its accuracy remains low, the current patent examination practices determining inventive step have not been changed. However, many researchers in intellectual properties and business associations point out that, as AI\u2019s accuracy continues to rise and becomes widely used, the difficulty of achieving new invention will be lowered for the skilled person in the field, raising the standard for what satisfies the definition of inventive step.[67][68][69][70][71] In this case, how the inventive step is legally determined may be changed in the future.\nHere we apply this argument to the finding of novel materials by the AES. At this time, the system is not yet widely used, but, in the future, the situation may change as this system becomes well implemented or shared in laboratories. Then, we believe the determination of the inventive step may be changed by a shift in the balance between factors (1) and (2). For example, because the AES excels at optimizing numerical conditions, it is conceivable that many invention cases involving such optimizations would be accessible to an ordinary skilled person in the field with access to the AES. Such cases would make factor (1) stronger because it is sufficiently reasoned for the ordinary skilled person to arrive at the invention, causing the standard for satisfying the inventive step to be raised. Although the possibility to surpass the raised standard by considering a combination of factor (2), such as advantageous effects, the chance of satisfying the inventive step decreases if the current research and development approach remains unchanged.\nTo obtain patent protection in a future where the AES is widely used, human researchers will have to achieve the inventive step by redirecting their creativity away from the strengths of the AES, in addition to the inventorship mentioned above. For example, the AES will never be able to conceive highly innovative synthesis techniques, or develop new characterization techniques. Furthermore, even for inventions involving optimization of numerical conditions, human creativity will contribute to the inventive step when it is difficult even for the AES to come up with a specific parameter. In addition, new materials discovered by an AES will still need to be put to use, requiring human input to create new devices by combining many materials, utilizing hierarchical structures. Lastly, it is human beings who will think about what issues facing society and how to solve them; thus, the role of the human researchers is to judge the value and to decide what to do next. This big-picture perspective will lead to new inventions that only humans can think of.\n\n Authorship \u2013 Who is an author of academic papers? \nEach journal and publisher has its authorship guidelines. One well-known example is the authorship guidelines published by the Committee on Publication Ethics (COPE)[72], which states that \"the minimum requirements for authorship, common to all definitions, are (A) substantial contribution to the work and (B) accountability for the work that was done and its presentation in a publication.\" Another well-known example from the International Committee of Medical Journal Editors (ICMJE)[73] also includes the same concept as requirements (A) and (B).\nAlthough most scientists disapprove of articles crediting the AI tool as an author, an AI tool such as ChatGPT has listed its name as an author.[74][75][76] However, these cases do not focus on the AES but mainly show that AI can generate sentences. Whether or not the AES can be treated as an author of an academic discovery is a quite important issue and should be discussed.\nSo, let\u2019s consider the case where human researchers write a manuscript on a new material discovered by the AES. At the current technological level, while the AES can satisfy substantial contribution (requirement A) because the system has performed the laboratory procedures leading to the new-material discovery, it cannot provide accountability (requirement B). There are many theories on whether AI can have accountability[77]; however, at present, the general view is that only human researchers can be held accountable.[78] Until AI can provide accountability, AI is not eligible for academic authorship.\n\nSummary \nAs the involvement of ML and robotics continues to expand in scientific research, the roles of human researchers have become a central question. ML and robots will not replace researchers; from the onset, AESs have aimed at providing researchers with the chance to think and to be more creative. The important point is that scientists understand both the limitations and the possibilities of ML and robotics, and apply these techniques to the right problem. Even in an age where ML and robots perform most of the laboratory procedures, human researchers will always be the main players. This approach to automation and autonomy will exponentially accelerate creative research and development, contributing to the future development of science, technology, and industry.\n\nAcknowledgements \nWe would like to thank R. Nakayama, S. Kobayashi, and T. Kimura at Tokyo Institute of Technology and D. Packwood at Kyoto University for their continuous discussions. We thank P. Han for editing the manuscript.\n\nFunding \nThis research was supported by JST-CREST [Grant No. JPMJCR1523], JST-MIRAI [Grant No. JPMJMI21G2], and MEXT Program: Data Creation and Utilization-Type Material Research and Development Project [Grant No. JPMXP1122712807].\n\nConflict of interest \nNo potential conflict of interest was reported by the author(s).\n\nReferences \n\n\n\u2191 Bohacek, R. S.; McMartin, C.; Guida, W. C. (1 January 1996). \"The art and practice of structure-based drug design: a molecular modeling perspective\". Medicinal Research Reviews 16 (1): 3\u201350. doi:10.1002\/(SICI)1098-1128(199601)16:1<3::AID-MED1>3.0.CO;2-6. ISSN 0198-6325. PMID 8788213. https:\/\/pubmed.ncbi.nlm.nih.gov\/8788213 .   \n \n\n\u2191 Kirkpatrick, Peter; Ellis, Clare (1 December 2004). \"Chemical space\" (in en). Nature 432 (7019): 823\u2013823. doi:10.1038\/432823a. ISSN 0028-0836. https:\/\/www.nature.com\/articles\/432823a .   \n \n\n\u2191 3.0 3.1 3.2 3.3 3.4 Shimizu, Ryota; Kobayashi, Shigeru; Watanabe, Yuki; Ando, Yasunobu; Hitosugi, Taro (1 November 2020). \"Autonomous materials synthesis by machine learning and robotics\" (in en). APL Materials 8 (11): 111110. doi:10.1063\/5.0020370. ISSN 2166-532X. https:\/\/pubs.aip.org\/apm\/article\/8\/11\/111110\/569817\/Autonomous-materials-synthesis-by-machine-learning .   \n \n\n\u2191 Kitano, Hiroaki (18 June 2021). \"Nobel Turing Challenge: creating the engine for scientific discovery\" (in en). npj Systems Biology and Applications 7 (1): 29. doi:10.1038\/s41540-021-00189-3. ISSN 2056-7189. PMC PMC8213706. PMID 34145287. https:\/\/www.nature.com\/articles\/s41540-021-00189-3 .   \n \n\n\u2191 5.0 5.1 Tabor, Daniel P.; Roch, Lo\u00efc M.; Saikin, Semion K.; Kreisbeck, Christoph; Sheberla, Dennis; Montoya, Joseph H.; Dwaraknath, Shyam; Aykol, Muratahan et al. (26 April 2018). \"Accelerating the discovery of materials for clean energy in the era of smart automation\" (in en). Nature Reviews Materials 3 (5): 5\u201320. doi:10.1038\/s41578-018-0005-z. ISSN 2058-8437. https:\/\/www.nature.com\/articles\/s41578-018-0005-z .   \n \n\n\u2191 6.0 6.1 Dimitrov, Tanja; Kreisbeck, Christoph; Becker, Jill S.; Aspuru-Guzik, Al\u00e1n; Saikin, Semion K. (17 July 2019). \"Autonomous Molecular Design: Then and Now\" (in en). ACS Applied Materials & Interfaces 11 (28): 24825\u201324836. doi:10.1021\/acsami.9b01226. ISSN 1944-8244. https:\/\/pubs.acs.org\/doi\/10.1021\/acsami.9b01226 .   \n \n\n\u2191 7.0 7.1 Flores-Leonar, Martha M.; Mej\u00eda-Mendoza, Luis M.; Aguilar-Granda, Andr\u00e9s; Sanchez-Lengeling, Benjamin; Tribukait, Hermann; Amador-Bedolla, Carlos; Aspuru-Guzik, Al\u00e1n (1 October 2020). \"Materials Acceleration Platforms: On the way to autonomous experimentation\" (in en). Current Opinion in Green and Sustainable Chemistry 25: 100370. doi:10.1016\/j.cogsc.2020.100370. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2452223620300596 .   \n \n\n\u2191 8.0 8.1 Coley, Connor W.; Eyke, Natalie S.; Jensen, Klavs F. (14 December 2020). \"Autonomous Discovery in the Chemical Sciences Part I: Progress\" (in en). Angewandte Chemie International Edition 59 (51): 22858\u201322893. doi:10.1002\/anie.201909987. ISSN 1433-7851. https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/anie.201909987 .   \n \n\n\u2191 9.0 9.1 Coley, Connor W.; Eyke, Natalie S.; Jensen, Klavs F. (21 December 2020). \"Autonomous Discovery in the Chemical Sciences Part II: Outlook\" (in en). Angewandte Chemie International Edition 59 (52): 23414\u201323436. doi:10.1002\/anie.201909989. ISSN 1433-7851. https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/anie.201909989 .   \n \n\n\u2191 10.0 10.1 Gromski, Piotr S.; Granda, Jaros\u0142aw M.; Cronin, Leroy (1 January 2020). \"Universal Chemical Synthesis and Discovery with \u2018The Chemputer\u2019\" (in en). Trends in Chemistry 2 (1): 4\u201312. doi:10.1016\/j.trechm.2019.07.004. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2589597419301868 .   \n \n\n\u2191 11.0 11.1 Wilbraham, Liam; Mehr, S. Hessam M.; Cronin, Leroy (19 January 2021). \"Digitizing Chemistry Using the Chemical Processing Unit: From Synthesis to Discovery\" (in en). Accounts of Chemical Research 54 (2): 253\u2013262. doi:10.1021\/acs.accounts.0c00674. ISSN 0001-4842. https:\/\/pubs.acs.org\/doi\/10.1021\/acs.accounts.0c00674 .   \n \n\n\u2191 12.0 12.1 Szymanski, Nathan J.; Zeng, Yan; Huo, Haoyan; Bartel, Christopher J.; Kim, Haegyeom; Ceder, Gerbrand (2021). \"Toward autonomous design and synthesis of novel inorganic materials\" (in en). Materials Horizons 8 (8): 2169\u20132198. doi:10.1039\/D1MH00495F. ISSN 2051-6347. http:\/\/xlink.rsc.org\/?DOI=D1MH00495F .   \n \n\n\u2191 13.0 13.1 Stach, Eric; DeCost, Brian; Kusne, A. Gilad; Hattrick-Simpers, Jason; Brown, Keith A.; Reyes, Kristofer G.; Schrier, Joshua; Billinge, Simon et al. (1 September 2021). \"Autonomous experimentation systems for materials development: A community perspective\" (in en). Matter 4 (9): 2702\u20132726. doi:10.1016\/j.matt.2021.06.036. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2590238521003064 .   \n \n\n\u2191 14.0 14.1 Fabry, David C.; Sugiono, Erli; Rueping, Magnus (1 April 2014). \"Self-Optimizing Reactor Systems: Algorithms, On-line Analytics, Setups, and Strategies for Accelerating Continuous Flow Process Optimization\" (in en). Israel Journal of Chemistry 54 (4): 341\u2013350. doi:10.1002\/ijch.201300080. https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/ijch.201300080 .   \n \n\n\u2191 15.0 15.1 15.2 Wagner, Jerrit; Berger, Christian G.; Du, Xiaoyan; Stubhan, Tobias; Hauch, Jens A.; Brabec, Christoph J. (1 October 2021). \"The evolution of Materials Acceleration Platforms: toward the laboratory of the future with AMANDA\" (in en). Journal of Materials Science 56 (29): 16422\u201316446. doi:10.1007\/s10853-021-06281-7. ISSN 0022-2461. https:\/\/link.springer.com\/10.1007\/s10853-021-06281-7 .   \n \n\n\u2191 16.0 16.1 Seifrid, Martin; Pollice, Robert; Aguilar-Granda, Andr\u00e9s; Morgan Chan, Zamyla; Hotta, Kazuhiro; Ser, Cher Tian; Vestfrid, Jenya; Wu, Tony C. et al. (6 September 2022). \"Autonomous Chemical Experiments: Challenges and Perspectives on Establishing a Self-Driving Lab\" (in en). Accounts of Chemical Research 55 (17): 2454\u20132466. doi:10.1021\/acs.accounts.2c00220. ISSN 0001-4842. PMC PMC9454899. PMID 35948428. https:\/\/pubs.acs.org\/doi\/10.1021\/acs.accounts.2c00220 .   \n \n\n\u2191 17.0 17.1 Abolhasani, Milad; Kumacheva, Eugenia (30 January 2023). \"The rise of self-driving labs in chemical and materials sciences\" (in en). Nature Synthesis 2 (6): 483\u2013492. doi:10.1038\/s44160-022-00231-0. ISSN 2731-0582. https:\/\/www.nature.com\/articles\/s44160-022-00231-0 .   \n \n\n\u2191 18.0 18.1 Matsuda, Rieko; Ishibashi, Mumio; Takeda, Yasushi (1988). \"Simplex optimization of reaction conditions with an automated system.\" (in en). Chemical and Pharmaceutical Bulletin 36 (9): 3512\u20133518. doi:10.1248\/cpb.36.3512. ISSN 0009-2363. http:\/\/www.jstage.jst.go.jp\/article\/cpb1958\/36\/9\/36_9_3512\/_article .   \n \n\n\u2191 19.0 19.1 Nikolaev, Pavel; Hooper, Daylond; Webber, Frederick; Rao, Rahul; Decker, Kevin; Krein, Michael; Poleski, Jason; Barto, Rick et al. (21 October 2016). \"Autonomy in materials research: a case study in carbon nanotube growth\" (in en). npj Computational Materials 2 (1): 16031. doi:10.1038\/npjcompumats.2016.31. ISSN 2057-3960. https:\/\/www.nature.com\/articles\/npjcompumats201631 .   \n \n\n\u2191 20.0 20.1 Christensen, Melodie; Yunker, Lars P. E.; Adedeji, Folarin; H\u00e4se, Florian; Roch, Lo\u00efc M.; Gensch, Tobias; dos Passos Gomes, Gabriel; Zepel, Tara et al. (2 August 2021). \"Data-science driven autonomous process optimization\" (in en). Communications Chemistry 4 (1): 112. doi:10.1038\/s42004-021-00550-x. ISSN 2399-3669. PMC PMC9814253. PMID 36697524. https:\/\/www.nature.com\/articles\/s42004-021-00550-x .   \n \n\n\u2191 21.0 21.1 McMullen, Jonathan P.; Stone, Matthew T.; Buchwald, Stephen L.; Jensen, Klavs F. (17 September 2010). \"An Integrated Microreactor System for Self-Optimization of a Heck Reaction: From Micro- to Mesoscale Flow Systems\" (in en). Angewandte Chemie International Edition 49 (39): 7076\u20137080. doi:10.1002\/anie.201002590. https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/anie.201002590 .   \n \n\n\u2191 22.0 22.1 Sans, Victor; Porwol, Luzian; Dragone, Vincenza; Cronin, Leroy (2015). \"A self optimizing synthetic organic reactor system using real-time in-line NMR spectroscopy\" (in en). Chemical Science 6 (2): 1258\u20131264. doi:10.1039\/C4SC03075C. ISSN 2041-6520. PMC PMC5811122. PMID 29560211. http:\/\/xlink.rsc.org\/?DOI=C4SC03075C .   \n \n\n\u2191 23.0 23.1 King, Ross D.; Rowland, Jem; Oliver, Stephen G.; Young, Michael; Aubrey, Wayne; Byrne, Emma; Liakata, Maria; Markham, Magdalena et al. (3 April 2009). \"The Automation of Science\" (in en). Science 324 (5923): 85\u201389. doi:10.1126\/science.1165620. ISSN 0036-8075. https:\/\/www.science.org\/doi\/10.1126\/science.1165620 .   \n \n\n\u2191 24.0 24.1 King, Ross D. (1 January 2011). \"Rise of the Robo Scientists\". Scientific American 304 (1): 72\u201377. doi:10.1038\/scientificamerican0111-72. ISSN 0036-8733. https:\/\/www.scientificamerican.com\/article\/rise-of-the-robo-scientists .   \n \n\n\u2191 25.0 25.1 25.2 Burger, Benjamin; Maffettone, Phillip M.; Gusev, Vladimir V.; Aitchison, Catherine M.; Bai, Yang; Wang, Xiaoyan; Li, Xiaobo; Alston, Ben M. et al. (9 July 2020). \"A mobile robotic chemist\" (in en). Nature 583 (7815): 237\u2013241. doi:10.1038\/s41586-020-2442-2. ISSN 0028-0836. https:\/\/www.nature.com\/articles\/s41586-020-2442-2 .   \n \n\n\u2191 26.0 26.1 MacLeod, B. P.; Parlane, F. G. L.; Morrissey, T. D.; H\u00e4se, F.; Roch, L. M.; Dettelbach, K. E.; Moreira, R.; Yunker, L. P. E. et al. (15 May 2020). \"Self-driving laboratory for accelerated discovery of thin-film materials\" (in en). Science Advances 6 (20): eaaz8867. doi:10.1126\/sciadv.aaz8867. ISSN 2375-2548. PMC PMC7220369. PMID 32426501. https:\/\/www.science.org\/doi\/10.1126\/sciadv.aaz8867 .   \n \n\n\u2191 27.0 27.1 Krishnadasan, S.; Brown, R. J. C.; deMello, A. J.; deMello, J. C. (2007). \"Intelligent routes to the controlled synthesis of nanoparticles\" (in en). Lab on a Chip 7 (11): 1434. doi:10.1039\/b711412e. ISSN 1473-0197. http:\/\/xlink.rsc.org\/?DOI=b711412e .   \n \n\n\u2191 28.0 28.1 Tao, Huachen; Wu, Tianyi; Kheiri, Sina; Aldeghi, Matteo; Aspuru\u2010Guzik, Al\u00e1n; Kumacheva, Eugenia (1 December 2021). \"Self\u2010Driving Platform for Metal Nanoparticle Synthesis: Combining Microfluidics and Machine Learning\" (in en). Advanced Functional Materials 31 (51): 2106725. doi:10.1002\/adfm.202106725. ISSN 1616-301X. https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/adfm.202106725 .   \n \n\n\u2191 29.0 29.1 Desai, Bimbisar; Dixon, Karen; Farrant, Elizabeth; Feng, Qixing; Gibson, Karl R.; van Hoorn, Willem P.; Mills, James; Morgan, Trevor et al. (11 April 2013). \"Rapid Discovery of a Novel Series of Abl Kinase Inhibitors by Application of an Integrated Microfluidic Synthesis and Screening Platform\" (in en). Journal of Medicinal Chemistry 56 (7): 3033\u20133047. doi:10.1021\/jm400099d. ISSN 0022-2623. https:\/\/pubs.acs.org\/doi\/10.1021\/jm400099d .   \n \n\n\u2191 30.0 30.1 Dave, Adarsh; Mitchell, Jared; Kandasamy, Kirthevasan; Wang, Han; Burke, Sven; Paria, Biswajit; P\u00f3czos, Barnab\u00e1s; Whitacre, Jay et al. (1 December 2020). \"Autonomous Discovery of Battery Electrolytes with Robotic Experimentation and Machine Learning\" (in en). Cell Reports Physical Science 1 (12): 100264. doi:10.1016\/j.xcrp.2020.100264. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2666386420302861 .   \n \n\n\u2191 Winicov, H.; Schainbaum, J.; Buckley, J.; Longino, G.; Hill, J.; Berkoff, C.E. (1 December 1978). \"Chemical process optimization by computer \u2014 a self-directed chemical synthesis system\" (in en). Analytica Chimica Acta 103 (4): 469\u2013476. doi:10.1016\/S0003-2670(01)83110-X. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S000326700183110X .   \n \n\n\u2191 Chang, Jorge; Nikolaev, Pavel; Carpena-N\u00fa\u00f1ez, Jennifer; Rao, Rahul; Decker, Kevin; Islam, Ahmad E.; Kim, Jiseob; Pitt, Mark A. et al. (3 June 2020). \"Efficient Closed-loop Maximization of Carbon Nanotube Growth Rate using Bayesian Optimization\" (in en). Scientific Reports 10 (1): 9040. doi:10.1038\/s41598-020-64397-3. ISSN 2045-2322. PMC PMC7271124. PMID 32493911. https:\/\/www.nature.com\/articles\/s41598-020-64397-3 .   \n \n\n\u2191 Roch, Lo\u00efc M.; H\u00e4se, Florian; Kreisbeck, Christoph; Tamayo-Mendoza, Teresa; Yunker, Lars P. E.; Hein, Jason E.; Aspuru-Guzik, Al\u00e1n (20 June 2018). \"ChemOS: Orchestrating autonomous experimentation\" (in en). Science Robotics 3 (19): eaat5559. doi:10.1126\/scirobotics.aat5559. ISSN 2470-9476. https:\/\/www.science.org\/doi\/10.1126\/scirobotics.aat5559 .   \n \n\n\u2191 Roch, Lo\u00efc M.; H\u00e4se, Florian; Kreisbeck, Christoph; Tamayo-Mendoza, Teresa; Yunker, Lars P. E.; Hein, Jason E.; Aspuru-Guzik, Al\u00e1n (16 April 2020). Hu, Jianjun. ed. \"ChemOS: An orchestration software to democratize autonomous discovery\" (in en). PLOS ONE 15 (4): e0229862. doi:10.1371\/journal.pone.0229862. ISSN 1932-6203. PMC PMC7161969. PMID 32298284. https:\/\/dx.plos.org\/10.1371\/journal.pone.0229862 .   \n \n\n\u2191 Pollice, Robert; dos Passos Gomes, Gabriel; Aldeghi, Matteo; Hickman, Riley J.; Krenn, Mario; Lavigne, Cyrille; Lindner-D\u2019Addario, Michael; Nigam, AkshatKumar et al. (16 February 2021). \"Data-Driven Strategies for Accelerated Materials Design\" (in en). Accounts of Chemical Research 54 (4): 849\u2013860. doi:10.1021\/acs.accounts.0c00785. ISSN 0001-4842. PMC PMC7893702. PMID 33528245. https:\/\/pubs.acs.org\/doi\/10.1021\/acs.accounts.0c00785 .   \n \n\n\u2191 H\u00e4se, Florian; Roch, Lo\u00efc M.; Kreisbeck, Christoph; Aspuru-Guzik, Al\u00e1n (26 September 2018). \"Phoenics: A Bayesian Optimizer for Chemistry\" (in en). ACS Central Science 4 (9): 1134\u20131145. doi:10.1021\/acscentsci.8b00307. ISSN 2374-7943. PMC PMC6161047. PMID 30276246. https:\/\/pubs.acs.org\/doi\/10.1021\/acscentsci.8b00307 .   \n \n\n\u2191 37.0 37.1 H\u00e4se, Florian; Aldeghi, Matteo; Hickman, Riley J.; Roch, Lo\u00efc M.; Aspuru-Guzik, Al\u00e1n (1 September 2021). \"G ryffin : An algorithm for Bayesian optimization of categorical variables informed by expert knowledge\" (in en). Applied Physics Reviews 8 (3): 031406. doi:10.1063\/5.0048164. ISSN 1931-9401. https:\/\/pubs.aip.org\/apr\/article\/8\/3\/031406\/998861\/Gryffin-An-algorithm-for-Bayesian-optimization-of .   \n \n\n\u2191 Reizman, Brandon J.; Wang, Yi-Ming; Buchwald, Stephen L.; Jensen, Klavs F. (2016). \"Suzuki\u2013Miyaura cross-coupling optimization enabled by automated feedback\" (in en). Reaction Chemistry & Engineering 1 (6): 658\u2013666. doi:10.1039\/C6RE00153J. ISSN 2058-9883. PMC PMC5123644. PMID 27928513. http:\/\/xlink.rsc.org\/?DOI=C6RE00153J .   \n \n\n\u2191 Baumgartner, Lorenz M.; Coley, Connor W.; Reizman, Brandon J.; Gao, Kevin W.; Jensen, Klavs F. (2018). \"Optimum catalyst selection over continuous and discrete process variables with a single droplet microfluidic reaction platform\" (in en). Reaction Chemistry & Engineering 3 (3): 301\u2013311. doi:10.1039\/C8RE00032H. ISSN 2058-9883. http:\/\/xlink.rsc.org\/?DOI=C8RE00032H .   \n \n\n\u2191 B\u00e9dard, Anne-Catherine; Adamo, Andrea; Aroh, Kosi C.; Russell, M. Grace; Bedermann, Aaron A.; Torosian, Jeremy; Yue, Brian; Jensen, Klavs F. et al. (21 September 2018). \"Reconfigurable system for automated optimization of diverse chemical reactions\" (in en). Science 361 (6408): 1220\u20131225. doi:10.1126\/science.aat0650. ISSN 0036-8075. https:\/\/www.science.org\/doi\/10.1126\/science.aat0650 .   \n \n\n\u2191 Granda, Jaros\u0142aw M.; Donina, Liva; Dragone, Vincenza; Long, De-Liang; Cronin, Leroy (1 July 2018). \"Controlling an organic synthesis robot with machine learning to search for new reactivity\" (in en). Nature 559 (7714): 377\u2013381. doi:10.1038\/s41586-018-0307-8. ISSN 0028-0836. PMC PMC6223543. PMID 30022133. https:\/\/www.nature.com\/articles\/s41586-018-0307-8 .   \n \n\n\u2191 Asche, Silke; Cooper, Geoffrey J. T.; Keenan, Graham; Mathis, Cole; Cronin, Leroy (10 June 2021). \"A robotic prebiotic chemist probes long term reactions of complexifying mixtures\" (in en). Nature Communications 12 (1): 3547. doi:10.1038\/s41467-021-23828-z. ISSN 2041-1723. PMC PMC8192940. PMID 34112788. https:\/\/www.nature.com\/articles\/s41467-021-23828-z .   \n \n\n\u2191 Bornemann\u2010Pfeiffer, Martin; Wolf, Jakob; Meyer, Klas; Kern, Simon; Angelone, Davide; Leonov, Artem; Cronin, Leroy; Emmerling, Franziska (18 October 2021). \"Standardization and Control of Grignard Reactions in a Universal Chemical Synthesis Machine using online NMR\" (in en). Angewandte Chemie International Edition 60 (43): 23202\u201323206. doi:10.1002\/anie.202106323. ISSN 1433-7851. PMC PMC8597166. PMID 34278673. https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/anie.202106323 .   \n \n\n\u2191 MacLeod, Benjamin P.; Parlane, Fraser G. L.; Rupnow, Connor C.; Dettelbach, Kevan E.; Elliott, Michael S.; Morrissey, Thomas D.; Haley, Ted H.; Proskurin, Oleksii et al. (22 February 2022). \"A self-driving laboratory advances the Pareto front for material properties\" (in en). Nature Communications 13 (1): 995. doi:10.1038\/s41467-022-28580-6. ISSN 2041-1723. PMC PMC8863835. PMID 35194074. https:\/\/www.nature.com\/articles\/s41467-022-28580-6 .   \n \n\n\u2191 Wu, Tony C; Aguilar\u2010Granda, Andr\u00e9s; Hotta, Kazuhiro; Yazdani, Sahar Alasvand; Pollice, Robert; Vestfrid, Jenya; Hao, Han; Lavigne, Cyrille et al. (1 February 2023). \"A Materials Acceleration Platform for Organic Laser Discovery\" (in en). Advanced Materials 35 (6): 2207070. doi:10.1002\/adma.202207070. ISSN 0935-9648. https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/adma.202207070 .   \n \n\n\u2191 Epps, Robert W.; Bowen, Michael S.; Volk, Amanda A.; Abdel\u2010Latif, Kameel; Han, Suyong; Reyes, Kristofer G.; Amassian, Aram; Abolhasani, Milad (1 July 2020). \"Artificial Chemist: An Autonomous Quantum Dot Synthesis Bot\" (in en). Advanced Materials 32 (30): 2001626. doi:10.1002\/adma.202001626. ISSN 0935-9648. https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/adma.202001626 .   \n \n\n\u2191 Abdel-Latif, Kameel; Epps, Robert W.; Bateni, Fazel; Han, Suyong; Reyes, Kristofer G.; Abolhasani, Milad (1 February 2021). \"Self\u2010Driven Multistep Quantum Dot Synthesis Enabled by Autonomous Robotic Experimentation in Flow\" (in en). Advanced Intelligent Systems 3 (2): 2000245. doi:10.1002\/aisy.202000245. ISSN 2640-4567. https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/aisy.202000245 .   \n \n\n\u2191 Mekki-Berrada, Flore; Ren, Zekun; Huang, Tan; Wong, Wai Kuan; Zheng, Fang; Xie, Jiaxun; Tian, Isaac Parker Siyu; Jayavelu, Senthilnath et al. (20 April 2021). \"Two-step machine learning enables optimized nanoparticle synthesis\" (in en). npj Computational Materials 7 (1): 55. doi:10.1038\/s41524-021-00520-w. ISSN 2057-3960. https:\/\/www.nature.com\/articles\/s41524-021-00520-w .   \n \n\n\u2191 Li, Jiagen; Lu, Yihua; Xu, Yanheng; Liu, Chongfeng; Tu, Yuxiao; Ye, Shuqian; Liu, Haochen; Xie, Yi et al. (21 November 2018). \"AIR-Chem: Authentic Intelligent Robotics for Chemistry\" (in en). The Journal of Physical Chemistry A 122 (46): 9142\u20139148. doi:10.1021\/acs.jpca.8b10680. ISSN 1089-5639. https:\/\/pubs.acs.org\/doi\/10.1021\/acs.jpca.8b10680 .   \n \n\n\u2191 Li, Jiagen; Li, Junzi; Liu, Rulin; Tu, Yuxiao; Li, Yiwen; Cheng, Jiaji; He, Tingchao; Zhu, Xi (27 April 2020). \"Autonomous discovery of optically active chiral inorganic perovskite nanocrystals through an intelligent cloud lab\" (in en). Nature Communications 11 (1): 2046. doi:10.1038\/s41467-020-15728-5. ISSN 2041-1723. PMC PMC7184584. PMID 32341340. https:\/\/www.nature.com\/articles\/s41467-020-15728-5 .   \n \n\n\u2191 von Drigalski, Felix; Tanaka, Kazutoshi; Hamaya, Masashi; Lee, Robert; Nakashima, Chisato; Shibata, Yoshiya; Ijiri, Yoshihisa (24 October 2020). \"A Compact, Cable-driven, Activatable Soft Wrist with Six Degrees of Freedom for Assembly Tasks\". 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) (Las Vegas, NV, USA: IEEE): 8752\u20138757. doi:10.1109\/IROS45743.2020.9341487. ISBN 978-1-7281-6212-6. https:\/\/ieeexplore.ieee.org\/document\/9341487\/ .   \n \n\n\u2191 Hamaya, Masashi; Lee, Robert; Tanaka, Kazutoshi; von Drigalski, Felix; Nakashima, Chisato; Shibata, Yoshiya; Ijiri, Yoshihisa (1 May 2020). \"Learning Robotic Assembly Tasks with Lower Dimensional Systems by Leveraging Physical Softness and Environmental Constraints\". 2020 IEEE International Conference on Robotics and Automation (ICRA) (Paris, France: IEEE): 7747\u20137753. doi:10.1109\/ICRA40945.2020.9197327. ISBN 978-1-7281-7395-5. https:\/\/ieeexplore.ieee.org\/document\/9197327\/ .   \n \n\n\u2191 Ge, M.; Su, F.; Zhao, Z.; Su, D. (1 August 2020). \"Deep learning analysis on microscopic imaging in materials science\" (in en). Materials Today Nano 11: 100087. doi:10.1016\/j.mtnano.2020.100087. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S258884202030016X .   \n \n\n\u2191 Suzuki, Yuta; Hino, Hideitsu; Kotsugi, Masato; Ono, Kanta (29 March 2019). \"Automated estimation of materials parameter from X-ray absorption and electron energy-loss spectra with similarity measures\" (in en). npj Computational Materials 5 (1): 39. doi:10.1038\/s41524-019-0176-1. ISSN 2057-3960. https:\/\/www.nature.com\/articles\/s41524-019-0176-1 .   \n \n\n\u2191 Tshitoyan, Vahe; Dagdelen, John; Weston, Leigh; Dunn, Alexander; Rong, Ziqin; Kononova, Olga; Persson, Kristin A.; Ceder, Gerbrand et al. (1 July 2019). \"Unsupervised word embeddings capture latent knowledge from materials science literature\" (in en). Nature 571 (7763): 95\u201398. doi:10.1038\/s41586-019-1335-8. ISSN 0028-0836. https:\/\/www.nature.com\/articles\/s41586-019-1335-8 .   \n \n\n\u2191 Abbott, Ryan (2016). \"I Think, Therefore I Invent: Creative Computers and the Future of Patent Law\" (in en). SSRN Electronic Journal. doi:10.2139\/ssrn.2727884. ISSN 1556-5068. http:\/\/www.ssrn.com\/abstract=2727884 .   \n \n\n\u2191 Chen, A. (8 January 2020). \"Can an AI be an inventor? Not yet.\". MIT Technology Review. Massachusetts Institute of Technology. https:\/\/www.technologyreview.com\/2020\/01\/08\/102298\/ai-inventor-patent-dabus-intellectual-property-uk-european-patent-office-law\/ . Retrieved 19 November 2022 .   \n \n\n\u2191 58.0 58.1 Carlson, Erika K. (1 November 2020). \"Artificial Intelligence Can Invent But Not Patent\u2014For Now\" (in en). Engineering 6 (11): 1212\u20131213. doi:10.1016\/j.eng.2020.09.003. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S209580992030254X .   \n \n\n\u2191 59.0 59.1 Abbott, R.; Austin, R.; Davidson, J.R. et al. (2022). \"The Artificial Inventor Project\". https:\/\/artificialinventor.com\/ . Retrieved 19 November 2022 .   \n \n\n\u2191 DABUS (2021). \"ZA 2021\/03242 - Food Container and Devices and Methods for Attracting Enhanced Attention\" (PDF). Patent Journal 54 (7): 255. ISSN 2223-4837. https:\/\/iponline.cipc.co.za\/Publications\/PublishedJournals\/E_Journal_July%202021%20Part%202.pdf .   \n \n\n\u2191 \"Commissioner of Patents v Thaler [2022 FCAFC 62\"]. Federal Court of Australia. 13 April 2022. https:\/\/www.judgments.fedcourt.gov.au\/judgments\/Judgments\/fca\/full\/2022\/2022fcafc0062 . Retrieved 19 November 2022 .   \n \n\n\u2191 \"AI cannot be named as inventor on patent applications\". European Patent Office. 21 December 2021. https:\/\/www.epo.org\/news-events\/news\/2021\/20211221.html . Retrieved 19 November 2022 .   \n \n\n\u2191 \"\u767a\u660e\u8005\u7b49\u306e\u8868\u793a\u306b\u3064\u3044\u3066\". Japan Patent Office. 30 July 2021. https:\/\/www.jpo.go.jp\/system\/process\/shutugan\/hatsumei.html . Retrieved 19 November 2022 .   \n \n\n\u2191 \"\u65e5\u672c\u306b\u304a\u3051\u308b\u767a\u660e\u8005\u306e\u6c7a\u5b9a\" (PDF). Japan Patent Office. https:\/\/www.jpo.go.jp\/resources\/shingikai\/sangyo-kouzou\/shousai\/tokkyo_shoi\/document\/seisakubukai-06-shiryou\/paper07_1.pdf . Retrieved 19 November 2022 .   \n \n\n\u2191 Kageyama, K. (1 October 2010). \"Formation of invention\/joint invention and recognition of inventor\/joint inventor\" (in en). Journal of Intellectual Property Law & Practice 5 (10): 699\u2013712. doi:10.1093\/jiplp\/jpq097. ISSN 1747-1532. https:\/\/academic.oup.com\/jiplp\/article-lookup\/doi\/10.1093\/jiplp\/jpq097 .   \n \n\n\u2191 \"Section 2 - Inventive Step\" (PDF). Japan Patent Office. https:\/\/www.jpo.go.jp\/e\/system\/laws\/rule\/guideline\/patent\/tukujitu_kijun\/document\/index\/03_0202_e.pdf . Retrieved 19 November 2022 .   \n \n\n\u2191 Nakayama, I. (2019). \"AI and Inventive Step \u2013 Proposal of Issues\". Patent 72 (12): 179\u201399.   \n \n\n\u2191 Abbott, Ryan (2017). \"Everything is Obvious\" (in en). SSRN Electronic Journal. doi:10.2139\/ssrn.3056915. ISSN 1556-5068. https:\/\/www.ssrn.com\/abstract=3056915 .   \n \n\n\u2191 \"Artificial Intelligence Collides with Patent Law\". World Economic Forum. 20 April 2018. https:\/\/www.weforum.org\/whitepapers\/artificial-intelligence-collides-with-patent-law . Retrieved 19 November 2022 .   \n \n\n\u2191 \"CCIA Comments on WIPO draft issues paper on IP and AI\" (PDF). Computer & Communications Industry Association. 14 February 2020. https:\/\/www.wipo.int\/export\/sites\/www\/about-ip\/en\/artificial_intelligence\/call_for_comments\/pdf\/org_ccia.pdf . Retrieved 19 November 2022 .   \n \n\n\u2191 Ramalho, Ana (2018). \"Patentability of AI-Generated Inventions: Is a Reform of the Patent System Needed?\" (in en). SSRN Electronic Journal. doi:10.2139\/ssrn.3168703. ISSN 1556-5068. https:\/\/www.ssrn.com\/abstract=3168703 .   \n \n\n\u2191  COPE Discussion Document: Authorship. 1 June 2014. doi:10.24318\/cope.2019.3.3. https:\/\/publicationethics.org\/node\/34946 .   \n \n\n\u2191 \"Defining the Role of Authors and Contributors\". International Committee of Medical Journal Editors. http:\/\/www.icmje.org\/recommendations\/browse\/roles-and-responsibilities\/defining-the-role-of-authors-and-contributors.html . Retrieved 19 November 2022 .   \n \n\n\u2191 Generative Pre-trained Transformer, ChatGPT; Zhavoronkov, Alex (21 December 2022). \"Rapamycin in the context of Pascal\u2019s Wager: generative pre-trained transformer perspective\" (in en). Oncoscience 9: 82\u201384. doi:10.18632\/oncoscience.571. ISSN 2331-4737. PMC PMC9796173. PMID 36589923. https:\/\/www.oncoscience.us\/lookup\/doi\/10.18632\/oncoscience.571 .   \n \n\n\u2191 O\u2019Connor, Siobhan; ChatGPT (1 January 2023). \"Open artificial intelligence platforms in nursing education: Tools for academic progress or abuse?\" (in en). Nurse Education in Practice 66: 103537. doi:10.1016\/j.nepr.2022.103537. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1471595322002517 .   \n \n\n\u2191 Stokel-Walker, Chris (26 January 2023). \"ChatGPT listed as author on research papers: many scientists disapprove\" (in en). Nature 613 (7945): 620\u2013621. doi:10.1038\/d41586-023-00107-z. ISSN 0028-0836. https:\/\/www.nature.com\/articles\/d41586-023-00107-z .   \n \n\n\u2191 Parviainen, Jaana; Coeckelbergh, Mark (1 September 2021). \"The political choreography of the Sophia robot: beyond robot rights and citizenship to political performances for the social robotics market\" (in en). AI & SOCIETY 36 (3): 715\u2013724. doi:10.1007\/s00146-020-01104-w. ISSN 0951-5666. https:\/\/link.springer.com\/10.1007\/s00146-020-01104-w .   \n \n\n\u2191 \"Recommendation of the Council on Artificial Intelligence\". OECD Legal Instruments. OECD. 21 May 2019. https:\/\/legalinstruments.oecd.org\/en\/instruments\/OECD-LEGAL-0449 . Retrieved 19 November 2022 .   \n \n\n\nNotes \nThis presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added. In the original, there are multiple instances of citing research work using the last name of the last author listed, rather than the last name of the first author listed; this may have been a product of Japanese culture tending to read text from right to left. For this version, the last name of the first author was used to be consistent with research norms.\n\n\n\n\n\nSource: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.limswiki.org\/index.php\/Journal:Autonomous_experimental_systems_in_materials_science\">https:\/\/www.limswiki.org\/index.php\/Journal:Autonomous_experimental_systems_in_materials_science<\/a>\nCategories: LIMSwiki journal articles (added in 2023LIMSwiki journal articles (all)LIMSwiki journal articles on laboratory automationLIMSwiki journal articles on laboratory informaticsLIMSwiki journal articles on materials informaticsNavigation menuPage actionsJournalDiscussionView sourceHistoryPage actionsJournalDiscussionMoreToolsIn other languagesPersonal toolsLog inNavigationMain pageEncyclopedic articlesRecent changesRandom pageHelp about MediaWikiSearch\u00a0 ToolsWhat links hereRelated changesSpecial pagesPermanent linkPage informationPopular publications\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\nPrint\/exportCreate a bookDownload as PDFDownload as PDFDownload as Plain textPrintable version This page was last edited on 1 September 2023, at 21:17.Content is available under a Creative Commons Attribution-ShareAlike 4.0 International License unless otherwise noted.This page has been accessed 606 times.Privacy policyAbout LIMSWikiDisclaimers\n\n\n\n","537370f6a0e7e5345701b0b5a2fb2d41_html":"<body class=\"mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-206 ns-subject page-Journal_Autonomous_experimental_systems_in_materials_science rootpage-Journal_Autonomous_experimental_systems_in_materials_science skin-monobook action-view skin--responsive\"><div id=\"rdp-ebb-globalWrapper\"><div id=\"rdp-ebb-column-content\"><div id=\"rdp-ebb-content\" class=\"mw-body\" role=\"main\"><a id=\"rdp-ebb-top\"><\/a>\n<h1 id=\"rdp-ebb-firstHeading\" class=\"firstHeading\" lang=\"en\">Journal:Autonomous experimental systems in materials science<\/h1><div id=\"rdp-ebb-bodyContent\" class=\"mw-body-content\"><!-- start content --><div id=\"rdp-ebb-mw-content-text\" lang=\"en\" dir=\"ltr\" class=\"mw-content-ltr\"><div class=\"mw-parser-output\">\n\n\n<h2><span class=\"mw-headline\" id=\"Abstract\">Abstract<\/span><\/h2>\n<p>The emergence of <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory_automation\" title=\"Laboratory automation\" class=\"wiki-link\" data-key=\"0061880849aeaca05f8aa27ae171f331\">autonomous experimental systems<\/a> (AESs) integrating <a href=\"https:\/\/www.limswiki.org\/index.php\/Machine_learning\" title=\"Machine learning\" class=\"wiki-link\" data-key=\"79aab39cfa124c958cd1dbcab3dde122\">machine learning<\/a> (ML) and robots is ushering in a paradigm shift in <a href=\"https:\/\/www.limswiki.org\/index.php\/Materials_science\" title=\"Materials science\" class=\"wiki-link\" data-key=\"89f5ce5de41da20cf3a2144a5731d5e6\">materials science<\/a>. Using computer algorithms and robots to decide and perform all experimental steps, these systems require no human intervention. A current direction focuses on discovering unexpected materials and theories with unconventional <a href=\"https:\/\/www.limswiki.org\/index.php\/Research\" title=\"Research\" class=\"wiki-link\" data-key=\"409634fd90113f119362927fe222f549\">research<\/a> approaches. This article reviews the latest achievements and discusses the impact of AESs, which will fundamentally change the way we understand research. Moreover, as AESs continue to develop, the need to think about the role of human researchers becomes more pressing. While ML and robotics can free us from the repetitive aspects of research, we need to understand the strengths and limitations of ML and robots and focus on how humans can perform higher creativity. In addition, we also discuss inventorship and authorship in the era of autonomous systems.\n<\/p><p><b>Keywords<\/b>: autonomous experimental system, closed-loop, machine learning, robots, materials science, inventorship, authorship, human researcher, human\u2019s role\n<\/p><p><b>Graphic abstract<\/b>: <a href=\"https:\/\/www.limswiki.org\/index.php\/File:GA_Ishizuki_SciTechAdvMatMeth2023_3-1.jpg\" class=\"image wiki-link\" data-key=\"1138c381dea34043f73321bf7342759c\"><img alt=\"GA Ishizuki SciTechAdvMatMeth2023 3-1.jpg\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/9\/9b\/GA_Ishizuki_SciTechAdvMatMeth2023_3-1.jpg\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<h2><span class=\"mw-headline\" id=\"Introduction\">Introduction<\/span><\/h2>\n<p>The total number of all possible small organic molecules is estimated to be at least 10<sup>60<\/sup><sup id=\"rdp-ebb-cite_ref-1\" class=\"reference\"><a href=\"#cite_note-1\">[1]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-2\" class=\"reference\"><a href=\"#cite_note-2\">[2]<\/a><\/sup>, and from that one can imagine a similarly large number of possible materials being derived using those molecules. This number helps illustrate the vastness of the materials search space, which must contain many materials that can help address current societal problems. In a way, the world of materials is a frontier for exploration, much like space or the deep sea.\n<\/p><p>How can we quickly and systematically find unexpected materials within this enormous search space? To this end, <a href=\"https:\/\/www.limswiki.org\/index.php\/Materials_science\" title=\"Materials science\" class=\"wiki-link\" data-key=\"89f5ce5de41da20cf3a2144a5731d5e6\">materials science<\/a> needs a tool that can transcend the limits of human capabilities to serve as a materials explorer (Figure 1), akin to a spaceship or a deep-sea exploration vessel.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig1_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg\" class=\"image wiki-link\" data-key=\"4cef5b4be7cf5cc32463c7520886357b\"><img alt=\"Fig1 Ishizuki SciTechAdvMatMeth2023 3-1.jpeg\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/d\/d6\/Fig1_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 1.<\/b> The vision of the materials explorer. The exploration involves an autonomous experimental system (AES), materials informatics, and human <a href=\"https:\/\/www.limswiki.org\/index.php\/Research\" title=\"Research\" class=\"wiki-link\" data-key=\"409634fd90113f119362927fe222f549\">researchers<\/a>. The heart of the materials explorer is an AES based on machine learning and robots (green, orange, and blue). This system is imbued with the skills of experts and generates large amounts of experimental data that could not have been generated by human researchers (data-production factory). The data generated by the AES is then processed by machine learning (ML) and simulations to predict new materials (i.e., materials informatics). In addition, the system organizes the data and generates \"materials maps\" and models, facilitating knowledge creation by providing researchers a sharable big-picture view of unexpected materials, thereby accelerating materials development.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>The core of such a materials explorer is the <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory_automation\" title=\"Laboratory automation\" class=\"wiki-link\" data-key=\"0061880849aeaca05f8aa27ae171f331\">autonomous experimental system<\/a> (AES) based on <a href=\"https:\/\/www.limswiki.org\/index.php\/Machine_learning\" title=\"Machine learning\" class=\"wiki-link\" data-key=\"79aab39cfa124c958cd1dbcab3dde122\">machine learning<\/a> (ML) and robots (green, orange, and purple in Figure 1). Here, the term \"autonomous\" means that a computer algorithm decides the next experimental steps while robots perform all experimental steps. This approach, which involves no human intervention, is called the closed-loop experiment (Figure 2).\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig2_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg\" class=\"image wiki-link\" data-key=\"3d00d989f7f56369a669e55f4dce024f\"><img alt=\"Fig2 Ishizuki SciTechAdvMatMeth2023 3-1.jpeg\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/a\/a8\/Fig2_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 2.<\/b> The concept of an autonomous experimental system (AES). The system autonomously synthesizes materials with optimal physical characteristics without human intervention. Autonomy leads to significant improvements: 1) fully digitalized experiments transforms all experimental parameters, including process conditions, into data; 2) removal of human error makes reproducibility reliable; and 3) implicit knowledge can be digitalized and embedded.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>In general, new materials are sought in a multi-dimensional space by optimizing many relevant experimental parameters. Because of the vastness of the search space, the manual optimization of these parameters by individual researchers only produces incremental results that do not show the big picture. However, this problem is ideally suited for the AES to address. Figure 2 illustrates one such example. Here, based on the initial instructions, ML decides which compound to synthesize and feeds the corresponding directions to the robots; the robots synthesize, test, and report the results back to the algorithm, repeating the cycle until the desired result is obtained. This autonomous experimental approach drastically speeds up the materials exploration processes.\n<\/p><p>The concept of the materials explorer fundamentally changes the way we understand and conduct materials research, across three stages:\n<\/p><p><b>Stage 1<\/b>: Optimization of the yield of target substances - Here, the target compound is known, but the optimum synthesis conditions are unknown. The target compound is decided by the human researchers before the experiment, and the AES quickly optimizes the synthesis conditions of the target compound within the search space specified by the human researchers.<sup id=\"rdp-ebb-cite_ref-:0_3-0\" class=\"reference\"><a href=\"#cite_note-:0-3\">[3]<\/a><\/sup>\n<\/p><p><b>Stage 2<\/b>: Finding new materials with desired properties - Here, the target physical properties are decided, but the compound possessing these properties is unknown. The AES quickly finds the best material within the search space specified by the human researchers. In contrast to Stage 1, it is the composition of the materials that is changed to find new compounds to meet the required physical properties. Materials with a variety of crystal structures and hierarchical structures are also explored.\n<\/p><p><b>Stage 3<\/b>: Finding new materials or principles that no one has thought of before - Here, new materials, theories, and principles that are unexpected to researchers are discovered by combining the results from autonomous experiments, <a href=\"https:\/\/www.limswiki.org\/index.php\/Materials_informatics\" title=\"Materials informatics\" class=\"wiki-link\" data-key=\"4dd2125beb9794d0a679b921981f1ddc\">materials informatics<\/a>, and human researchers (Figure 1).<sup id=\"rdp-ebb-cite_ref-4\" class=\"reference\"><a href=\"#cite_note-4\">[4]<\/a><\/sup>\n<\/p><p>At present, the proofs of concepts of Stages 1 and 2 have been demonstrated in a variety of fields.<sup id=\"rdp-ebb-cite_ref-:1_5-0\" class=\"reference\"><a href=\"#cite_note-:1-5\">[5]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:2_6-0\" class=\"reference\"><a href=\"#cite_note-:2-6\">[6]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:3_7-0\" class=\"reference\"><a href=\"#cite_note-:3-7\">[7]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:4_8-0\" class=\"reference\"><a href=\"#cite_note-:4-8\">[8]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:5_9-0\" class=\"reference\"><a href=\"#cite_note-:5-9\">[9]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:6_10-0\" class=\"reference\"><a href=\"#cite_note-:6-10\">[10]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:7_11-0\" class=\"reference\"><a href=\"#cite_note-:7-11\">[11]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:8_12-0\" class=\"reference\"><a href=\"#cite_note-:8-12\">[12]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:9_13-0\" class=\"reference\"><a href=\"#cite_note-:9-13\">[13]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:10_14-0\" class=\"reference\"><a href=\"#cite_note-:10-14\">[14]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:11_15-0\" class=\"reference\"><a href=\"#cite_note-:11-15\">[15]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:12_16-0\" class=\"reference\"><a href=\"#cite_note-:12-16\">[16]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:13_17-0\" class=\"reference\"><a href=\"#cite_note-:13-17\">[17]<\/a><\/sup> Expanding the application of AESs to a variety of experiments has become the next objective. Meanwhile, Stage 3 is rapidly advancing owing to the development of ML, robotics, and materials informatics. In this stage, it is critical to embed the researchers\u2019 intuition and experience into the construction of theories. This inclusion of the human role is often referred to as having a \"human-in-the-loop\" or \"researcher-in-the-loop.\"\n<\/p><p>An important point to note is that the transformation resulting from the three stages will change the way researchers think, giving completely new perspectives that researchers cannot obtain using conventional research methods. In addition, the transformation is not limited to a single <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory\" title=\"Laboratory\" class=\"wiki-link\" data-key=\"c57fc5aac9e4abf31dccae81df664c33\">laboratory<\/a>; instead, it will change how we conduct research through a digital transformation of materials science. For example, experimentalists can remotely fabricate materials via the internet. In the same way, theorists can fabricate materials to test their predictions.\n<\/p><p>The data generated by such a system would fall into the domain of \"big data.\" Researchers use this big data to extract human-readable <a href=\"https:\/\/www.limswiki.org\/index.php\/Information\" title=\"Information\" class=\"wiki-link\" data-key=\"6300a14d9c2776dcca0999b5ed940e7d\">information<\/a> (i.e., via materials informatics). Because ML and robotics alone can neither find insights nor discover concepts in physics and chemistry, human researchers will always remain central to the research. The key point is that materials scientists must understand what ML and robotics can solve, and must set the right problem to be solved. The strength of human researchers lies in concept creation or problem identification in the larger context. Combining these strengths with ML and robotics is critical to accelerating research (i.e., the researcher-in-the-loop).\n<\/p><p>The process of autonomous materials exploration, however, raises one fundamental question: when experiments are performed autonomously and new materials are found without human intervention, who is the discoverer and inventor? This question not only relates to the authorship of papers and the inventorship of patents, but it also concerns researchers\u2019 motivations. It is important to address this discussion as the autonomous experimental approach develops.\n<\/p><p>In this article, we review the trends and the prospects of AESs in materials science, specifically through the following aspects:\n<\/p>\n<ul><li>the overall status of autonomous experiments in materials science;<\/li>\n<li>the history and the latest topics of autonomous materials synthesis;<\/li>\n<li>the future of research using the AES;<\/li>\n<li>some important take-home lessons we have learned when developing and using such a technology; and<\/li>\n<li>the future of authorship and inventorship.<\/li><\/ul>\n<p>Throughout this review, we address what human researchers should focus on in the era of autonomous research. The year 2020 was momentous for this era; there were significant advancements in the field of autonomous research. This review aims to further contribute to the expansion of the era of autonomous materials research.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Present_status_of_autonomous_experiments\">Present status of autonomous experiments<\/span><\/h2>\n<h3><span class=\"mw-headline\" id=\"A_brief_description_of_our_own_autonomous_experiments\">A brief description of our own autonomous experiments<\/span><\/h3>\n<p>In the closed-loop cycle shown in Figure 2, researchers only need to choose the material properties to optimize and provide the system with the necessary raw materials; the automatic system then takes control, repeatedly synthesizing and measuring the properties of new compounds until the best one is found. The ML algorithm uses previous knowledge to decide how the synthesis conditions should be changed to approach the desired outcome with each cycle.\n<\/p><p>Recently, we have developed an autonomous synthesis of inorganic thin films.<sup id=\"rdp-ebb-cite_ref-:0_3-1\" class=\"reference\"><a href=\"#cite_note-:0-3\">[3]<\/a><\/sup> In this proof-of-concept study, we demonstrated the autonomous fabrication of TiO<sub>2<\/sub> thin films with low resistance and showed that this system accelerates experiments by tenfold.\n<\/p><p>To obtain these results, we have used robotic modules of a sputter deposition apparatus and a robotic device for measuring resistance. Other modules with robotic synthesis and measurement equipment can be connected to this system to adapt to the desired research. The robotic arm transfers the samples from module to module as needed, and the Bayesian optimization algorithm predicts the synthesis parameters for the next iteration.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"AES_in_the_world\">AES in the world<\/span><\/h3>\n<p>Recent years have witnessed the rapid progress of autonomous experiments, including a) development of ML technology, b) improvement of robot technology and expansion of its range of applications, and c) realization of autonomous experiments using ML and robots. As discussed in the introduction, examples of autonomous materials syntheses can be found to demonstrate Stages 1 and 2.<sup id=\"rdp-ebb-cite_ref-:1_5-1\" class=\"reference\"><a href=\"#cite_note-:1-5\">[5]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:2_6-1\" class=\"reference\"><a href=\"#cite_note-:2-6\">[6]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:3_7-1\" class=\"reference\"><a href=\"#cite_note-:3-7\">[7]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:4_8-1\" class=\"reference\"><a href=\"#cite_note-:4-8\">[8]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:5_9-1\" class=\"reference\"><a href=\"#cite_note-:5-9\">[9]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:6_10-1\" class=\"reference\"><a href=\"#cite_note-:6-10\">[10]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:7_11-1\" class=\"reference\"><a href=\"#cite_note-:7-11\">[11]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:8_12-1\" class=\"reference\"><a href=\"#cite_note-:8-12\">[12]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:9_13-1\" class=\"reference\"><a href=\"#cite_note-:9-13\">[13]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:10_14-1\" class=\"reference\"><a href=\"#cite_note-:10-14\">[14]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:11_15-1\" class=\"reference\"><a href=\"#cite_note-:11-15\">[15]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:12_16-1\" class=\"reference\"><a href=\"#cite_note-:12-16\">[16]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:13_17-1\" class=\"reference\"><a href=\"#cite_note-:13-17\">[17]<\/a><\/sup> In this section, we review the history and the latest advances demonstrating these two stages (Table 1).\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"4\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 1.<\/b> Examples of autonomous materials synthesis using robots.\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Research group (year)\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Robot mechanism\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Optimization objective\/Synthesized material\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Variables\/Algorithm\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Matsuda <i>et al.<\/i> (1988)<sup id=\"rdp-ebb-cite_ref-:14_18-0\" class=\"reference\"><a href=\"#cite_note-:14-18\">[18]<\/a><\/sup>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Robotic arm to manipulate the test tube\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Maximizing color-developing reactions for analysis (Stage 1)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Optimizing the amount of reagent, reaction time, etc. using the simplex method\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Nikolaev <i>et al.<\/i> (2016)<sup id=\"rdp-ebb-cite_ref-:15_19-0\" class=\"reference\"><a href=\"#cite_note-:15-19\">[19]<\/a><\/sup>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Laser to perform both heating and spectroscopy\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Carbon nanotubes with maximized growth rate (Stage 1)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Optimizing temperature, pressure, and gas composition using a genetic algorithm\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Christensen <i>et al.<\/i> (2021)<sup id=\"rdp-ebb-cite_ref-:16_20-0\" class=\"reference\"><a href=\"#cite_note-:16-20\">[20]<\/a><\/sup>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Liquid handling robot\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Maximizing the yield of Suzuki-Miyaura coupling reaction (Stage 1)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Optimizing continuous variables (amount of ingredients, etc.) and categorical variables (catalyst type) using Bayesian optimization\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">McMullen <i>et al<\/i> (2010)<sup id=\"rdp-ebb-cite_ref-:17_21-0\" class=\"reference\"><a href=\"#cite_note-:17-21\">[21]<\/a><\/sup>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Flow reactor (synthesis in liquid phase)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Maximizing the yield of Heck reaction (Stage 1)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Optimizing temperature and raw materials ratio using simplex method\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Sans <i>et al.<\/i> (2015)<sup id=\"rdp-ebb-cite_ref-:18_22-0\" class=\"reference\"><a href=\"#cite_note-:18-22\">[22]<\/a><\/sup>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Flow reactor\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Maximizing the yield of imine synthesis (Stage 1)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Optimizing temperature and raw materials ratio using simplex method\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">King <i>et al.<\/i> (2009)<sup id=\"rdp-ebb-cite_ref-:19_23-0\" class=\"reference\"><a href=\"#cite_note-:19-23\">[23]<\/a><\/sup> and King (2011)<sup id=\"rdp-ebb-cite_ref-:20_24-0\" class=\"reference\"><a href=\"#cite_note-:20-24\">[24]<\/a><\/sup>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Liquid handling robot, a robotic arm, etc.\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Discovering yeast genes (Stage 2)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Generating hypotheses and experimentally testing these hypotheses\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Burger <i>et al.<\/i> (2020)<sup id=\"rdp-ebb-cite_ref-:21_25-0\" class=\"reference\"><a href=\"#cite_note-:21-25\">[25]<\/a><\/sup>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Free-roaming robot\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Photocatalyst mixtures with maximized photocatalytic activity (Stage 2)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Optimizing the concentration of photocatalyst and additives using Bayesian optimization\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">MacLeod <i>et al.<\/i> (2020)<sup id=\"rdp-ebb-cite_ref-:22_26-0\" class=\"reference\"><a href=\"#cite_note-:22-26\">[26]<\/a><\/sup>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">A gripper, a pipette mount, and a robotic arm\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Organic hole transport materials with maximized hole mobility (Stage 2)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Optimizing annealing times and dopant concentrations using Bayesian optimization\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Krishnadasan <i>et al.<\/i> (2007)<sup id=\"rdp-ebb-cite_ref-:23_27-0\" class=\"reference\"><a href=\"#cite_note-:23-27\">[27]<\/a><\/sup>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Flow reactor\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">CdSe nanoparticles with targeted spectroscopic characteristics (Stage 2)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Optimizing temperature and precursor concentrations using SNOBFIT method\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Tao <i>et al.<\/i> (2021)<sup id=\"rdp-ebb-cite_ref-:24_28-0\" class=\"reference\"><a href=\"#cite_note-:24-28\">[28]<\/a><\/sup>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Flow reactor\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Au nanoparticles with targeted spectroscopic characteristics (Stage 2)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Optimizing reaction time and precursor concentrations using Bayesian optimization\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Desai <i>et al.<\/i> (2013)<sup id=\"rdp-ebb-cite_ref-:25_29-0\" class=\"reference\"><a href=\"#cite_note-:25-29\">[29]<\/a><\/sup>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Flow reactor\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Abl kinase inhibitors with maximum activity (Stage 2)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Synthesizing from 27\u2009\u00d7\u200910 row materials using random forest\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Dave <i>et al.<\/i> (2020)<sup id=\"rdp-ebb-cite_ref-:26_30-0\" class=\"reference\"><a href=\"#cite_note-:26-30\">[30]<\/a><\/sup>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Flow reactor\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Aqueous electrolytes with maximum electrochemical window (Stage 2)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Optimizing each precursor solution volume using Bayesian optimization\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Shimizu <i>et al.<\/i> (2020)<sup id=\"rdp-ebb-cite_ref-:0_3-2\" class=\"reference\"><a href=\"#cite_note-:0-3\">[3]<\/a><\/sup>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">A robot arm, a sputtering system, a resistance meter\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Nb-doped TiO<sub>2<\/sub> thin film with minimized resistance (Stage 2)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Optimizing oxygen partial pressure using Bayesian optimization\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h4><span class=\"mw-headline\" id=\"Stage_1:_Optimization_of_the_yield_of_target_substances_or_reaction_conditions\">Stage 1: Optimization of the yield of target substances or reaction conditions<\/span><\/h4>\n<p>The idea of autonomous materials synthesis using robots has a long history. The first proposal of a fully automated closed-loop robot aiming for the optimization of chemical reaction parameters was published in 1978.<sup id=\"rdp-ebb-cite_ref-31\" class=\"reference\"><a href=\"#cite_note-31\">[31]<\/a><\/sup> However, no experimental results were reported.\n<\/p><p>In the 1980s, Matsuda <i>et al.<\/i> reported the optimization of reaction conditions using an autonomous system.<sup id=\"rdp-ebb-cite_ref-:14_18-1\" class=\"reference\"><a href=\"#cite_note-:14-18\">[18]<\/a><\/sup> The repetition of reagent adjustment, reaction, measurement, and prediction of the next experimental conditions based on the measured results were automatically performed. Here, test tubes were manipulated by a robot. The color-developing reactions for chemical analysis were optimized using the simplex method with three parameters: the amount of two reagents and the reaction time. The exhaustive grid search required 130 experiments, but the robotic system optimized the reaction in less than 28 experiments. While the purpose of this experiment was not to find a new compound but to optimize the color-developing reaction, it is a seminal pioneering work of autonomous experiments.\n<\/p><p>In 2016, Nikolaev <i>et al.<\/i> demonstrated an autonomous synthesis of inorganic materials. The authors synthesized carbon nanotubes using chemical vapor deposition.<sup id=\"rdp-ebb-cite_ref-:15_19-1\" class=\"reference\"><a href=\"#cite_note-:15-19\">[19]<\/a><\/sup> They fabricated multiple columns containing a catalyst layer on a substrate in advance. Then, they heated the individual columns one by one with a laser while repeatedly moving the substrate to synthesize carbon nanotubes with different growth conditions. This heating laser was also used as an excitation source for <a href=\"https:\/\/www.limswiki.org\/index.php\/Raman_spectroscopy\" title=\"Raman spectroscopy\" class=\"wiki-link\" data-key=\"60e69470fcd47644f07a6969414597ea\">Raman spectroscopy<\/a> to observe the growth rate <i>in situ<\/i>. A genetic algorithm maximized the growth rate; the system optimized the temperature, pressure, and gas composition. Later the group utilized a Bayesian optimization as an optimization algorithm.<sup id=\"rdp-ebb-cite_ref-32\" class=\"reference\"><a href=\"#cite_note-32\">[32]<\/a><\/sup>\n<\/p><p>For organics, Christensen <i>et al.<\/i> maximized the yield of the Suzuki-Miyaura coupling reaction using Bayesian optimization.<sup id=\"rdp-ebb-cite_ref-:16_20-1\" class=\"reference\"><a href=\"#cite_note-:16-20\">[20]<\/a><\/sup> They used a commercially available liquid handling robot and ChemOS<sup id=\"rdp-ebb-cite_ref-33\" class=\"reference\"><a href=\"#cite_note-33\">[33]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-34\" class=\"reference\"><a href=\"#cite_note-34\">[34]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-35\" class=\"reference\"><a href=\"#cite_note-35\">[35]<\/a><\/sup> for autonomous experiments. In the Bayesian optimization, Phoenics<sup id=\"rdp-ebb-cite_ref-36\" class=\"reference\"><a href=\"#cite_note-36\">[36]<\/a><\/sup> and Gryffin<sup id=\"rdp-ebb-cite_ref-:27_37-0\" class=\"reference\"><a href=\"#cite_note-:27-37\">[37]<\/a><\/sup> algorithms were used to optimize categorical variables (catalyst type) in addition to continuous variables (amount of catalyst, amount of feedstock, reaction temperature). Parallel autonomous process optimization experiments in batches were performed to shorten the time to complete the optimization.\n<\/p><p>The autonomous synthesis of organics using flow reactors was reported by McMullen <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-:17_21-1\" class=\"reference\"><a href=\"#cite_note-:17-21\">[21]<\/a><\/sup> The authors first reported on a Heck reaction, where the yield was maximized by optimizing the raw materials ratio and reaction time as independent variables, using the simplex method (Nelder-Mead method). Subsequently, the authors worked on other autonomous syntheses using flow reactors<sup id=\"rdp-ebb-cite_ref-38\" class=\"reference\"><a href=\"#cite_note-38\">[38]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-39\" class=\"reference\"><a href=\"#cite_note-39\">[39]<\/a><\/sup>. For example, they applied SNOBFIT (Stable Noisy Optimization by Branch and Fit) to a variety of reactions using a modular system.<sup id=\"rdp-ebb-cite_ref-40\" class=\"reference\"><a href=\"#cite_note-40\">[40]<\/a><\/sup> In 2015, Sans <i>et al.<\/i> optimized the yield of an imine synthesis.<sup id=\"rdp-ebb-cite_ref-:18_22-1\" class=\"reference\"><a href=\"#cite_note-:18-22\">[22]<\/a><\/sup> The authors also used the simplex method to tune the raw materials ratio and reaction time. The system was equipped with in-line <a href=\"https:\/\/www.limswiki.org\/index.php\/Nuclear_magnetic_resonance_spectroscopy\" title=\"Nuclear magnetic resonance spectroscopy\" class=\"wiki-link\" data-key=\"a05c6a4eb8775761248c099371cdb82f\">nuclear magnetic resonance spectroscopy<\/a>. The group also developed an organic synthesis robot that navigates chemical reaction spaces.<sup id=\"rdp-ebb-cite_ref-41\" class=\"reference\"><a href=\"#cite_note-41\">[41]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-42\" class=\"reference\"><a href=\"#cite_note-42\">[42]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-43\" class=\"reference\"><a href=\"#cite_note-43\">[43]<\/a><\/sup>\n<\/p>\n<h4><span class=\"mw-headline\" id=\"Stage_2:_Finding_new_materials_with_desired_properties\">Stage 2: Finding new materials with desired properties<\/span><\/h4>\n<p>In 2009, King <i>et al.<\/i> reported a seminal work<sup id=\"rdp-ebb-cite_ref-:19_23-1\" class=\"reference\"><a href=\"#cite_note-:19-23\">[23]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:20_24-1\" class=\"reference\"><a href=\"#cite_note-:20-24\">[24]<\/a><\/sup>, where a robot \"Adam\" autonomously generated functional genomics hypotheses and experimentally tested the hypotheses using robots. The system measured the growth curves of selected microbial strains growing in defined media, \"discovering\" three novel yeast genes. The equipment comprised a liquid handling robot, a robot arm, and an incubator, all fixed in one place.\n<\/p><p>In 2020, Burger <i>et al.<\/i> demonstrated a free-roaming robot that moved around the <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory\" title=\"Laboratory\" class=\"wiki-link\" data-key=\"c57fc5aac9e4abf31dccae81df664c33\">laboratory<\/a> to perform autonomous experiments using the same equipment as those used by its human counterparts.<sup id=\"rdp-ebb-cite_ref-:21_25-1\" class=\"reference\"><a href=\"#cite_note-:21-25\">[25]<\/a><\/sup> The robot aimed to maximize photocatalytic activity by optimizing the concentrations of photocatalyst and additives. Based on Bayesian optimization, the robot identified the photocatalyst mixtures that were six times more active than the initial formulation. The robot completed 688 experiments in eight\u2009days. This number of experiments would take a human researcher several months. For reagent weighing, the robot handled both powder and liquid materials.\n<\/p><p>In the same year, MacLeod <i>et al.<\/i> reported the autonomous fabrication of organic thin films.<sup id=\"rdp-ebb-cite_ref-:22_26-1\" class=\"reference\"><a href=\"#cite_note-:22-26\">[26]<\/a><\/sup> A robot \"Ada,\" equipped with a gripper, a pipette mount, and a robotic arm for liquid injection and substrate transfer, maximized the hole mobility of organic hole transport materials used in perovskite solar cells. The dopant concentrations and annealing time were optimized using Bayesian optimization. Ada finished the experiment in five days instead of nine months. Later, the group used Ada to define a Pareto front of conductivities and processing temperatures for palladium films formed by combustion synthesis.<sup id=\"rdp-ebb-cite_ref-44\" class=\"reference\"><a href=\"#cite_note-44\">[44]<\/a><\/sup> There are other liquid handling robots developed for closed-loop, e.g., Wu <i>et al.<\/i> reported an automated platform for organic laser discovery, including not only synthesis and compound identification but also integrated target property characterization.<sup id=\"rdp-ebb-cite_ref-45\" class=\"reference\"><a href=\"#cite_note-45\">[45]<\/a><\/sup>\n<\/p><p>There are several autonomous experiments based on flow reactors (synthesis in the liquid phase). For inorganic nanoparticles, Krishnadasan <i>et al.<\/i> autonomously synthesized CdSe nanoparticles with optimized intensity for a chosen emission wavelength in 2007 (SNOBFIT method).<sup id=\"rdp-ebb-cite_ref-:23_27-1\" class=\"reference\"><a href=\"#cite_note-:23-27\">[27]<\/a><\/sup> Later, other groups reported various autonomous experiments on inorganic nanoparticles.<sup id=\"rdp-ebb-cite_ref-46\" class=\"reference\"><a href=\"#cite_note-46\">[46]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-47\" class=\"reference\"><a href=\"#cite_note-47\">[47]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-48\" class=\"reference\"><a href=\"#cite_note-48\">[48]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-49\" class=\"reference\"><a href=\"#cite_note-49\">[49]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-50\" class=\"reference\"><a href=\"#cite_note-50\">[50]<\/a><\/sup> For example, Tao <i>et al.<\/i> synthesized Au nanoparticles<sup id=\"rdp-ebb-cite_ref-:24_28-1\" class=\"reference\"><a href=\"#cite_note-:24-28\">[28]<\/a><\/sup> with targeted spectroscopic characteristics using the above-mentioned Gryffin<sup id=\"rdp-ebb-cite_ref-:27_37-1\" class=\"reference\"><a href=\"#cite_note-:27-37\">[37]<\/a><\/sup> in 2021.\n<\/p><p>For organic compounds, Desai <i>et al.<\/i> synthesized Abl kinase inhibitors with maximum activity in 2013.<sup id=\"rdp-ebb-cite_ref-:25_29-1\" class=\"reference\"><a href=\"#cite_note-:25-29\">[29]<\/a><\/sup> Out of 270 possible combinations (10 types\u2009\u00d7\u200927 types) from the raw materials, the system found the novel Abl kinase inhibitor after synthesizing only 21 compounds. The prediction model used random forest regression to handle the types of raw materials with different structures.\n<\/p><p>Dave <i>et al.<\/i> reported isolating aqueous electrolytes with the maximum electrochemical window in 2020.<sup id=\"rdp-ebb-cite_ref-:26_30-1\" class=\"reference\"><a href=\"#cite_note-:26-30\">[30]<\/a><\/sup> The authors used Bayesian optimization to optimize the solution volume of three aqueous Li salts or four aqueous Na salts. The results indicated that non-smooth chemical responses are observed along the axis of the amount of NaBr. The authors pointed out that log-scaling, which varies rapidly with quantity, such as the amount of NaBr, gives the response surface both smaller gradients along this axis and a much-improved performance because Gaussian process regression requires an assumption of smoothness on the response surface. They also pointed out that the selection and presentation of the design space for materials search are of utmost importance to autonomous task design.\n<\/p>\n<h4><span class=\"mw-headline\" id=\"Future_of_Stages_1_and_2:_Autonomously_search_within_the_preset_search_space\">Future of Stages 1 and 2: Autonomously search within the preset search space<\/span><\/h4>\n<p>Currently, Stages 1 and 2 are not widespread because the range of applicable experiments is still limited. As more sophisticated robots are developed, the application of autonomous research will expand in turn. The problem lies in that conventional robots are rigidly designed for precision, aiming for zero error. Because they are programmed based on if-then statements, these robots cannot respond flexibly to changing situations. Therefore, it is difficult to implement the tacit knowledge of skilled human researchers in robot systems. With soft control, soft structure, multiple sensing devices, and ML, a more flexible robot can absorb errors and disruptions, and can operate without stoppage or breakdowns.<sup id=\"rdp-ebb-cite_ref-51\" class=\"reference\"><a href=\"#cite_note-51\">[51]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-52\" class=\"reference\"><a href=\"#cite_note-52\">[52]<\/a><\/sup> Thus, developing robot technology that can respond flexibly until the goal is achieved is important. To this end, it is essential to develop new techniques for robots to learn the minute details of any required motions.\n<\/p><p>When versatile robots capable of performing a variety of experiments become affordable, the introduction of robots will rapidly expand into many research fields. At the time of this writing, the cost of robots is reducing rapidly to the point where labs can consider using such robot-experiment automation. However, more effort is needed to help materials scientists implement robots into their experiments.\n<\/p><p>We note that not all researchers will adopt this style of research. The conventional research approach\u2014which relies on researchers\u2019 experience, knowledge, and intuition\u2014will remain essential for conceiving new synthetic methods and analytical techniques. We expect there will be a hybrid between conventional and autonomous approaches.\n<\/p><p>AESs are not only useful for performing high-throughput experiments; the most critical aim is to help human researchers deepen their research. Human researchers need to think about how to analyze the collected data, how the new materials can be used, and what theories can be deduced from unexpected results. AESs free human researchers from repetitive tasks and enable researchers to conduct innovative research. Furthermore, combining the system with advanced materials prediction based on computer simulation is the step that enables human researchers to perform more creative work (Figure 3).\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig3_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg\" class=\"image wiki-link\" data-key=\"b5409f1fd6dbc4b81c559658087b7083\"><img alt=\"Fig3 Ishizuki SciTechAdvMatMeth2023 3-1.jpeg\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/e\/e3\/Fig3_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 3.<\/b> The human researchers will always play the leading role. The autonomous laboratory is for researchers. This technology makes it possible to narrow down candidate materials with high predictive performance and to pinpoint materials synthesis experiments. The researchers then work on highly creative tasks and acquire \"tacit knowledge\" of an even higher level. In turn, the new knowledge is digitized again. It is essential to repeat this cycle.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>With the implementation of AESs, theorists can test their original ideas. Once theorists predict a promising material, they can log on to the AES via the internet and synthesize the new material. Currently, we are developing a system capable of automatically synthesizing materials from simple instructions, such as \"synthesize AxByCz with a crystal structure of D.\" The system automatically measures X-ray diffraction patterns and feeds the results back to the algorithm to predict the next synthesis condition.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Stage_3:_Finding_materials_that_no_one_has_thought_of_before\">Stage 3: Finding materials that no one has thought of before<\/span><\/h2>\n<p>Because the AES will generate increasingly larger amounts of data, it is essential to combine this type of system with materials informatics and computer simulations to extend human thinking. In so doing, the generated data will promote the emergence of unexpected results that are beyond conventional theories and experiences. It is also important to embed the researchers\u2019 intuition and experience, which are tacit knowledge, into the autonomous system and share them with other researchers (Figure 4). \n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig4_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg\" class=\"image wiki-link\" data-key=\"27fbb8e847d77f1be8d8f80af67a20c7\"><img alt=\"Fig4 Ishizuki SciTechAdvMatMeth2023 3-1.jpeg\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/0\/0a\/Fig4_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 4.<\/b> A vision of how research will be conducted. In the conventional way of research, there has been a disruption between researchers and robots\/<a href=\"https:\/\/www.limswiki.org\/index.php\/Artificial_intelligence\" title=\"Artificial intelligence\" class=\"wiki-link\" data-key=\"0c45a597361ca47e1cd8112af676276e\">artificial intelligence<\/a> (AI). In the near future, researchers will develop materials based on an enormous amount of experimental and simulation results. In this \"researcher-in-the-loop\" process, human researchers will obtain new insights, a bird\u2019s eye view of materials through multifaceted materials characterization and resulting materials big data.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>The following are some of the remaining challenges.\n<\/p>\n<h3><span id=\"rdp-ebb-Increasing_the_amount_of_reliable_materials_data_(materials_checkup_system_and_measurement)\"><\/span><span class=\"mw-headline\" id=\"Increasing_the_amount_of_reliable_materials_data_.28materials_checkup_system_and_measurement.29\">Increasing the amount of reliable materials data (materials checkup system and measurement)<\/span><\/h3>\n<p>To date, materials informatics is still limited by the lack of a reliable materials experiment database. Such a database cannot be manually acquired. To this end, AESs enable the storage of all data, ranging from the fabrication processes (synthesis conditions and experimental environment) to the structural and composition information, to the measured materials properties. Even the negative data (data that do not show the desired values) are stored for later use. This inclusive approach to data storage drastically increases the amount of data available for accurate prediction based on ML. In addition, all data are highly reproducible because they are not acquired by humans (i.e., devoid of human errors). Therefore, it is possible to analyze in detail which process parameters are correlated with physical properties, leading to technology for predicting synthetic processes (i.e., process informatics).\n<\/p><p>The sheer increase in experimental speed directly translates to an increase in the amount of materials data. The speed of autonomous experiments is estimated to be at least one order of magnitude faster than conventional experiments; it is possible to accelerate the acquisition of measurement data by another order of magnitude using the materials checkup system (Figure 5).<sup id=\"rdp-ebb-cite_ref-:0_3-3\" class=\"reference\"><a href=\"#cite_note-:0-3\">[3]<\/a><\/sup>\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig5_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg\" class=\"image wiki-link\" data-key=\"3805f81c2ccd3afe7af75cc3b23dabdf\"><img alt=\"Fig5 Ishizuki SciTechAdvMatMeth2023 3-1.jpeg\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/a\/a3\/Fig5_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 5.<\/b> Concept of the materials checkup system for finding unexpected results. All the material-property measurements are done automatically. Original source Shimizu <i>et al.<\/i> 2020, reprinted with permission under a Creative Commons Attribution (CC BY) license.<sup id=\"rdp-ebb-cite_ref-:0_3-4\" class=\"reference\"><a href=\"#cite_note-:0-3\">[3]<\/a><\/sup><\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>In conventional research, human researchers often need to transport their samples to multiple analysis systems to evaluate the physical properties. To avoid errors associated with sample transport and shorten the time for analysis, all analytical modules should be connected to each other. Connecting multiple materials property measurement devices and automatically measuring one after another enable multifaceted materials characterization (Figure 5). With the recent development of measurement and analysis informatics<sup id=\"rdp-ebb-cite_ref-53\" class=\"reference\"><a href=\"#cite_note-53\">[53]<\/a><\/sup>, it is possible to perform measurements with even higher accuracy, precision, and sensitivity measurements in a short period of time.<sup id=\"rdp-ebb-cite_ref-54\" class=\"reference\"><a href=\"#cite_note-54\">[54]<\/a><\/sup>\n<\/p><p>The key to the multifaceted evaluation is to increase the probability of finding unexpected results. We have already experienced finding an unexpected electrode material using multifaceted evaluation while targeting the synthesis of a solid electrolyte.\n<\/p>\n<h3><span id=\"rdp-ebb-Constructing_scientific_theories_from_a_bird\u2019s-eye_view\"><\/span><span class=\"mw-headline\" id=\"Constructing_scientific_theories_from_a_bird.E2.80.99s-eye_view\">Constructing scientific theories from a bird\u2019s-eye view<\/span><\/h3>\n<p>The AES does not search aimlessly in the search space. Instead, it creates a materials map<sup id=\"rdp-ebb-cite_ref-55\" class=\"reference\"><a href=\"#cite_note-55\">[55]<\/a><\/sup> based on the large amount of data described in the previous section to give a bird\u2019s-eye view of the materials world (Figure 6).\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig6_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg\" class=\"image wiki-link\" data-key=\"a7c549587177d186452c36621c5d1d91\"><img alt=\"Fig6 Ishizuki SciTechAdvMatMeth2023 3-1.jpeg\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/9\/9b\/Fig6_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 6.<\/b> Creation of a \"materials map\" to identify unexplored materials. Autonomous experiments enable the creation of materials maps using experimental data. We put particular emphasis on unexpected experimental results and explore their surroundings.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>Specifically, materials informatics methods help organize the enormous amount of multi-dimensional data (e.g., actual experimental data and high-throughput first-principles simulations) into human-readable forms. This data is overlaid with previously reported experimental and simulated data to create a materials map.\n<\/p><p>The materials map depicts the untapped and promising areas that should be experimentally verified. This depiction stimulates the researchers\u2019 thinking in ways conventional research methods cannot. The new ideas can then be immediately tested using the AES (Figure 1).\n<\/p><p>The unveiling of the hidden correlations and relationships between materials properties and processes will promote the construction of new theories. Until now, completely unexpected and never-before-observed results are often ignored because it has been difficult to distinguish whether they are real or whether they are caused by experimental error. However, these results can be reproduced when using the AES; unexpected results (i.e., \"outliers\") will not be ignored, making it possible to create new scientific theories that explain them (Figure 7).\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig7_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg\" class=\"image wiki-link\" data-key=\"ca01d0f57cb88fb191f0e3c7b2fab3c3\"><img alt=\"Fig7 Ishizuki SciTechAdvMatMeth2023 3-1.jpeg\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/a\/ae\/Fig7_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 7.<\/b> Importance of outliers. Suppose there is a number hidden in the tiles in (a). With no data, nothing can be predicted. (b) With some amount of data, hypotheses can be proposed. Typical researchers could not predict that the number \"1\" is hidden in this figure. Only an exceptional researcher can recognize the hidden \"1.\" Materials science to date has been \"local\" (i.e., belonging to particular researchers) in this way. (c) When big data is available, researchers can easily recognize \"1,\" without relying on such exceptional researchers. Furthermore, by considering the \"outliers\" (shown in red), we can see the true situation: a \"4\" rather than a \"1.\" We can thus construct new theories (i.e., big-picture materials science).<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>Lastly, to overview a massive amount of materials data, it is necessary to have the technology capable of converting this data for humans to understand. In particular, the development of technologies for explainable ML and <a href=\"https:\/\/www.limswiki.org\/index.php\/Data_visualization\" title=\"Data visualization\" class=\"wiki-link\" data-key=\"4a3b86cba74bc7bb7471aa3fc2fcccc3\">visualization of information<\/a> in a multi-dimensional space through dimensionality reduction is mandatory.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Important_points_for_the_actual_operation_of_an_autonomous_system\">Important points for the actual operation of an autonomous system<\/span><\/h2>\n<p>This section discusses some important take-home lessons we have learned from our constructing and operating the AES.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Setting_the_right_research_topic\">Setting the right research topic<\/span><\/h3>\n<p>It is not practical to leave all experimental operations to ML and robots; instead, it is crucial to choose the right tasks. The current autonomous experimental technology works best when applied to processes with well-developed instructions. Techniques with broad and general applications that many researchers use are better suited for applying to autonomous systems. For example, we chose to automate sputtering thin-film deposition because it is widely used in the field, including the semiconductor industry.\n<\/p><p>Another point concerns the human operator\u2019s psychological aspect. In our experiments, the human operators perform two tasks: setting up the raw materials and cleaning up the pieces used in the experiments. Because the operator performs these tasks according to the robot\u2019s schedule, in some cases, the operator may feel as if being controlled by robots. It is critical that robots perform these tasks in the future.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Cost_issues\">Cost issues<\/span><\/h3>\n<p>It is difficult to introduce an autonomous system to every laboratory because of its cost. There are three points to consider. The first point is that robots are becoming less expensive, and their range of applications is expanding. The price depends on positioning accuracy and repeatability. Because robots are generally capable of working 24\u2009hours a day, 7\u2009days a week, the operational cost depends on the tasks and the labor cost. The software cost has also decreased, as robots can be run using LabVIEW or <a href=\"https:\/\/www.limswiki.org\/index.php\/Python_(programming_language)\" title=\"Python (programming language)\" class=\"wiki-link\" data-key=\"ef6905a29cbb75d3c71e6bdf6e2915dd\">Python<\/a>, two widely used programming environments. In this way, the total system cost is becoming affordable, and we are now in an era when robots can be introduced to laboratories. We note that it is important to provide open labs for researchers unaware of the possibilities of autonomous systems, to get acquainted with the actual setups and operations of such systems.\n<\/p><p>The second point is to promote <a href=\"https:\/\/www.limswiki.org\/index.php\/Data_sharing\" title=\"Data sharing\" class=\"wiki-link\" data-key=\"a99d5fda27f755c693c65864d9286130\">sharing<\/a> the autonomous system via the internet, providing access to the system from anywhere in the world (via <a href=\"https:\/\/www.limswiki.org\/index.php\/Cloud_computing\" title=\"Cloud computing\" class=\"wiki-link\" data-key=\"fcfe5882eaa018d920cedb88398b604f\">cloud computing<\/a> for the laboratory, sharing experiments over the cloud; Figure 8). Currently, the equipment in the laboratory is only used during the daytime on weekdays and therefore is not fully utilized. Full utilization around the clock through sharing will enhance cost-effectiveness; it will also enable materials fabrication while researchers are at home, making a remote style of laboratory work possible.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig8_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg\" class=\"image wiki-link\" data-key=\"f7f82dc948b6f72ccc45b40f1b03a12a\"><img alt=\"Fig8 Ishizuki SciTechAdvMatMeth2023 3-1.jpeg\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/9\/9b\/Fig8_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 8.<\/b> Experiments in the cloud space. Sharing the autonomous system via the internet has become possible.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>The last point is the perspective of multifaceted characterization. As shown in Figure 5, multiple analytic instruments automatically characterize a range of properties. The time previously spent by researchers on measurements can now be allocated to higher value-added tasks, further enhancing cost-effectiveness.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Change_in_the_experimental_sequences\">Change in the experimental sequences<\/span><\/h3>\n<p>Conventionally, the materials to be synthesized are decided before the synthesis. After the synthesis, compositions and structures are analyzed to confirm that the aimed materials have been synthesized. After identifying the compounds, the properties are evaluated.\n<\/p><p>Autonomous experiments reverse this sequence. Because the goal of the research is to determine the material with superior physical properties, there is no need to decide what materials to synthesize in advance. The compositional and structural analyses to identify the materials are only performed once the robot has synthesized materials with the optimum properties.\n<\/p><p>Autonomous experiments applied to device fabrications show a similar sequence reversal. The system automatically fabricates a device, evaluates its performance, and tunes the next materials based on the results. Again, there is no need to know the details of the materials prior to device fabrications.\n<\/p><p>From a broader perspective, autonomous systems change our attitude toward research. New ideas\u2014which previously required careful experimental planning to be tested with a small number of trials\u2014can now be tested immediately using an autonomous system. With automation, we can comfortably conduct experiments even with a naive idea because the autonomous system performs experiments, and the number of experiments can be increased at will.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Steps_for_implementation_and_the_rise_of_the_lab-system_integrator\">Steps for implementation and the rise of the lab-system integrator<\/span><\/h3>\n<p>How should a materials research lab start out with automation and autonomous experiments? The following points should be considered when integrating materials synthesis and evaluation, robotics, ML, and overall system control.\n<\/p>\n<ol><li>Clarify the workflow and choose the appropriate tasks for automation (see the prior discussion about setting the right research topic).<\/li>\n<li>Set up a team, including an engineer who understands <a href=\"https:\/\/www.limswiki.org\/index.php\/Informatics_(academic_field)\" title=\"Informatics (academic field)\" class=\"wiki-link\" data-key=\"0391318826a5d9f9a1a1bcc88394739f\">informatics<\/a>. In addition, include an engineer familiar with equipment control and systemization.<\/li>\n<li>The introduction of robots can be expensive. It may be difficult to introduce the whole system at once. Therefore, it is necessary to create a chain of successes to maintain continuous funding. The critical point is to draw an overall plan and accumulate small successes.<\/li><\/ol>\n<p>Standardizing the lab equipment is essential to reduce the time to set up and the cost. Shortly, lab-system integrators will emerge. There is a movement in the world towards aligning the format of all mechanical interfaces and communication protocol; each piece of equipment will be integrated to form an automation system.<sup id=\"rdp-ebb-cite_ref-:11_15-2\" class=\"reference\"><a href=\"#cite_note-:11-15\">[15]<\/a><\/sup> Standardization and <i>de facto<\/i> standards are key for lab-system integrators, and the technology must be launched quickly.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"The_autonomous_system_and_intellectual_property\">The autonomous system and intellectual property<\/span><\/h2>\n<p>Currently, the intellectual property of newly discovered or invented materials is effectuated through scientific publication or patent application. However, when the discovery\/invention is accomplished without human intervention, can we say that a human discovered or invented the materials? Can the AES be an inventor under patent law?<sup id=\"rdp-ebb-cite_ref-:21_25-2\" class=\"reference\"><a href=\"#cite_note-:21-25\">[25]<\/a><\/sup> Can the system be an author of a paper? Furthermore, patents are designed to protect inventions that are not only new, but also involve an inventive step; will the autonomous system change how inventive steps are determined? In other words, will the autonomous system change the standard of what constitutes an easy invention? In this section, we discuss inventorship, inventive step, and authorship.\n<\/p>\n<h3><span id=\"rdp-ebb-Inventorship_\u2013_Who_is_an_inventor?\"><\/span><span class=\"mw-headline\" id=\"Inventorship_.E2.80.93_Who_is_an_inventor.3F\">Inventorship \u2013 Who is an inventor?<\/span><\/h3>\n<p>Whether AI systems, including autonomous systems, should be treated as an inventor under patent law is a hot topic.<sup id=\"rdp-ebb-cite_ref-56\" class=\"reference\"><a href=\"#cite_note-56\">[56]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-57\" class=\"reference\"><a href=\"#cite_note-57\">[57]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:28_58-0\" class=\"reference\"><a href=\"#cite_note-:28-58\">[58]<\/a><\/sup> A person who makes an invention is called the inventor (we note that the patent owner is often not the inventor but the company that employs the inventor). However, if AI makes the invention, can it be considered the inventor under patent law?\n<\/p><p>Abbott <i>et al.<\/i> applied for patents naming the AI \"DABUS\" as the inventor in several countries and regions to raise this issue.<sup id=\"rdp-ebb-cite_ref-:29_59-0\" class=\"reference\"><a href=\"#cite_note-:29-59\">[59]<\/a><\/sup> They proposed that the inventor is DABUS and the patent owner would be the person who owns DABUS. For this case, the South African patent authorities have accepted the patent applications with DABUS as the sole inventor<sup id=\"rdp-ebb-cite_ref-60\" class=\"reference\"><a href=\"#cite_note-60\">[60]<\/a><\/sup>; however, we note that the decision can still be overturned by the respective superior courts.\n<\/p><p>Since the wording of patent acts and their interpretations differ from one jurisdiction to another, whether AI can be treated by patent acts as an inventor also differs. In contrast to the above examples, the patent offices and courts in the U.S., U.K., Australia, the European Patent Office, and the Japan Patent Office have all ruled that only human beings can be the inventor under the patent acts, rejecting the notion of DABUS being treated as an inventor.<sup id=\"rdp-ebb-cite_ref-:28_58-1\" class=\"reference\"><a href=\"#cite_note-:28-58\">[58]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:29_59-1\" class=\"reference\"><a href=\"#cite_note-:29-59\">[59]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-61\" class=\"reference\"><a href=\"#cite_note-61\">[61]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-62\" class=\"reference\"><a href=\"#cite_note-62\">[62]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-63\" class=\"reference\"><a href=\"#cite_note-63\">[63]<\/a><\/sup> In such countries\/regions, even when a new material is discovered by an AES, only human researchers who have operated the system and contributed creatively may be treated as inventors. As mentioned prior, the current level of technology still needs significant human contribution to conduct the research.\n<\/p><p>However, there is an argument that if the AES becomes more autonomous and the level of human contribution continues to decrease, the human researcher may no longer be treated as the inventor. For example, in Japan, the inventor is recognized as a person who conceives an invention or creates the specific materials based on the conceived idea.<sup id=\"rdp-ebb-cite_ref-64\" class=\"reference\"><a href=\"#cite_note-64\">[64]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-65\" class=\"reference\"><a href=\"#cite_note-65\">[65]<\/a><\/sup> From this perspective, if the AES both conceives and creates a specific material, the human would no longer be considered the inventor. As the autonomous system evolves, in order to be recognized as the inventor, future human researchers will have to either exercise creativity independently from the AES or demonstrate that the AES is used as a tool.\n<\/p>\n<h3><span id=\"rdp-ebb-Inventive_step_\u2013_Is_an_invention_easily_invented?\"><\/span><span class=\"mw-headline\" id=\"Inventive_step_.E2.80.93_Is_an_invention_easily_invented.3F\">Inventive step \u2013 Is an invention easily invented?<\/span><\/h3>\n<p>To be protected by a patent, an invention must satisfy the definition of inventive step (non-obviousness in the US or Canada). The question here is in what way the AES will change how the inventive step is determined. In the following, we discuss the impact of the system on the inventive step in the future, according to the Japanese patent system.\n<\/p><p>By definition, an inventive step requires that an ordinary skilled person in the field not be able to make the invention easily at the time of filing the patent application. In Japan<sup id=\"rdp-ebb-cite_ref-66\" class=\"reference\"><a href=\"#cite_note-66\">[66]<\/a><\/sup>, patent examiners determine the inventive step by assessing (1) factors indicating the absence of inventive step, and (2) factors supporting the existence of inventive step. Specifically, factor (1) questions whether a skilled person in the field could have easily made the invention, and factor (2) questions, for example, the existence of advantageous effects exceeding predictability based on the state of the art.\n<\/p><p>At present, because AI in materials exploration is still in its infancy and its accuracy remains low, the current patent examination practices determining inventive step have not been changed. However, many researchers in intellectual properties and business associations point out that, as AI\u2019s accuracy continues to rise and becomes widely used, the difficulty of achieving new invention will be lowered for the skilled person in the field, raising the standard for what satisfies the definition of inventive step.<sup id=\"rdp-ebb-cite_ref-67\" class=\"reference\"><a href=\"#cite_note-67\">[67]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-68\" class=\"reference\"><a href=\"#cite_note-68\">[68]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-69\" class=\"reference\"><a href=\"#cite_note-69\">[69]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-70\" class=\"reference\"><a href=\"#cite_note-70\">[70]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-71\" class=\"reference\"><a href=\"#cite_note-71\">[71]<\/a><\/sup> In this case, how the inventive step is legally determined may be changed in the future.\n<\/p><p>Here we apply this argument to the finding of novel materials by the AES. At this time, the system is not yet widely used, but, in the future, the situation may change as this system becomes well implemented or shared in laboratories. Then, we believe the determination of the inventive step may be changed by a shift in the balance between factors (1) and (2). For example, because the AES excels at optimizing numerical conditions, it is conceivable that many invention cases involving such optimizations would be accessible to an ordinary skilled person in the field with access to the AES. Such cases would make factor (1) stronger because it is sufficiently reasoned for the ordinary skilled person to arrive at the invention, causing the standard for satisfying the inventive step to be raised. Although the possibility to surpass the raised standard by considering a combination of factor (2), such as advantageous effects, the chance of satisfying the inventive step decreases if the current research and development approach remains unchanged.\n<\/p><p>To obtain patent protection in a future where the AES is widely used, human researchers will have to achieve the inventive step by redirecting their creativity away from the strengths of the AES, in addition to the inventorship mentioned above. For example, the AES will never be able to conceive highly innovative synthesis techniques, or develop new characterization techniques. Furthermore, even for inventions involving optimization of numerical conditions, human creativity will contribute to the inventive step when it is difficult even for the AES to come up with a specific parameter. In addition, new materials discovered by an AES will still need to be put to use, requiring human input to create new devices by combining many materials, utilizing hierarchical structures. Lastly, it is human beings who will think about what issues facing society and how to solve them; thus, the role of the human researchers is to judge the value and to decide what to do next. This big-picture perspective will lead to new inventions that only humans can think of.\n<\/p>\n<h3><span id=\"rdp-ebb-Authorship_\u2013_Who_is_an_author_of_academic_papers?\"><\/span><span class=\"mw-headline\" id=\"Authorship_.E2.80.93_Who_is_an_author_of_academic_papers.3F\">Authorship \u2013 Who is an author of academic papers?<\/span><\/h3>\n<p>Each journal and publisher has its authorship guidelines. One well-known example is the authorship guidelines published by the Committee on Publication Ethics (COPE)<sup id=\"rdp-ebb-cite_ref-72\" class=\"reference\"><a href=\"#cite_note-72\">[72]<\/a><\/sup>, which states that \"the minimum requirements for authorship, common to all definitions, are (A) substantial contribution to the work and (B) accountability for the work that was done and its presentation in a publication.\" Another well-known example from the International Committee of Medical Journal Editors (ICMJE)<sup id=\"rdp-ebb-cite_ref-73\" class=\"reference\"><a href=\"#cite_note-73\">[73]<\/a><\/sup> also includes the same concept as requirements (A) and (B).\n<\/p><p>Although most scientists disapprove of articles crediting the AI tool as an author, an AI tool such as ChatGPT has listed its name as an author.<sup id=\"rdp-ebb-cite_ref-74\" class=\"reference\"><a href=\"#cite_note-74\">[74]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-75\" class=\"reference\"><a href=\"#cite_note-75\">[75]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-76\" class=\"reference\"><a href=\"#cite_note-76\">[76]<\/a><\/sup> However, these cases do not focus on the AES but mainly show that AI can generate sentences. Whether or not the AES can be treated as an author of an academic discovery is a quite important issue and should be discussed.\n<\/p><p>So, let\u2019s consider the case where human researchers write a manuscript on a new material discovered by the AES. At the current technological level, while the AES can satisfy substantial contribution (requirement A) because the system has performed the laboratory procedures leading to the new-material discovery, it cannot provide accountability (requirement B). There are many theories on whether AI can have accountability<sup id=\"rdp-ebb-cite_ref-77\" class=\"reference\"><a href=\"#cite_note-77\">[77]<\/a><\/sup>; however, at present, the general view is that only human researchers can be held accountable.<sup id=\"rdp-ebb-cite_ref-78\" class=\"reference\"><a href=\"#cite_note-78\">[78]<\/a><\/sup> Until AI can provide accountability, AI is not eligible for academic authorship.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Summary\">Summary<\/span><\/h2>\n<p>As the involvement of ML and robotics continues to expand in scientific research, the roles of human researchers have become a central question. ML and robots will not replace researchers; from the onset, AESs have aimed at providing researchers with the chance to think and to be more creative. The important point is that scientists understand both the limitations and the possibilities of ML and robotics, and apply these techniques to the right problem. Even in an age where ML and robots perform most of the laboratory procedures, human researchers will always be the main players. This approach to automation and autonomy will exponentially accelerate creative research and development, contributing to the future development of science, technology, and industry.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Acknowledgements\">Acknowledgements<\/span><\/h2>\n<p>We would like to thank R. Nakayama, S. Kobayashi, and T. Kimura at Tokyo Institute of Technology and D. Packwood at Kyoto University for their continuous discussions. We thank P. Han for editing the manuscript.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Funding\">Funding<\/span><\/h3>\n<p>This research was supported by JST-CREST [Grant No. JPMJCR1523], JST-MIRAI [Grant No. JPMJMI21G2], and MEXT Program: Data Creation and Utilization-Type Material Research and Development Project [Grant No. JPMXP1122712807].\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Conflict_of_interest\">Conflict of interest<\/span><\/h3>\n<p>No potential conflict of interest was reported by the author(s).\n<\/p>\n<h2><span class=\"mw-headline\" id=\"References\">References<\/span><\/h2>\n<div class=\"reflist references-column-width\" style=\"-moz-column-width: 30em; -webkit-column-width: 30em; column-width: 30em; list-style-type: decimal;\">\n<div class=\"mw-references-wrap mw-references-columns\"><ol class=\"references\">\n<li id=\"cite_note-1\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-1\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Bohacek, R. S.; McMartin, C.; Guida, W. C. (1 January 1996). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/8788213\" target=\"_blank\">\"The art and practice of structure-based drug design: a molecular modeling perspective\"<\/a>. <i>Medicinal Research Reviews<\/i> <b>16<\/b> (1): 3\u201350. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1002%2F%28SICI%291098-1128%28199601%2916%3A1%3C3%3A%3AAID-MED1%3E3.0.CO%3B2-6\" target=\"_blank\">10.1002\/(SICI)1098-1128(199601)16:1<3::AID-MED1>3.0.CO;2-6<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0198-6325\" target=\"_blank\">0198-6325<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/8788213\" target=\"_blank\">8788213<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/8788213\" target=\"_blank\">https:\/\/pubmed.ncbi.nlm.nih.gov\/8788213<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+art+and+practice+of+structure-based+drug+design%3A+a+molecular+modeling+perspective&rft.jtitle=Medicinal+Research+Reviews&rft.aulast=Bohacek&rft.aufirst=R.+S.&rft.au=Bohacek%2C%26%2332%3BR.+S.&rft.au=McMartin%2C%26%2332%3BC.&rft.au=Guida%2C%26%2332%3BW.+C.&rft.date=1+January+1996&rft.volume=16&rft.issue=1&rft.pages=3%E2%80%9350&rft_id=info:doi\/10.1002%2F%28SICI%291098-1128%28199601%2916%3A1%3C3%3A%3AAID-MED1%3E3.0.CO%3B2-6&rft.issn=0198-6325&rft_id=info:pmid\/8788213&rft_id=https%3A%2F%2Fpubmed.ncbi.nlm.nih.gov%2F8788213&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-2\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-2\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Kirkpatrick, Peter; Ellis, Clare (1 December 2004). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/432823a\" target=\"_blank\">\"Chemical space\"<\/a> (in en). <i>Nature<\/i> <b>432<\/b> (7019): 823\u2013823. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2F432823a\" target=\"_blank\">10.1038\/432823a<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0028-0836\" target=\"_blank\">0028-0836<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/432823a\" target=\"_blank\">https:\/\/www.nature.com\/articles\/432823a<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Chemical+space&rft.jtitle=Nature&rft.aulast=Kirkpatrick&rft.aufirst=Peter&rft.au=Kirkpatrick%2C%26%2332%3BPeter&rft.au=Ellis%2C%26%2332%3BClare&rft.date=1+December+2004&rft.volume=432&rft.issue=7019&rft.pages=823%E2%80%93823&rft_id=info:doi\/10.1038%2F432823a&rft.issn=0028-0836&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2F432823a&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:0-3\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:0_3-0\">3.0<\/a><\/sup> <sup><a href=\"#cite_ref-:0_3-1\">3.1<\/a><\/sup> <sup><a href=\"#cite_ref-:0_3-2\">3.2<\/a><\/sup> <sup><a href=\"#cite_ref-:0_3-3\">3.3<\/a><\/sup> <sup><a href=\"#cite_ref-:0_3-4\">3.4<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Shimizu, Ryota; Kobayashi, Shigeru; Watanabe, Yuki; Ando, Yasunobu; Hitosugi, Taro (1 November 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubs.aip.org\/apm\/article\/8\/11\/111110\/569817\/Autonomous-materials-synthesis-by-machine-learning\" target=\"_blank\">\"Autonomous materials synthesis by machine learning and robotics\"<\/a> (in en). <i>APL Materials<\/i> <b>8<\/b> (11): 111110. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1063%2F5.0020370\" target=\"_blank\">10.1063\/5.0020370<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2166-532X\" target=\"_blank\">2166-532X<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubs.aip.org\/apm\/article\/8\/11\/111110\/569817\/Autonomous-materials-synthesis-by-machine-learning\" target=\"_blank\">https:\/\/pubs.aip.org\/apm\/article\/8\/11\/111110\/569817\/Autonomous-materials-synthesis-by-machine-learning<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Autonomous+materials+synthesis+by+machine+learning+and+robotics&rft.jtitle=APL+Materials&rft.aulast=Shimizu&rft.aufirst=Ryota&rft.au=Shimizu%2C%26%2332%3BRyota&rft.au=Kobayashi%2C%26%2332%3BShigeru&rft.au=Watanabe%2C%26%2332%3BYuki&rft.au=Ando%2C%26%2332%3BYasunobu&rft.au=Hitosugi%2C%26%2332%3BTaro&rft.date=1+November+2020&rft.volume=8&rft.issue=11&rft.pages=111110&rft_id=info:doi\/10.1063%2F5.0020370&rft.issn=2166-532X&rft_id=https%3A%2F%2Fpubs.aip.org%2Fapm%2Farticle%2F8%2F11%2F111110%2F569817%2FAutonomous-materials-synthesis-by-machine-learning&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-4\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-4\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Kitano, Hiroaki (18 June 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41540-021-00189-3\" target=\"_blank\">\"Nobel Turing Challenge: creating the engine for scientific discovery\"<\/a> (in en). <i>npj Systems Biology and Applications<\/i> <b>7<\/b> (1): 29. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41540-021-00189-3\" target=\"_blank\">10.1038\/s41540-021-00189-3<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2056-7189\" target=\"_blank\">2056-7189<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC8213706\/\" target=\"_blank\">PMC8213706<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/34145287\" target=\"_blank\">34145287<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41540-021-00189-3\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41540-021-00189-3<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Nobel+Turing+Challenge%3A+creating+the+engine+for+scientific+discovery&rft.jtitle=npj+Systems+Biology+and+Applications&rft.aulast=Kitano&rft.aufirst=Hiroaki&rft.au=Kitano%2C%26%2332%3BHiroaki&rft.date=18+June+2021&rft.volume=7&rft.issue=1&rft.pages=29&rft_id=info:doi\/10.1038%2Fs41540-021-00189-3&rft.issn=2056-7189&rft_id=info:pmc\/PMC8213706&rft_id=info:pmid\/34145287&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41540-021-00189-3&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:1-5\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:1_5-0\">5.0<\/a><\/sup> <sup><a href=\"#cite_ref-:1_5-1\">5.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Tabor, Daniel P.; Roch, Lo\u00efc M.; Saikin, Semion K.; Kreisbeck, Christoph; Sheberla, Dennis; Montoya, Joseph H.; Dwaraknath, Shyam; Aykol, Muratahan <i>et al.<\/i> (26 April 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41578-018-0005-z\" target=\"_blank\">\"Accelerating the discovery of materials for clean energy in the era of smart automation\"<\/a> (in en). <i>Nature Reviews Materials<\/i> <b>3<\/b> (5): 5\u201320. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41578-018-0005-z\" target=\"_blank\">10.1038\/s41578-018-0005-z<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2058-8437\" target=\"_blank\">2058-8437<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41578-018-0005-z\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41578-018-0005-z<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Accelerating+the+discovery+of+materials+for+clean+energy+in+the+era+of+smart+automation&rft.jtitle=Nature+Reviews+Materials&rft.aulast=Tabor&rft.aufirst=Daniel+P.&rft.au=Tabor%2C%26%2332%3BDaniel+P.&rft.au=Roch%2C%26%2332%3BLo%C3%AFc+M.&rft.au=Saikin%2C%26%2332%3BSemion+K.&rft.au=Kreisbeck%2C%26%2332%3BChristoph&rft.au=Sheberla%2C%26%2332%3BDennis&rft.au=Montoya%2C%26%2332%3BJoseph+H.&rft.au=Dwaraknath%2C%26%2332%3BShyam&rft.au=Aykol%2C%26%2332%3BMuratahan&rft.au=Ortiz%2C%26%2332%3BCarlos&rft.date=26+April+2018&rft.volume=3&rft.issue=5&rft.pages=5%E2%80%9320&rft_id=info:doi\/10.1038%2Fs41578-018-0005-z&rft.issn=2058-8437&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41578-018-0005-z&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:2-6\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:2_6-0\">6.0<\/a><\/sup> <sup><a href=\"#cite_ref-:2_6-1\">6.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Dimitrov, Tanja; Kreisbeck, Christoph; Becker, Jill S.; Aspuru-Guzik, Al\u00e1n; Saikin, Semion K. (17 July 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acsami.9b01226\" target=\"_blank\">\"Autonomous Molecular Design: Then and Now\"<\/a> (in en). <i>ACS Applied Materials & Interfaces<\/i> <b>11<\/b> (28): 24825\u201324836. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1021%2Facsami.9b01226\" target=\"_blank\">10.1021\/acsami.9b01226<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1944-8244\" target=\"_blank\">1944-8244<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acsami.9b01226\" target=\"_blank\">https:\/\/pubs.acs.org\/doi\/10.1021\/acsami.9b01226<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Autonomous+Molecular+Design%3A+Then+and+Now&rft.jtitle=ACS+Applied+Materials+%26+Interfaces&rft.aulast=Dimitrov&rft.aufirst=Tanja&rft.au=Dimitrov%2C%26%2332%3BTanja&rft.au=Kreisbeck%2C%26%2332%3BChristoph&rft.au=Becker%2C%26%2332%3BJill+S.&rft.au=Aspuru-Guzik%2C%26%2332%3BAl%C3%A1n&rft.au=Saikin%2C%26%2332%3BSemion+K.&rft.date=17+July+2019&rft.volume=11&rft.issue=28&rft.pages=24825%E2%80%9324836&rft_id=info:doi\/10.1021%2Facsami.9b01226&rft.issn=1944-8244&rft_id=https%3A%2F%2Fpubs.acs.org%2Fdoi%2F10.1021%2Facsami.9b01226&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:3-7\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:3_7-0\">7.0<\/a><\/sup> <sup><a href=\"#cite_ref-:3_7-1\">7.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Flores-Leonar, Martha M.; Mej\u00eda-Mendoza, Luis M.; Aguilar-Granda, Andr\u00e9s; Sanchez-Lengeling, Benjamin; Tribukait, Hermann; Amador-Bedolla, Carlos; Aspuru-Guzik, Al\u00e1n (1 October 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2452223620300596\" target=\"_blank\">\"Materials Acceleration Platforms: On the way to autonomous experimentation\"<\/a> (in en). <i>Current Opinion in Green and Sustainable Chemistry<\/i> <b>25<\/b>: 100370. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.cogsc.2020.100370\" target=\"_blank\">10.1016\/j.cogsc.2020.100370<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2452223620300596\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2452223620300596<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Materials+Acceleration+Platforms%3A+On+the+way+to+autonomous+experimentation&rft.jtitle=Current+Opinion+in+Green+and+Sustainable+Chemistry&rft.aulast=Flores-Leonar&rft.aufirst=Martha+M.&rft.au=Flores-Leonar%2C%26%2332%3BMartha+M.&rft.au=Mej%C3%ADa-Mendoza%2C%26%2332%3BLuis+M.&rft.au=Aguilar-Granda%2C%26%2332%3BAndr%C3%A9s&rft.au=Sanchez-Lengeling%2C%26%2332%3BBenjamin&rft.au=Tribukait%2C%26%2332%3BHermann&rft.au=Amador-Bedolla%2C%26%2332%3BCarlos&rft.au=Aspuru-Guzik%2C%26%2332%3BAl%C3%A1n&rft.date=1+October+2020&rft.volume=25&rft.pages=100370&rft_id=info:doi\/10.1016%2Fj.cogsc.2020.100370&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2452223620300596&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:4-8\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:4_8-0\">8.0<\/a><\/sup> <sup><a href=\"#cite_ref-:4_8-1\">8.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Coley, Connor W.; Eyke, Natalie S.; Jensen, Klavs F. (14 December 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/anie.201909987\" target=\"_blank\">\"Autonomous Discovery in the Chemical Sciences Part I: Progress\"<\/a> (in en). <i>Angewandte Chemie International Edition<\/i> <b>59<\/b> (51): 22858\u201322893. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1002%2Fanie.201909987\" target=\"_blank\">10.1002\/anie.201909987<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1433-7851\" target=\"_blank\">1433-7851<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/anie.201909987\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/anie.201909987<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Autonomous+Discovery+in+the+Chemical+Sciences+Part+I%3A+Progress&rft.jtitle=Angewandte+Chemie+International+Edition&rft.aulast=Coley&rft.aufirst=Connor+W.&rft.au=Coley%2C%26%2332%3BConnor+W.&rft.au=Eyke%2C%26%2332%3BNatalie+S.&rft.au=Jensen%2C%26%2332%3BKlavs+F.&rft.date=14+December+2020&rft.volume=59&rft.issue=51&rft.pages=22858%E2%80%9322893&rft_id=info:doi\/10.1002%2Fanie.201909987&rft.issn=1433-7851&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1002%2Fanie.201909987&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:5-9\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:5_9-0\">9.0<\/a><\/sup> <sup><a href=\"#cite_ref-:5_9-1\">9.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Coley, Connor W.; Eyke, Natalie S.; Jensen, Klavs F. (21 December 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/anie.201909989\" target=\"_blank\">\"Autonomous Discovery in the Chemical Sciences Part II: Outlook\"<\/a> (in en). <i>Angewandte Chemie International Edition<\/i> <b>59<\/b> (52): 23414\u201323436. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1002%2Fanie.201909989\" target=\"_blank\">10.1002\/anie.201909989<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1433-7851\" target=\"_blank\">1433-7851<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/anie.201909989\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/anie.201909989<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Autonomous+Discovery+in+the+Chemical+Sciences+Part+II%3A+Outlook&rft.jtitle=Angewandte+Chemie+International+Edition&rft.aulast=Coley&rft.aufirst=Connor+W.&rft.au=Coley%2C%26%2332%3BConnor+W.&rft.au=Eyke%2C%26%2332%3BNatalie+S.&rft.au=Jensen%2C%26%2332%3BKlavs+F.&rft.date=21+December+2020&rft.volume=59&rft.issue=52&rft.pages=23414%E2%80%9323436&rft_id=info:doi\/10.1002%2Fanie.201909989&rft.issn=1433-7851&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1002%2Fanie.201909989&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:6-10\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:6_10-0\">10.0<\/a><\/sup> <sup><a href=\"#cite_ref-:6_10-1\">10.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Gromski, Piotr S.; Granda, Jaros\u0142aw M.; Cronin, Leroy (1 January 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2589597419301868\" target=\"_blank\">\"Universal Chemical Synthesis and Discovery with \u2018The Chemputer\u2019\"<\/a> (in en). <i>Trends in Chemistry<\/i> <b>2<\/b> (1): 4\u201312. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.trechm.2019.07.004\" target=\"_blank\">10.1016\/j.trechm.2019.07.004<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2589597419301868\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2589597419301868<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Universal+Chemical+Synthesis+and+Discovery+with+%E2%80%98The+Chemputer%E2%80%99&rft.jtitle=Trends+in+Chemistry&rft.aulast=Gromski&rft.aufirst=Piotr+S.&rft.au=Gromski%2C%26%2332%3BPiotr+S.&rft.au=Granda%2C%26%2332%3BJaros%C5%82aw+M.&rft.au=Cronin%2C%26%2332%3BLeroy&rft.date=1+January+2020&rft.volume=2&rft.issue=1&rft.pages=4%E2%80%9312&rft_id=info:doi\/10.1016%2Fj.trechm.2019.07.004&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2589597419301868&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:7-11\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:7_11-0\">11.0<\/a><\/sup> <sup><a href=\"#cite_ref-:7_11-1\">11.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Wilbraham, Liam; Mehr, S. Hessam M.; Cronin, Leroy (19 January 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acs.accounts.0c00674\" target=\"_blank\">\"Digitizing Chemistry Using the Chemical Processing Unit: From Synthesis to Discovery\"<\/a> (in en). <i>Accounts of Chemical Research<\/i> <b>54<\/b> (2): 253\u2013262. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1021%2Facs.accounts.0c00674\" target=\"_blank\">10.1021\/acs.accounts.0c00674<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0001-4842\" target=\"_blank\">0001-4842<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acs.accounts.0c00674\" target=\"_blank\">https:\/\/pubs.acs.org\/doi\/10.1021\/acs.accounts.0c00674<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Digitizing+Chemistry+Using+the+Chemical+Processing+Unit%3A+From+Synthesis+to+Discovery&rft.jtitle=Accounts+of+Chemical+Research&rft.aulast=Wilbraham&rft.aufirst=Liam&rft.au=Wilbraham%2C%26%2332%3BLiam&rft.au=Mehr%2C%26%2332%3BS.+Hessam+M.&rft.au=Cronin%2C%26%2332%3BLeroy&rft.date=19+January+2021&rft.volume=54&rft.issue=2&rft.pages=253%E2%80%93262&rft_id=info:doi\/10.1021%2Facs.accounts.0c00674&rft.issn=0001-4842&rft_id=https%3A%2F%2Fpubs.acs.org%2Fdoi%2F10.1021%2Facs.accounts.0c00674&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:8-12\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:8_12-0\">12.0<\/a><\/sup> <sup><a href=\"#cite_ref-:8_12-1\">12.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Szymanski, Nathan J.; Zeng, Yan; Huo, Haoyan; Bartel, Christopher J.; Kim, Haegyeom; Ceder, Gerbrand (2021). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/xlink.rsc.org\/?DOI=D1MH00495F\" target=\"_blank\">\"Toward autonomous design and synthesis of novel inorganic materials\"<\/a> (in en). <i>Materials Horizons<\/i> <b>8<\/b> (8): 2169\u20132198. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1039%2FD1MH00495F\" target=\"_blank\">10.1039\/D1MH00495F<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2051-6347\" target=\"_blank\">2051-6347<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/xlink.rsc.org\/?DOI=D1MH00495F\" target=\"_blank\">http:\/\/xlink.rsc.org\/?DOI=D1MH00495F<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Toward+autonomous+design+and+synthesis+of+novel+inorganic+materials&rft.jtitle=Materials+Horizons&rft.aulast=Szymanski&rft.aufirst=Nathan+J.&rft.au=Szymanski%2C%26%2332%3BNathan+J.&rft.au=Zeng%2C%26%2332%3BYan&rft.au=Huo%2C%26%2332%3BHaoyan&rft.au=Bartel%2C%26%2332%3BChristopher+J.&rft.au=Kim%2C%26%2332%3BHaegyeom&rft.au=Ceder%2C%26%2332%3BGerbrand&rft.date=2021&rft.volume=8&rft.issue=8&rft.pages=2169%E2%80%932198&rft_id=info:doi\/10.1039%2FD1MH00495F&rft.issn=2051-6347&rft_id=http%3A%2F%2Fxlink.rsc.org%2F%3FDOI%3DD1MH00495F&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:9-13\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:9_13-0\">13.0<\/a><\/sup> <sup><a href=\"#cite_ref-:9_13-1\">13.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Stach, Eric; DeCost, Brian; Kusne, A. Gilad; Hattrick-Simpers, Jason; Brown, Keith A.; Reyes, Kristofer G.; Schrier, Joshua; Billinge, Simon <i>et al.<\/i> (1 September 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2590238521003064\" target=\"_blank\">\"Autonomous experimentation systems for materials development: A community perspective\"<\/a> (in en). <i>Matter<\/i> <b>4<\/b> (9): 2702\u20132726. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.matt.2021.06.036\" target=\"_blank\">10.1016\/j.matt.2021.06.036<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2590238521003064\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2590238521003064<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Autonomous+experimentation+systems+for+materials+development%3A+A+community+perspective&rft.jtitle=Matter&rft.aulast=Stach&rft.aufirst=Eric&rft.au=Stach%2C%26%2332%3BEric&rft.au=DeCost%2C%26%2332%3BBrian&rft.au=Kusne%2C%26%2332%3BA.+Gilad&rft.au=Hattrick-Simpers%2C%26%2332%3BJason&rft.au=Brown%2C%26%2332%3BKeith+A.&rft.au=Reyes%2C%26%2332%3BKristofer+G.&rft.au=Schrier%2C%26%2332%3BJoshua&rft.au=Billinge%2C%26%2332%3BSimon&rft.au=Buonassisi%2C%26%2332%3BTonio&rft.date=1+September+2021&rft.volume=4&rft.issue=9&rft.pages=2702%E2%80%932726&rft_id=info:doi\/10.1016%2Fj.matt.2021.06.036&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2590238521003064&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:10-14\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:10_14-0\">14.0<\/a><\/sup> <sup><a href=\"#cite_ref-:10_14-1\">14.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Fabry, David C.; Sugiono, Erli; Rueping, Magnus (1 April 2014). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/ijch.201300080\" target=\"_blank\">\"Self-Optimizing Reactor Systems: Algorithms, On-line Analytics, Setups, and Strategies for Accelerating Continuous Flow Process Optimization\"<\/a> (in en). <i>Israel Journal of Chemistry<\/i> <b>54<\/b> (4): 341\u2013350. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1002%2Fijch.201300080\" target=\"_blank\">10.1002\/ijch.201300080<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/ijch.201300080\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/ijch.201300080<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Self-Optimizing+Reactor+Systems%3A+Algorithms%2C+On-line+Analytics%2C+Setups%2C+and+Strategies+for+Accelerating+Continuous+Flow+Process+Optimization&rft.jtitle=Israel+Journal+of+Chemistry&rft.aulast=Fabry&rft.aufirst=David+C.&rft.au=Fabry%2C%26%2332%3BDavid+C.&rft.au=Sugiono%2C%26%2332%3BErli&rft.au=Rueping%2C%26%2332%3BMagnus&rft.date=1+April+2014&rft.volume=54&rft.issue=4&rft.pages=341%E2%80%93350&rft_id=info:doi\/10.1002%2Fijch.201300080&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1002%2Fijch.201300080&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:11-15\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:11_15-0\">15.0<\/a><\/sup> <sup><a href=\"#cite_ref-:11_15-1\">15.1<\/a><\/sup> <sup><a href=\"#cite_ref-:11_15-2\">15.2<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Wagner, Jerrit; Berger, Christian G.; Du, Xiaoyan; Stubhan, Tobias; Hauch, Jens A.; Brabec, Christoph J. (1 October 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/link.springer.com\/10.1007\/s10853-021-06281-7\" target=\"_blank\">\"The evolution of Materials Acceleration Platforms: toward the laboratory of the future with AMANDA\"<\/a> (in en). <i>Journal of Materials Science<\/i> <b>56<\/b> (29): 16422\u201316446. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2Fs10853-021-06281-7\" target=\"_blank\">10.1007\/s10853-021-06281-7<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0022-2461\" target=\"_blank\">0022-2461<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/link.springer.com\/10.1007\/s10853-021-06281-7\" target=\"_blank\">https:\/\/link.springer.com\/10.1007\/s10853-021-06281-7<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+evolution+of+Materials+Acceleration+Platforms%3A+toward+the+laboratory+of+the+future+with+AMANDA&rft.jtitle=Journal+of+Materials+Science&rft.aulast=Wagner&rft.aufirst=Jerrit&rft.au=Wagner%2C%26%2332%3BJerrit&rft.au=Berger%2C%26%2332%3BChristian+G.&rft.au=Du%2C%26%2332%3BXiaoyan&rft.au=Stubhan%2C%26%2332%3BTobias&rft.au=Hauch%2C%26%2332%3BJens+A.&rft.au=Brabec%2C%26%2332%3BChristoph+J.&rft.date=1+October+2021&rft.volume=56&rft.issue=29&rft.pages=16422%E2%80%9316446&rft_id=info:doi\/10.1007%2Fs10853-021-06281-7&rft.issn=0022-2461&rft_id=https%3A%2F%2Flink.springer.com%2F10.1007%2Fs10853-021-06281-7&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:12-16\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:12_16-0\">16.0<\/a><\/sup> <sup><a href=\"#cite_ref-:12_16-1\">16.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Seifrid, Martin; Pollice, Robert; Aguilar-Granda, Andr\u00e9s; Morgan Chan, Zamyla; Hotta, Kazuhiro; Ser, Cher Tian; Vestfrid, Jenya; Wu, Tony C. <i>et al.<\/i> (6 September 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acs.accounts.2c00220\" target=\"_blank\">\"Autonomous Chemical Experiments: Challenges and Perspectives on Establishing a Self-Driving Lab\"<\/a> (in en). <i>Accounts of Chemical Research<\/i> <b>55<\/b> (17): 2454\u20132466. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1021%2Facs.accounts.2c00220\" target=\"_blank\">10.1021\/acs.accounts.2c00220<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0001-4842\" target=\"_blank\">0001-4842<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC9454899\/\" target=\"_blank\">PMC9454899<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/35948428\" target=\"_blank\">35948428<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acs.accounts.2c00220\" target=\"_blank\">https:\/\/pubs.acs.org\/doi\/10.1021\/acs.accounts.2c00220<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Autonomous+Chemical+Experiments%3A+Challenges+and+Perspectives+on+Establishing+a+Self-Driving+Lab&rft.jtitle=Accounts+of+Chemical+Research&rft.aulast=Seifrid&rft.aufirst=Martin&rft.au=Seifrid%2C%26%2332%3BMartin&rft.au=Pollice%2C%26%2332%3BRobert&rft.au=Aguilar-Granda%2C%26%2332%3BAndr%C3%A9s&rft.au=Morgan+Chan%2C%26%2332%3BZamyla&rft.au=Hotta%2C%26%2332%3BKazuhiro&rft.au=Ser%2C%26%2332%3BCher+Tian&rft.au=Vestfrid%2C%26%2332%3BJenya&rft.au=Wu%2C%26%2332%3BTony+C.&rft.au=Aspuru-Guzik%2C%26%2332%3BAl%C3%A1n&rft.date=6+September+2022&rft.volume=55&rft.issue=17&rft.pages=2454%E2%80%932466&rft_id=info:doi\/10.1021%2Facs.accounts.2c00220&rft.issn=0001-4842&rft_id=info:pmc\/PMC9454899&rft_id=info:pmid\/35948428&rft_id=https%3A%2F%2Fpubs.acs.org%2Fdoi%2F10.1021%2Facs.accounts.2c00220&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:13-17\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:13_17-0\">17.0<\/a><\/sup> <sup><a href=\"#cite_ref-:13_17-1\">17.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Abolhasani, Milad; Kumacheva, Eugenia (30 January 2023). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s44160-022-00231-0\" target=\"_blank\">\"The rise of self-driving labs in chemical and materials sciences\"<\/a> (in en). <i>Nature Synthesis<\/i> <b>2<\/b> (6): 483\u2013492. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs44160-022-00231-0\" target=\"_blank\">10.1038\/s44160-022-00231-0<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2731-0582\" target=\"_blank\">2731-0582<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s44160-022-00231-0\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s44160-022-00231-0<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+rise+of+self-driving+labs+in+chemical+and+materials+sciences&rft.jtitle=Nature+Synthesis&rft.aulast=Abolhasani&rft.aufirst=Milad&rft.au=Abolhasani%2C%26%2332%3BMilad&rft.au=Kumacheva%2C%26%2332%3BEugenia&rft.date=30+January+2023&rft.volume=2&rft.issue=6&rft.pages=483%E2%80%93492&rft_id=info:doi\/10.1038%2Fs44160-022-00231-0&rft.issn=2731-0582&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs44160-022-00231-0&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:14-18\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:14_18-0\">18.0<\/a><\/sup> <sup><a href=\"#cite_ref-:14_18-1\">18.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Matsuda, Rieko; Ishibashi, Mumio; Takeda, Yasushi (1988). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.jstage.jst.go.jp\/article\/cpb1958\/36\/9\/36_9_3512\/_article\" target=\"_blank\">\"Simplex optimization of reaction conditions with an automated system.\"<\/a> (in en). <i>Chemical and Pharmaceutical Bulletin<\/i> <b>36<\/b> (9): 3512\u20133518. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1248%2Fcpb.36.3512\" target=\"_blank\">10.1248\/cpb.36.3512<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0009-2363\" target=\"_blank\">0009-2363<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.jstage.jst.go.jp\/article\/cpb1958\/36\/9\/36_9_3512\/_article\" target=\"_blank\">http:\/\/www.jstage.jst.go.jp\/article\/cpb1958\/36\/9\/36_9_3512\/_article<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Simplex+optimization+of+reaction+conditions+with+an+automated+system.&rft.jtitle=Chemical+and+Pharmaceutical+Bulletin&rft.aulast=Matsuda&rft.aufirst=Rieko&rft.au=Matsuda%2C%26%2332%3BRieko&rft.au=Ishibashi%2C%26%2332%3BMumio&rft.au=Takeda%2C%26%2332%3BYasushi&rft.date=1988&rft.volume=36&rft.issue=9&rft.pages=3512%E2%80%933518&rft_id=info:doi\/10.1248%2Fcpb.36.3512&rft.issn=0009-2363&rft_id=http%3A%2F%2Fwww.jstage.jst.go.jp%2Farticle%2Fcpb1958%2F36%2F9%2F36_9_3512%2F_article&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:15-19\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:15_19-0\">19.0<\/a><\/sup> <sup><a href=\"#cite_ref-:15_19-1\">19.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Nikolaev, Pavel; Hooper, Daylond; Webber, Frederick; Rao, Rahul; Decker, Kevin; Krein, Michael; Poleski, Jason; Barto, Rick <i>et al.<\/i> (21 October 2016). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/npjcompumats201631\" target=\"_blank\">\"Autonomy in materials research: a case study in carbon nanotube growth\"<\/a> (in en). <i>npj Computational Materials<\/i> <b>2<\/b> (1): 16031. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fnpjcompumats.2016.31\" target=\"_blank\">10.1038\/npjcompumats.2016.31<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2057-3960\" target=\"_blank\">2057-3960<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/npjcompumats201631\" target=\"_blank\">https:\/\/www.nature.com\/articles\/npjcompumats201631<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Autonomy+in+materials+research%3A+a+case+study+in+carbon+nanotube+growth&rft.jtitle=npj+Computational+Materials&rft.aulast=Nikolaev&rft.aufirst=Pavel&rft.au=Nikolaev%2C%26%2332%3BPavel&rft.au=Hooper%2C%26%2332%3BDaylond&rft.au=Webber%2C%26%2332%3BFrederick&rft.au=Rao%2C%26%2332%3BRahul&rft.au=Decker%2C%26%2332%3BKevin&rft.au=Krein%2C%26%2332%3BMichael&rft.au=Poleski%2C%26%2332%3BJason&rft.au=Barto%2C%26%2332%3BRick&rft.au=Maruyama%2C%26%2332%3BBenji&rft.date=21+October+2016&rft.volume=2&rft.issue=1&rft.pages=16031&rft_id=info:doi\/10.1038%2Fnpjcompumats.2016.31&rft.issn=2057-3960&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fnpjcompumats201631&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:16-20\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:16_20-0\">20.0<\/a><\/sup> <sup><a href=\"#cite_ref-:16_20-1\">20.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Christensen, Melodie; Yunker, Lars P. E.; Adedeji, Folarin; H\u00e4se, Florian; Roch, Lo\u00efc M.; Gensch, Tobias; dos Passos Gomes, Gabriel; Zepel, Tara <i>et al.<\/i> (2 August 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s42004-021-00550-x\" target=\"_blank\">\"Data-science driven autonomous process optimization\"<\/a> (in en). <i>Communications Chemistry<\/i> <b>4<\/b> (1): 112. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs42004-021-00550-x\" target=\"_blank\">10.1038\/s42004-021-00550-x<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2399-3669\" target=\"_blank\">2399-3669<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC9814253\/\" target=\"_blank\">PMC9814253<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/36697524\" target=\"_blank\">36697524<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s42004-021-00550-x\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s42004-021-00550-x<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Data-science+driven+autonomous+process+optimization&rft.jtitle=Communications+Chemistry&rft.aulast=Christensen&rft.aufirst=Melodie&rft.au=Christensen%2C%26%2332%3BMelodie&rft.au=Yunker%2C%26%2332%3BLars+P.+E.&rft.au=Adedeji%2C%26%2332%3BFolarin&rft.au=H%C3%A4se%2C%26%2332%3BFlorian&rft.au=Roch%2C%26%2332%3BLo%C3%AFc+M.&rft.au=Gensch%2C%26%2332%3BTobias&rft.au=dos+Passos+Gomes%2C%26%2332%3BGabriel&rft.au=Zepel%2C%26%2332%3BTara&rft.au=Sigman%2C%26%2332%3BMatthew+S.&rft.date=2+August+2021&rft.volume=4&rft.issue=1&rft.pages=112&rft_id=info:doi\/10.1038%2Fs42004-021-00550-x&rft.issn=2399-3669&rft_id=info:pmc\/PMC9814253&rft_id=info:pmid\/36697524&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs42004-021-00550-x&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:17-21\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:17_21-0\">21.0<\/a><\/sup> <sup><a href=\"#cite_ref-:17_21-1\">21.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">McMullen, Jonathan P.; Stone, Matthew T.; Buchwald, Stephen L.; Jensen, Klavs F. (17 September 2010). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/anie.201002590\" target=\"_blank\">\"An Integrated Microreactor System for Self-Optimization of a Heck Reaction: From Micro- to Mesoscale Flow Systems\"<\/a> (in en). <i>Angewandte Chemie International Edition<\/i> <b>49<\/b> (39): 7076\u20137080. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1002%2Fanie.201002590\" target=\"_blank\">10.1002\/anie.201002590<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/anie.201002590\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/anie.201002590<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=An+Integrated+Microreactor+System+for+Self-Optimization+of+a+Heck+Reaction%3A+From+Micro-+to+Mesoscale+Flow+Systems&rft.jtitle=Angewandte+Chemie+International+Edition&rft.aulast=McMullen&rft.aufirst=Jonathan+P.&rft.au=McMullen%2C%26%2332%3BJonathan+P.&rft.au=Stone%2C%26%2332%3BMatthew+T.&rft.au=Buchwald%2C%26%2332%3BStephen+L.&rft.au=Jensen%2C%26%2332%3BKlavs+F.&rft.date=17+September+2010&rft.volume=49&rft.issue=39&rft.pages=7076%E2%80%937080&rft_id=info:doi\/10.1002%2Fanie.201002590&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1002%2Fanie.201002590&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:18-22\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:18_22-0\">22.0<\/a><\/sup> <sup><a href=\"#cite_ref-:18_22-1\">22.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Sans, Victor; Porwol, Luzian; Dragone, Vincenza; Cronin, Leroy (2015). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/xlink.rsc.org\/?DOI=C4SC03075C\" target=\"_blank\">\"A self optimizing synthetic organic reactor system using real-time in-line NMR spectroscopy\"<\/a> (in en). <i>Chemical Science<\/i> <b>6<\/b> (2): 1258\u20131264. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1039%2FC4SC03075C\" target=\"_blank\">10.1039\/C4SC03075C<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2041-6520\" target=\"_blank\">2041-6520<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC5811122\/\" target=\"_blank\">PMC5811122<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/29560211\" target=\"_blank\">29560211<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/xlink.rsc.org\/?DOI=C4SC03075C\" target=\"_blank\">http:\/\/xlink.rsc.org\/?DOI=C4SC03075C<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+self+optimizing+synthetic+organic+reactor+system+using+real-time+in-line+NMR+spectroscopy&rft.jtitle=Chemical+Science&rft.aulast=Sans&rft.aufirst=Victor&rft.au=Sans%2C%26%2332%3BVictor&rft.au=Porwol%2C%26%2332%3BLuzian&rft.au=Dragone%2C%26%2332%3BVincenza&rft.au=Cronin%2C%26%2332%3BLeroy&rft.date=2015&rft.volume=6&rft.issue=2&rft.pages=1258%E2%80%931264&rft_id=info:doi\/10.1039%2FC4SC03075C&rft.issn=2041-6520&rft_id=info:pmc\/PMC5811122&rft_id=info:pmid\/29560211&rft_id=http%3A%2F%2Fxlink.rsc.org%2F%3FDOI%3DC4SC03075C&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:19-23\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:19_23-0\">23.0<\/a><\/sup> <sup><a href=\"#cite_ref-:19_23-1\">23.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">King, Ross D.; Rowland, Jem; Oliver, Stephen G.; Young, Michael; Aubrey, Wayne; Byrne, Emma; Liakata, Maria; Markham, Magdalena <i>et al.<\/i> (3 April 2009). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.science.org\/doi\/10.1126\/science.1165620\" target=\"_blank\">\"The Automation of Science\"<\/a> (in en). <i>Science<\/i> <b>324<\/b> (5923): 85\u201389. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1126%2Fscience.1165620\" target=\"_blank\">10.1126\/science.1165620<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0036-8075\" target=\"_blank\">0036-8075<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.science.org\/doi\/10.1126\/science.1165620\" target=\"_blank\">https:\/\/www.science.org\/doi\/10.1126\/science.1165620<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+Automation+of+Science&rft.jtitle=Science&rft.aulast=King&rft.aufirst=Ross+D.&rft.au=King%2C%26%2332%3BRoss+D.&rft.au=Rowland%2C%26%2332%3BJem&rft.au=Oliver%2C%26%2332%3BStephen+G.&rft.au=Young%2C%26%2332%3BMichael&rft.au=Aubrey%2C%26%2332%3BWayne&rft.au=Byrne%2C%26%2332%3BEmma&rft.au=Liakata%2C%26%2332%3BMaria&rft.au=Markham%2C%26%2332%3BMagdalena&rft.au=Pir%2C%26%2332%3BPinar&rft.date=3+April+2009&rft.volume=324&rft.issue=5923&rft.pages=85%E2%80%9389&rft_id=info:doi\/10.1126%2Fscience.1165620&rft.issn=0036-8075&rft_id=https%3A%2F%2Fwww.science.org%2Fdoi%2F10.1126%2Fscience.1165620&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:20-24\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:20_24-0\">24.0<\/a><\/sup> <sup><a href=\"#cite_ref-:20_24-1\">24.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">King, Ross D. (1 January 2011). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.scientificamerican.com\/article\/rise-of-the-robo-scientists\" target=\"_blank\">\"Rise of the Robo Scientists\"<\/a>. <i>Scientific American<\/i> <b>304<\/b> (1): 72\u201377. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fscientificamerican0111-72\" target=\"_blank\">10.1038\/scientificamerican0111-72<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0036-8733\" target=\"_blank\">0036-8733<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.scientificamerican.com\/article\/rise-of-the-robo-scientists\" target=\"_blank\">https:\/\/www.scientificamerican.com\/article\/rise-of-the-robo-scientists<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Rise+of+the+Robo+Scientists&rft.jtitle=Scientific+American&rft.aulast=King&rft.aufirst=Ross+D.&rft.au=King%2C%26%2332%3BRoss+D.&rft.date=1+January+2011&rft.volume=304&rft.issue=1&rft.pages=72%E2%80%9377&rft_id=info:doi\/10.1038%2Fscientificamerican0111-72&rft.issn=0036-8733&rft_id=https%3A%2F%2Fwww.scientificamerican.com%2Farticle%2Frise-of-the-robo-scientists&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:21-25\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:21_25-0\">25.0<\/a><\/sup> <sup><a href=\"#cite_ref-:21_25-1\">25.1<\/a><\/sup> <sup><a href=\"#cite_ref-:21_25-2\">25.2<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Burger, Benjamin; Maffettone, Phillip M.; Gusev, Vladimir V.; Aitchison, Catherine M.; Bai, Yang; Wang, Xiaoyan; Li, Xiaobo; Alston, Ben M. <i>et al.<\/i> (9 July 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41586-020-2442-2\" target=\"_blank\">\"A mobile robotic chemist\"<\/a> (in en). <i>Nature<\/i> <b>583<\/b> (7815): 237\u2013241. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41586-020-2442-2\" target=\"_blank\">10.1038\/s41586-020-2442-2<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0028-0836\" target=\"_blank\">0028-0836<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41586-020-2442-2\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41586-020-2442-2<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+mobile+robotic+chemist&rft.jtitle=Nature&rft.aulast=Burger&rft.aufirst=Benjamin&rft.au=Burger%2C%26%2332%3BBenjamin&rft.au=Maffettone%2C%26%2332%3BPhillip+M.&rft.au=Gusev%2C%26%2332%3BVladimir+V.&rft.au=Aitchison%2C%26%2332%3BCatherine+M.&rft.au=Bai%2C%26%2332%3BYang&rft.au=Wang%2C%26%2332%3BXiaoyan&rft.au=Li%2C%26%2332%3BXiaobo&rft.au=Alston%2C%26%2332%3BBen+M.&rft.au=Li%2C%26%2332%3BBuyi&rft.date=9+July+2020&rft.volume=583&rft.issue=7815&rft.pages=237%E2%80%93241&rft_id=info:doi\/10.1038%2Fs41586-020-2442-2&rft.issn=0028-0836&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41586-020-2442-2&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:22-26\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:22_26-0\">26.0<\/a><\/sup> <sup><a href=\"#cite_ref-:22_26-1\">26.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">MacLeod, B. P.; Parlane, F. G. L.; Morrissey, T. D.; H\u00e4se, F.; Roch, L. M.; Dettelbach, K. E.; Moreira, R.; Yunker, L. P. E. <i>et al.<\/i> (15 May 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.science.org\/doi\/10.1126\/sciadv.aaz8867\" target=\"_blank\">\"Self-driving laboratory for accelerated discovery of thin-film materials\"<\/a> (in en). <i>Science Advances<\/i> <b>6<\/b> (20): eaaz8867. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1126%2Fsciadv.aaz8867\" target=\"_blank\">10.1126\/sciadv.aaz8867<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2375-2548\" target=\"_blank\">2375-2548<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC7220369\/\" target=\"_blank\">PMC7220369<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/32426501\" target=\"_blank\">32426501<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.science.org\/doi\/10.1126\/sciadv.aaz8867\" target=\"_blank\">https:\/\/www.science.org\/doi\/10.1126\/sciadv.aaz8867<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Self-driving+laboratory+for+accelerated+discovery+of+thin-film+materials&rft.jtitle=Science+Advances&rft.aulast=MacLeod&rft.aufirst=B.+P.&rft.au=MacLeod%2C%26%2332%3BB.+P.&rft.au=Parlane%2C%26%2332%3BF.+G.+L.&rft.au=Morrissey%2C%26%2332%3BT.+D.&rft.au=H%C3%A4se%2C%26%2332%3BF.&rft.au=Roch%2C%26%2332%3BL.+M.&rft.au=Dettelbach%2C%26%2332%3BK.+E.&rft.au=Moreira%2C%26%2332%3BR.&rft.au=Yunker%2C%26%2332%3BL.+P.+E.&rft.au=Rooney%2C%26%2332%3BM.+B.&rft.date=15+May+2020&rft.volume=6&rft.issue=20&rft.pages=eaaz8867&rft_id=info:doi\/10.1126%2Fsciadv.aaz8867&rft.issn=2375-2548&rft_id=info:pmc\/PMC7220369&rft_id=info:pmid\/32426501&rft_id=https%3A%2F%2Fwww.science.org%2Fdoi%2F10.1126%2Fsciadv.aaz8867&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:23-27\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:23_27-0\">27.0<\/a><\/sup> <sup><a href=\"#cite_ref-:23_27-1\">27.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Krishnadasan, S.; Brown, R. J. C.; deMello, A. J.; deMello, J. C. (2007). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/xlink.rsc.org\/?DOI=b711412e\" target=\"_blank\">\"Intelligent routes to the controlled synthesis of nanoparticles\"<\/a> (in en). <i>Lab on a Chip<\/i> <b>7<\/b> (11): 1434. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1039%2Fb711412e\" target=\"_blank\">10.1039\/b711412e<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1473-0197\" target=\"_blank\">1473-0197<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/xlink.rsc.org\/?DOI=b711412e\" target=\"_blank\">http:\/\/xlink.rsc.org\/?DOI=b711412e<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Intelligent+routes+to+the+controlled+synthesis+of+nanoparticles&rft.jtitle=Lab+on+a+Chip&rft.aulast=Krishnadasan&rft.aufirst=S.&rft.au=Krishnadasan%2C%26%2332%3BS.&rft.au=Brown%2C%26%2332%3BR.+J.+C.&rft.au=deMello%2C%26%2332%3BA.+J.&rft.au=deMello%2C%26%2332%3BJ.+C.&rft.date=2007&rft.volume=7&rft.issue=11&rft.pages=1434&rft_id=info:doi\/10.1039%2Fb711412e&rft.issn=1473-0197&rft_id=http%3A%2F%2Fxlink.rsc.org%2F%3FDOI%3Db711412e&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:24-28\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:24_28-0\">28.0<\/a><\/sup> <sup><a href=\"#cite_ref-:24_28-1\">28.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Tao, Huachen; Wu, Tianyi; Kheiri, Sina; Aldeghi, Matteo; Aspuru\u2010Guzik, Al\u00e1n; Kumacheva, Eugenia (1 December 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/adfm.202106725\" target=\"_blank\">\"Self\u2010Driving Platform for Metal Nanoparticle Synthesis: Combining Microfluidics and Machine Learning\"<\/a> (in en). <i>Advanced Functional Materials<\/i> <b>31<\/b> (51): 2106725. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1002%2Fadfm.202106725\" target=\"_blank\">10.1002\/adfm.202106725<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1616-301X\" target=\"_blank\">1616-301X<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/adfm.202106725\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/adfm.202106725<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Self%E2%80%90Driving+Platform+for+Metal+Nanoparticle+Synthesis%3A+Combining+Microfluidics+and+Machine+Learning&rft.jtitle=Advanced+Functional+Materials&rft.aulast=Tao&rft.aufirst=Huachen&rft.au=Tao%2C%26%2332%3BHuachen&rft.au=Wu%2C%26%2332%3BTianyi&rft.au=Kheiri%2C%26%2332%3BSina&rft.au=Aldeghi%2C%26%2332%3BMatteo&rft.au=Aspuru%E2%80%90Guzik%2C%26%2332%3BAl%C3%A1n&rft.au=Kumacheva%2C%26%2332%3BEugenia&rft.date=1+December+2021&rft.volume=31&rft.issue=51&rft.pages=2106725&rft_id=info:doi\/10.1002%2Fadfm.202106725&rft.issn=1616-301X&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1002%2Fadfm.202106725&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:25-29\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:25_29-0\">29.0<\/a><\/sup> <sup><a href=\"#cite_ref-:25_29-1\">29.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Desai, Bimbisar; Dixon, Karen; Farrant, Elizabeth; Feng, Qixing; Gibson, Karl R.; van Hoorn, Willem P.; Mills, James; Morgan, Trevor <i>et al.<\/i> (11 April 2013). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/jm400099d\" target=\"_blank\">\"Rapid Discovery of a Novel Series of Abl Kinase Inhibitors by Application of an Integrated Microfluidic Synthesis and Screening Platform\"<\/a> (in en). <i>Journal of Medicinal Chemistry<\/i> <b>56<\/b> (7): 3033\u20133047. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1021%2Fjm400099d\" target=\"_blank\">10.1021\/jm400099d<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0022-2623\" target=\"_blank\">0022-2623<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/jm400099d\" target=\"_blank\">https:\/\/pubs.acs.org\/doi\/10.1021\/jm400099d<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Rapid+Discovery+of+a+Novel+Series+of+Abl+Kinase+Inhibitors+by+Application+of+an+Integrated+Microfluidic+Synthesis+and+Screening+Platform&rft.jtitle=Journal+of+Medicinal+Chemistry&rft.aulast=Desai&rft.aufirst=Bimbisar&rft.au=Desai%2C%26%2332%3BBimbisar&rft.au=Dixon%2C%26%2332%3BKaren&rft.au=Farrant%2C%26%2332%3BElizabeth&rft.au=Feng%2C%26%2332%3BQixing&rft.au=Gibson%2C%26%2332%3BKarl+R.&rft.au=van+Hoorn%2C%26%2332%3BWillem+P.&rft.au=Mills%2C%26%2332%3BJames&rft.au=Morgan%2C%26%2332%3BTrevor&rft.au=Parry%2C%26%2332%3BDavid+M.&rft.date=11+April+2013&rft.volume=56&rft.issue=7&rft.pages=3033%E2%80%933047&rft_id=info:doi\/10.1021%2Fjm400099d&rft.issn=0022-2623&rft_id=https%3A%2F%2Fpubs.acs.org%2Fdoi%2F10.1021%2Fjm400099d&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:26-30\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:26_30-0\">30.0<\/a><\/sup> <sup><a href=\"#cite_ref-:26_30-1\">30.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Dave, Adarsh; Mitchell, Jared; Kandasamy, Kirthevasan; Wang, Han; Burke, Sven; Paria, Biswajit; P\u00f3czos, Barnab\u00e1s; Whitacre, Jay <i>et al.<\/i> (1 December 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2666386420302861\" target=\"_blank\">\"Autonomous Discovery of Battery Electrolytes with Robotic Experimentation and Machine Learning\"<\/a> (in en). <i>Cell Reports Physical Science<\/i> <b>1<\/b> (12): 100264. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.xcrp.2020.100264\" target=\"_blank\">10.1016\/j.xcrp.2020.100264<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2666386420302861\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2666386420302861<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Autonomous+Discovery+of+Battery+Electrolytes+with+Robotic+Experimentation+and+Machine+Learning&rft.jtitle=Cell+Reports+Physical+Science&rft.aulast=Dave&rft.aufirst=Adarsh&rft.au=Dave%2C%26%2332%3BAdarsh&rft.au=Mitchell%2C%26%2332%3BJared&rft.au=Kandasamy%2C%26%2332%3BKirthevasan&rft.au=Wang%2C%26%2332%3BHan&rft.au=Burke%2C%26%2332%3BSven&rft.au=Paria%2C%26%2332%3BBiswajit&rft.au=P%C3%B3czos%2C%26%2332%3BBarnab%C3%A1s&rft.au=Whitacre%2C%26%2332%3BJay&rft.au=Viswanathan%2C%26%2332%3BVenkatasubramanian&rft.date=1+December+2020&rft.volume=1&rft.issue=12&rft.pages=100264&rft_id=info:doi\/10.1016%2Fj.xcrp.2020.100264&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2666386420302861&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-31\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-31\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Winicov, H.; Schainbaum, J.; Buckley, J.; Longino, G.; Hill, J.; Berkoff, C.E. (1 December 1978). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S000326700183110X\" target=\"_blank\">\"Chemical process optimization by computer \u2014 a self-directed chemical synthesis system\"<\/a> (in en). <i>Analytica Chimica Acta<\/i> <b>103<\/b> (4): 469\u2013476. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2FS0003-2670%2801%2983110-X\" target=\"_blank\">10.1016\/S0003-2670(01)83110-X<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S000326700183110X\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S000326700183110X<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Chemical+process+optimization+by+computer+%E2%80%94+a+self-directed+chemical+synthesis+system&rft.jtitle=Analytica+Chimica+Acta&rft.aulast=Winicov&rft.aufirst=H.&rft.au=Winicov%2C%26%2332%3BH.&rft.au=Schainbaum%2C%26%2332%3BJ.&rft.au=Buckley%2C%26%2332%3BJ.&rft.au=Longino%2C%26%2332%3BG.&rft.au=Hill%2C%26%2332%3BJ.&rft.au=Berkoff%2C%26%2332%3BC.E.&rft.date=1+December+1978&rft.volume=103&rft.issue=4&rft.pages=469%E2%80%93476&rft_id=info:doi\/10.1016%2FS0003-2670%2801%2983110-X&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS000326700183110X&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-32\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-32\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Chang, Jorge; Nikolaev, Pavel; Carpena-N\u00fa\u00f1ez, Jennifer; Rao, Rahul; Decker, Kevin; Islam, Ahmad E.; Kim, Jiseob; Pitt, Mark A. <i>et al.<\/i> (3 June 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41598-020-64397-3\" target=\"_blank\">\"Efficient Closed-loop Maximization of Carbon Nanotube Growth Rate using Bayesian Optimization\"<\/a> (in en). <i>Scientific Reports<\/i> <b>10<\/b> (1): 9040. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41598-020-64397-3\" target=\"_blank\">10.1038\/s41598-020-64397-3<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2045-2322\" target=\"_blank\">2045-2322<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC7271124\/\" target=\"_blank\">PMC7271124<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/32493911\" target=\"_blank\">32493911<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41598-020-64397-3\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41598-020-64397-3<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Efficient+Closed-loop+Maximization+of+Carbon+Nanotube+Growth+Rate+using+Bayesian+Optimization&rft.jtitle=Scientific+Reports&rft.aulast=Chang&rft.aufirst=Jorge&rft.au=Chang%2C%26%2332%3BJorge&rft.au=Nikolaev%2C%26%2332%3BPavel&rft.au=Carpena-N%C3%BA%C3%B1ez%2C%26%2332%3BJennifer&rft.au=Rao%2C%26%2332%3BRahul&rft.au=Decker%2C%26%2332%3BKevin&rft.au=Islam%2C%26%2332%3BAhmad+E.&rft.au=Kim%2C%26%2332%3BJiseob&rft.au=Pitt%2C%26%2332%3BMark+A.&rft.au=Myung%2C%26%2332%3BJay+I.&rft.date=3+June+2020&rft.volume=10&rft.issue=1&rft.pages=9040&rft_id=info:doi\/10.1038%2Fs41598-020-64397-3&rft.issn=2045-2322&rft_id=info:pmc\/PMC7271124&rft_id=info:pmid\/32493911&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41598-020-64397-3&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-33\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-33\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Roch, Lo\u00efc M.; H\u00e4se, Florian; Kreisbeck, Christoph; Tamayo-Mendoza, Teresa; Yunker, Lars P. E.; Hein, Jason E.; Aspuru-Guzik, Al\u00e1n (20 June 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.science.org\/doi\/10.1126\/scirobotics.aat5559\" target=\"_blank\">\"ChemOS: Orchestrating autonomous experimentation\"<\/a> (in en). <i>Science Robotics<\/i> <b>3<\/b> (19): eaat5559. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1126%2Fscirobotics.aat5559\" target=\"_blank\">10.1126\/scirobotics.aat5559<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2470-9476\" target=\"_blank\">2470-9476<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.science.org\/doi\/10.1126\/scirobotics.aat5559\" target=\"_blank\">https:\/\/www.science.org\/doi\/10.1126\/scirobotics.aat5559<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=ChemOS%3A+Orchestrating+autonomous+experimentation&rft.jtitle=Science+Robotics&rft.aulast=Roch&rft.aufirst=Lo%C3%AFc+M.&rft.au=Roch%2C%26%2332%3BLo%C3%AFc+M.&rft.au=H%C3%A4se%2C%26%2332%3BFlorian&rft.au=Kreisbeck%2C%26%2332%3BChristoph&rft.au=Tamayo-Mendoza%2C%26%2332%3BTeresa&rft.au=Yunker%2C%26%2332%3BLars+P.+E.&rft.au=Hein%2C%26%2332%3BJason+E.&rft.au=Aspuru-Guzik%2C%26%2332%3BAl%C3%A1n&rft.date=20+June+2018&rft.volume=3&rft.issue=19&rft.pages=eaat5559&rft_id=info:doi\/10.1126%2Fscirobotics.aat5559&rft.issn=2470-9476&rft_id=https%3A%2F%2Fwww.science.org%2Fdoi%2F10.1126%2Fscirobotics.aat5559&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-34\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-34\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Roch, Lo\u00efc M.; H\u00e4se, Florian; Kreisbeck, Christoph; Tamayo-Mendoza, Teresa; Yunker, Lars P. E.; Hein, Jason E.; Aspuru-Guzik, Al\u00e1n (16 April 2020). Hu, Jianjun. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/dx.plos.org\/10.1371\/journal.pone.0229862\" target=\"_blank\">\"ChemOS: An orchestration software to democratize autonomous discovery\"<\/a> (in en). <i>PLOS ONE<\/i> <b>15<\/b> (4): e0229862. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1371%2Fjournal.pone.0229862\" target=\"_blank\">10.1371\/journal.pone.0229862<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1932-6203\" target=\"_blank\">1932-6203<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC7161969\/\" target=\"_blank\">PMC7161969<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/32298284\" target=\"_blank\">32298284<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/dx.plos.org\/10.1371\/journal.pone.0229862\" target=\"_blank\">https:\/\/dx.plos.org\/10.1371\/journal.pone.0229862<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=ChemOS%3A+An+orchestration+software+to+democratize+autonomous+discovery&rft.jtitle=PLOS+ONE&rft.aulast=Roch&rft.aufirst=Lo%C3%AFc+M.&rft.au=Roch%2C%26%2332%3BLo%C3%AFc+M.&rft.au=H%C3%A4se%2C%26%2332%3BFlorian&rft.au=Kreisbeck%2C%26%2332%3BChristoph&rft.au=Tamayo-Mendoza%2C%26%2332%3BTeresa&rft.au=Yunker%2C%26%2332%3BLars+P.+E.&rft.au=Hein%2C%26%2332%3BJason+E.&rft.au=Aspuru-Guzik%2C%26%2332%3BAl%C3%A1n&rft.date=16+April+2020&rft.volume=15&rft.issue=4&rft.pages=e0229862&rft_id=info:doi\/10.1371%2Fjournal.pone.0229862&rft.issn=1932-6203&rft_id=info:pmc\/PMC7161969&rft_id=info:pmid\/32298284&rft_id=https%3A%2F%2Fdx.plos.org%2F10.1371%2Fjournal.pone.0229862&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-35\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-35\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Pollice, Robert; dos Passos Gomes, Gabriel; Aldeghi, Matteo; Hickman, Riley J.; Krenn, Mario; Lavigne, Cyrille; Lindner-D\u2019Addario, Michael; Nigam, AkshatKumar <i>et al.<\/i> (16 February 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acs.accounts.0c00785\" target=\"_blank\">\"Data-Driven Strategies for Accelerated Materials Design\"<\/a> (in en). <i>Accounts of Chemical Research<\/i> <b>54<\/b> (4): 849\u2013860. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1021%2Facs.accounts.0c00785\" target=\"_blank\">10.1021\/acs.accounts.0c00785<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0001-4842\" target=\"_blank\">0001-4842<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC7893702\/\" target=\"_blank\">PMC7893702<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/33528245\" target=\"_blank\">33528245<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acs.accounts.0c00785\" target=\"_blank\">https:\/\/pubs.acs.org\/doi\/10.1021\/acs.accounts.0c00785<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Data-Driven+Strategies+for+Accelerated+Materials+Design&rft.jtitle=Accounts+of+Chemical+Research&rft.aulast=Pollice&rft.aufirst=Robert&rft.au=Pollice%2C%26%2332%3BRobert&rft.au=dos+Passos+Gomes%2C%26%2332%3BGabriel&rft.au=Aldeghi%2C%26%2332%3BMatteo&rft.au=Hickman%2C%26%2332%3BRiley+J.&rft.au=Krenn%2C%26%2332%3BMario&rft.au=Lavigne%2C%26%2332%3BCyrille&rft.au=Lindner-D%E2%80%99Addario%2C%26%2332%3BMichael&rft.au=Nigam%2C%26%2332%3BAkshatKumar&rft.au=Ser%2C%26%2332%3BCher+Tian&rft.date=16+February+2021&rft.volume=54&rft.issue=4&rft.pages=849%E2%80%93860&rft_id=info:doi\/10.1021%2Facs.accounts.0c00785&rft.issn=0001-4842&rft_id=info:pmc\/PMC7893702&rft_id=info:pmid\/33528245&rft_id=https%3A%2F%2Fpubs.acs.org%2Fdoi%2F10.1021%2Facs.accounts.0c00785&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-36\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-36\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">H\u00e4se, Florian; Roch, Lo\u00efc M.; Kreisbeck, Christoph; Aspuru-Guzik, Al\u00e1n (26 September 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acscentsci.8b00307\" target=\"_blank\">\"Phoenics: A Bayesian Optimizer for Chemistry\"<\/a> (in en). <i>ACS Central Science<\/i> <b>4<\/b> (9): 1134\u20131145. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1021%2Facscentsci.8b00307\" target=\"_blank\">10.1021\/acscentsci.8b00307<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2374-7943\" target=\"_blank\">2374-7943<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC6161047\/\" target=\"_blank\">PMC6161047<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/30276246\" target=\"_blank\">30276246<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acscentsci.8b00307\" target=\"_blank\">https:\/\/pubs.acs.org\/doi\/10.1021\/acscentsci.8b00307<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Phoenics%3A+A+Bayesian+Optimizer+for+Chemistry&rft.jtitle=ACS+Central+Science&rft.aulast=H%C3%A4se&rft.aufirst=Florian&rft.au=H%C3%A4se%2C%26%2332%3BFlorian&rft.au=Roch%2C%26%2332%3BLo%C3%AFc+M.&rft.au=Kreisbeck%2C%26%2332%3BChristoph&rft.au=Aspuru-Guzik%2C%26%2332%3BAl%C3%A1n&rft.date=26+September+2018&rft.volume=4&rft.issue=9&rft.pages=1134%E2%80%931145&rft_id=info:doi\/10.1021%2Facscentsci.8b00307&rft.issn=2374-7943&rft_id=info:pmc\/PMC6161047&rft_id=info:pmid\/30276246&rft_id=https%3A%2F%2Fpubs.acs.org%2Fdoi%2F10.1021%2Facscentsci.8b00307&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:27-37\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:27_37-0\">37.0<\/a><\/sup> <sup><a href=\"#cite_ref-:27_37-1\">37.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">H\u00e4se, Florian; Aldeghi, Matteo; Hickman, Riley J.; Roch, Lo\u00efc M.; Aspuru-Guzik, Al\u00e1n (1 September 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubs.aip.org\/apr\/article\/8\/3\/031406\/998861\/Gryffin-An-algorithm-for-Bayesian-optimization-of\" target=\"_blank\">\"G ryffin : An algorithm for Bayesian optimization of categorical variables informed by expert knowledge\"<\/a> (in en). <i>Applied Physics Reviews<\/i> <b>8<\/b> (3): 031406. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1063%2F5.0048164\" target=\"_blank\">10.1063\/5.0048164<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1931-9401\" target=\"_blank\">1931-9401<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubs.aip.org\/apr\/article\/8\/3\/031406\/998861\/Gryffin-An-algorithm-for-Bayesian-optimization-of\" target=\"_blank\">https:\/\/pubs.aip.org\/apr\/article\/8\/3\/031406\/998861\/Gryffin-An-algorithm-for-Bayesian-optimization-of<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=G+ryffin+%3A+An+algorithm+for+Bayesian+optimization+of+categorical+variables+informed+by+expert+knowledge&rft.jtitle=Applied+Physics+Reviews&rft.aulast=H%C3%A4se&rft.aufirst=Florian&rft.au=H%C3%A4se%2C%26%2332%3BFlorian&rft.au=Aldeghi%2C%26%2332%3BMatteo&rft.au=Hickman%2C%26%2332%3BRiley+J.&rft.au=Roch%2C%26%2332%3BLo%C3%AFc+M.&rft.au=Aspuru-Guzik%2C%26%2332%3BAl%C3%A1n&rft.date=1+September+2021&rft.volume=8&rft.issue=3&rft.pages=031406&rft_id=info:doi\/10.1063%2F5.0048164&rft.issn=1931-9401&rft_id=https%3A%2F%2Fpubs.aip.org%2Fapr%2Farticle%2F8%2F3%2F031406%2F998861%2FGryffin-An-algorithm-for-Bayesian-optimization-of&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-38\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-38\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Reizman, Brandon J.; Wang, Yi-Ming; Buchwald, Stephen L.; Jensen, Klavs F. (2016). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/xlink.rsc.org\/?DOI=C6RE00153J\" target=\"_blank\">\"Suzuki\u2013Miyaura cross-coupling optimization enabled by automated feedback\"<\/a> (in en). <i>Reaction Chemistry & Engineering<\/i> <b>1<\/b> (6): 658\u2013666. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1039%2FC6RE00153J\" target=\"_blank\">10.1039\/C6RE00153J<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2058-9883\" target=\"_blank\">2058-9883<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC5123644\/\" target=\"_blank\">PMC5123644<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/27928513\" target=\"_blank\">27928513<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/xlink.rsc.org\/?DOI=C6RE00153J\" target=\"_blank\">http:\/\/xlink.rsc.org\/?DOI=C6RE00153J<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Suzuki%E2%80%93Miyaura+cross-coupling+optimization+enabled+by+automated+feedback&rft.jtitle=Reaction+Chemistry+%26+Engineering&rft.aulast=Reizman&rft.aufirst=Brandon+J.&rft.au=Reizman%2C%26%2332%3BBrandon+J.&rft.au=Wang%2C%26%2332%3BYi-Ming&rft.au=Buchwald%2C%26%2332%3BStephen+L.&rft.au=Jensen%2C%26%2332%3BKlavs+F.&rft.date=2016&rft.volume=1&rft.issue=6&rft.pages=658%E2%80%93666&rft_id=info:doi\/10.1039%2FC6RE00153J&rft.issn=2058-9883&rft_id=info:pmc\/PMC5123644&rft_id=info:pmid\/27928513&rft_id=http%3A%2F%2Fxlink.rsc.org%2F%3FDOI%3DC6RE00153J&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-39\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-39\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Baumgartner, Lorenz M.; Coley, Connor W.; Reizman, Brandon J.; Gao, Kevin W.; Jensen, Klavs F. (2018). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/xlink.rsc.org\/?DOI=C8RE00032H\" target=\"_blank\">\"Optimum catalyst selection over continuous and discrete process variables with a single droplet microfluidic reaction platform\"<\/a> (in en). <i>Reaction Chemistry & Engineering<\/i> <b>3<\/b> (3): 301\u2013311. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1039%2FC8RE00032H\" target=\"_blank\">10.1039\/C8RE00032H<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2058-9883\" target=\"_blank\">2058-9883<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/xlink.rsc.org\/?DOI=C8RE00032H\" target=\"_blank\">http:\/\/xlink.rsc.org\/?DOI=C8RE00032H<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Optimum+catalyst+selection+over+continuous+and+discrete+process+variables+with+a+single+droplet+microfluidic+reaction+platform&rft.jtitle=Reaction+Chemistry+%26+Engineering&rft.aulast=Baumgartner&rft.aufirst=Lorenz+M.&rft.au=Baumgartner%2C%26%2332%3BLorenz+M.&rft.au=Coley%2C%26%2332%3BConnor+W.&rft.au=Reizman%2C%26%2332%3BBrandon+J.&rft.au=Gao%2C%26%2332%3BKevin+W.&rft.au=Jensen%2C%26%2332%3BKlavs+F.&rft.date=2018&rft.volume=3&rft.issue=3&rft.pages=301%E2%80%93311&rft_id=info:doi\/10.1039%2FC8RE00032H&rft.issn=2058-9883&rft_id=http%3A%2F%2Fxlink.rsc.org%2F%3FDOI%3DC8RE00032H&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-40\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-40\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">B\u00e9dard, Anne-Catherine; Adamo, Andrea; Aroh, Kosi C.; Russell, M. Grace; Bedermann, Aaron A.; Torosian, Jeremy; Yue, Brian; Jensen, Klavs F. <i>et al.<\/i> (21 September 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.science.org\/doi\/10.1126\/science.aat0650\" target=\"_blank\">\"Reconfigurable system for automated optimization of diverse chemical reactions\"<\/a> (in en). <i>Science<\/i> <b>361<\/b> (6408): 1220\u20131225. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1126%2Fscience.aat0650\" target=\"_blank\">10.1126\/science.aat0650<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0036-8075\" target=\"_blank\">0036-8075<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.science.org\/doi\/10.1126\/science.aat0650\" target=\"_blank\">https:\/\/www.science.org\/doi\/10.1126\/science.aat0650<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Reconfigurable+system+for+automated+optimization+of+diverse+chemical+reactions&rft.jtitle=Science&rft.aulast=B%C3%A9dard&rft.aufirst=Anne-Catherine&rft.au=B%C3%A9dard%2C%26%2332%3BAnne-Catherine&rft.au=Adamo%2C%26%2332%3BAndrea&rft.au=Aroh%2C%26%2332%3BKosi+C.&rft.au=Russell%2C%26%2332%3BM.+Grace&rft.au=Bedermann%2C%26%2332%3BAaron+A.&rft.au=Torosian%2C%26%2332%3BJeremy&rft.au=Yue%2C%26%2332%3BBrian&rft.au=Jensen%2C%26%2332%3BKlavs+F.&rft.au=Jamison%2C%26%2332%3BTimothy+F.&rft.date=21+September+2018&rft.volume=361&rft.issue=6408&rft.pages=1220%E2%80%931225&rft_id=info:doi\/10.1126%2Fscience.aat0650&rft.issn=0036-8075&rft_id=https%3A%2F%2Fwww.science.org%2Fdoi%2F10.1126%2Fscience.aat0650&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-41\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-41\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Granda, Jaros\u0142aw M.; Donina, Liva; Dragone, Vincenza; Long, De-Liang; Cronin, Leroy (1 July 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41586-018-0307-8\" target=\"_blank\">\"Controlling an organic synthesis robot with machine learning to search for new reactivity\"<\/a> (in en). <i>Nature<\/i> <b>559<\/b> (7714): 377\u2013381. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41586-018-0307-8\" target=\"_blank\">10.1038\/s41586-018-0307-8<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0028-0836\" target=\"_blank\">0028-0836<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC6223543\/\" target=\"_blank\">PMC6223543<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/30022133\" target=\"_blank\">30022133<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41586-018-0307-8\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41586-018-0307-8<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Controlling+an+organic+synthesis+robot+with+machine+learning+to+search+for+new+reactivity&rft.jtitle=Nature&rft.aulast=Granda&rft.aufirst=Jaros%C5%82aw+M.&rft.au=Granda%2C%26%2332%3BJaros%C5%82aw+M.&rft.au=Donina%2C%26%2332%3BLiva&rft.au=Dragone%2C%26%2332%3BVincenza&rft.au=Long%2C%26%2332%3BDe-Liang&rft.au=Cronin%2C%26%2332%3BLeroy&rft.date=1+July+2018&rft.volume=559&rft.issue=7714&rft.pages=377%E2%80%93381&rft_id=info:doi\/10.1038%2Fs41586-018-0307-8&rft.issn=0028-0836&rft_id=info:pmc\/PMC6223543&rft_id=info:pmid\/30022133&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41586-018-0307-8&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-42\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-42\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Asche, Silke; Cooper, Geoffrey J. T.; Keenan, Graham; Mathis, Cole; Cronin, Leroy (10 June 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41467-021-23828-z\" target=\"_blank\">\"A robotic prebiotic chemist probes long term reactions of complexifying mixtures\"<\/a> (in en). <i>Nature Communications<\/i> <b>12<\/b> (1): 3547. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41467-021-23828-z\" target=\"_blank\">10.1038\/s41467-021-23828-z<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2041-1723\" target=\"_blank\">2041-1723<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC8192940\/\" target=\"_blank\">PMC8192940<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/34112788\" target=\"_blank\">34112788<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41467-021-23828-z\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41467-021-23828-z<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+robotic+prebiotic+chemist+probes+long+term+reactions+of+complexifying+mixtures&rft.jtitle=Nature+Communications&rft.aulast=Asche&rft.aufirst=Silke&rft.au=Asche%2C%26%2332%3BSilke&rft.au=Cooper%2C%26%2332%3BGeoffrey+J.+T.&rft.au=Keenan%2C%26%2332%3BGraham&rft.au=Mathis%2C%26%2332%3BCole&rft.au=Cronin%2C%26%2332%3BLeroy&rft.date=10+June+2021&rft.volume=12&rft.issue=1&rft.pages=3547&rft_id=info:doi\/10.1038%2Fs41467-021-23828-z&rft.issn=2041-1723&rft_id=info:pmc\/PMC8192940&rft_id=info:pmid\/34112788&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41467-021-23828-z&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-43\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-43\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Bornemann\u2010Pfeiffer, Martin; Wolf, Jakob; Meyer, Klas; Kern, Simon; Angelone, Davide; Leonov, Artem; Cronin, Leroy; Emmerling, Franziska (18 October 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/anie.202106323\" target=\"_blank\">\"Standardization and Control of Grignard Reactions in a Universal Chemical Synthesis Machine using online NMR\"<\/a> (in en). <i>Angewandte Chemie International Edition<\/i> <b>60<\/b> (43): 23202\u201323206. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1002%2Fanie.202106323\" target=\"_blank\">10.1002\/anie.202106323<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1433-7851\" target=\"_blank\">1433-7851<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC8597166\/\" target=\"_blank\">PMC8597166<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/34278673\" target=\"_blank\">34278673<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/anie.202106323\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/anie.202106323<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Standardization+and+Control+of+Grignard+Reactions+in+a+Universal+Chemical+Synthesis+Machine+using+online+NMR&rft.jtitle=Angewandte+Chemie+International+Edition&rft.aulast=Bornemann%E2%80%90Pfeiffer&rft.aufirst=Martin&rft.au=Bornemann%E2%80%90Pfeiffer%2C%26%2332%3BMartin&rft.au=Wolf%2C%26%2332%3BJakob&rft.au=Meyer%2C%26%2332%3BKlas&rft.au=Kern%2C%26%2332%3BSimon&rft.au=Angelone%2C%26%2332%3BDavide&rft.au=Leonov%2C%26%2332%3BArtem&rft.au=Cronin%2C%26%2332%3BLeroy&rft.au=Emmerling%2C%26%2332%3BFranziska&rft.date=18+October+2021&rft.volume=60&rft.issue=43&rft.pages=23202%E2%80%9323206&rft_id=info:doi\/10.1002%2Fanie.202106323&rft.issn=1433-7851&rft_id=info:pmc\/PMC8597166&rft_id=info:pmid\/34278673&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1002%2Fanie.202106323&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-44\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-44\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">MacLeod, Benjamin P.; Parlane, Fraser G. L.; Rupnow, Connor C.; Dettelbach, Kevan E.; Elliott, Michael S.; Morrissey, Thomas D.; Haley, Ted H.; Proskurin, Oleksii <i>et al.<\/i> (22 February 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41467-022-28580-6\" target=\"_blank\">\"A self-driving laboratory advances the Pareto front for material properties\"<\/a> (in en). <i>Nature Communications<\/i> <b>13<\/b> (1): 995. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41467-022-28580-6\" target=\"_blank\">10.1038\/s41467-022-28580-6<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2041-1723\" target=\"_blank\">2041-1723<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC8863835\/\" target=\"_blank\">PMC8863835<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/35194074\" target=\"_blank\">35194074<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41467-022-28580-6\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41467-022-28580-6<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+self-driving+laboratory+advances+the+Pareto+front+for+material+properties&rft.jtitle=Nature+Communications&rft.aulast=MacLeod&rft.aufirst=Benjamin+P.&rft.au=MacLeod%2C%26%2332%3BBenjamin+P.&rft.au=Parlane%2C%26%2332%3BFraser+G.+L.&rft.au=Rupnow%2C%26%2332%3BConnor+C.&rft.au=Dettelbach%2C%26%2332%3BKevan+E.&rft.au=Elliott%2C%26%2332%3BMichael+S.&rft.au=Morrissey%2C%26%2332%3BThomas+D.&rft.au=Haley%2C%26%2332%3BTed+H.&rft.au=Proskurin%2C%26%2332%3BOleksii&rft.au=Rooney%2C%26%2332%3BMichael+B.&rft.date=22+February+2022&rft.volume=13&rft.issue=1&rft.pages=995&rft_id=info:doi\/10.1038%2Fs41467-022-28580-6&rft.issn=2041-1723&rft_id=info:pmc\/PMC8863835&rft_id=info:pmid\/35194074&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41467-022-28580-6&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-45\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-45\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Wu, Tony C; Aguilar\u2010Granda, Andr\u00e9s; Hotta, Kazuhiro; Yazdani, Sahar Alasvand; Pollice, Robert; Vestfrid, Jenya; Hao, Han; Lavigne, Cyrille <i>et al.<\/i> (1 February 2023). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/adma.202207070\" target=\"_blank\">\"A Materials Acceleration Platform for Organic Laser Discovery\"<\/a> (in en). <i>Advanced Materials<\/i> <b>35<\/b> (6): 2207070. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1002%2Fadma.202207070\" target=\"_blank\">10.1002\/adma.202207070<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0935-9648\" target=\"_blank\">0935-9648<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/adma.202207070\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/adma.202207070<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Materials+Acceleration+Platform+for+Organic+Laser+Discovery&rft.jtitle=Advanced+Materials&rft.aulast=Wu&rft.aufirst=Tony+C&rft.au=Wu%2C%26%2332%3BTony+C&rft.au=Aguilar%E2%80%90Granda%2C%26%2332%3BAndr%C3%A9s&rft.au=Hotta%2C%26%2332%3BKazuhiro&rft.au=Yazdani%2C%26%2332%3BSahar+Alasvand&rft.au=Pollice%2C%26%2332%3BRobert&rft.au=Vestfrid%2C%26%2332%3BJenya&rft.au=Hao%2C%26%2332%3BHan&rft.au=Lavigne%2C%26%2332%3BCyrille&rft.au=Seifrid%2C%26%2332%3BMartin&rft.date=1+February+2023&rft.volume=35&rft.issue=6&rft.pages=2207070&rft_id=info:doi\/10.1002%2Fadma.202207070&rft.issn=0935-9648&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1002%2Fadma.202207070&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-46\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-46\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Epps, Robert W.; Bowen, Michael S.; Volk, Amanda A.; Abdel\u2010Latif, Kameel; Han, Suyong; Reyes, Kristofer G.; Amassian, Aram; Abolhasani, Milad (1 July 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/adma.202001626\" target=\"_blank\">\"Artificial Chemist: An Autonomous Quantum Dot Synthesis Bot\"<\/a> (in en). <i>Advanced Materials<\/i> <b>32<\/b> (30): 2001626. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1002%2Fadma.202001626\" target=\"_blank\">10.1002\/adma.202001626<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0935-9648\" target=\"_blank\">0935-9648<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/adma.202001626\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/adma.202001626<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Artificial+Chemist%3A+An+Autonomous+Quantum+Dot+Synthesis+Bot&rft.jtitle=Advanced+Materials&rft.aulast=Epps&rft.aufirst=Robert+W.&rft.au=Epps%2C%26%2332%3BRobert+W.&rft.au=Bowen%2C%26%2332%3BMichael+S.&rft.au=Volk%2C%26%2332%3BAmanda+A.&rft.au=Abdel%E2%80%90Latif%2C%26%2332%3BKameel&rft.au=Han%2C%26%2332%3BSuyong&rft.au=Reyes%2C%26%2332%3BKristofer+G.&rft.au=Amassian%2C%26%2332%3BAram&rft.au=Abolhasani%2C%26%2332%3BMilad&rft.date=1+July+2020&rft.volume=32&rft.issue=30&rft.pages=2001626&rft_id=info:doi\/10.1002%2Fadma.202001626&rft.issn=0935-9648&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1002%2Fadma.202001626&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-47\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-47\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Abdel-Latif, Kameel; Epps, Robert W.; Bateni, Fazel; Han, Suyong; Reyes, Kristofer G.; Abolhasani, Milad (1 February 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/aisy.202000245\" target=\"_blank\">\"Self\u2010Driven Multistep Quantum Dot Synthesis Enabled by Autonomous Robotic Experimentation in Flow\"<\/a> (in en). <i>Advanced Intelligent Systems<\/i> <b>3<\/b> (2): 2000245. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1002%2Faisy.202000245\" target=\"_blank\">10.1002\/aisy.202000245<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2640-4567\" target=\"_blank\">2640-4567<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/aisy.202000245\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/aisy.202000245<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Self%E2%80%90Driven+Multistep+Quantum+Dot+Synthesis+Enabled+by+Autonomous+Robotic+Experimentation+in+Flow&rft.jtitle=Advanced+Intelligent+Systems&rft.aulast=Abdel-Latif&rft.aufirst=Kameel&rft.au=Abdel-Latif%2C%26%2332%3BKameel&rft.au=Epps%2C%26%2332%3BRobert+W.&rft.au=Bateni%2C%26%2332%3BFazel&rft.au=Han%2C%26%2332%3BSuyong&rft.au=Reyes%2C%26%2332%3BKristofer+G.&rft.au=Abolhasani%2C%26%2332%3BMilad&rft.date=1+February+2021&rft.volume=3&rft.issue=2&rft.pages=2000245&rft_id=info:doi\/10.1002%2Faisy.202000245&rft.issn=2640-4567&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1002%2Faisy.202000245&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-48\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-48\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Mekki-Berrada, Flore; Ren, Zekun; Huang, Tan; Wong, Wai Kuan; Zheng, Fang; Xie, Jiaxun; Tian, Isaac Parker Siyu; Jayavelu, Senthilnath <i>et al.<\/i> (20 April 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41524-021-00520-w\" target=\"_blank\">\"Two-step machine learning enables optimized nanoparticle synthesis\"<\/a> (in en). <i>npj Computational Materials<\/i> <b>7<\/b> (1): 55. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41524-021-00520-w\" target=\"_blank\">10.1038\/s41524-021-00520-w<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2057-3960\" target=\"_blank\">2057-3960<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41524-021-00520-w\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41524-021-00520-w<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Two-step+machine+learning+enables+optimized+nanoparticle+synthesis&rft.jtitle=npj+Computational+Materials&rft.aulast=Mekki-Berrada&rft.aufirst=Flore&rft.au=Mekki-Berrada%2C%26%2332%3BFlore&rft.au=Ren%2C%26%2332%3BZekun&rft.au=Huang%2C%26%2332%3BTan&rft.au=Wong%2C%26%2332%3BWai+Kuan&rft.au=Zheng%2C%26%2332%3BFang&rft.au=Xie%2C%26%2332%3BJiaxun&rft.au=Tian%2C%26%2332%3BIsaac+Parker+Siyu&rft.au=Jayavelu%2C%26%2332%3BSenthilnath&rft.au=Mahfoud%2C%26%2332%3BZackaria&rft.date=20+April+2021&rft.volume=7&rft.issue=1&rft.pages=55&rft_id=info:doi\/10.1038%2Fs41524-021-00520-w&rft.issn=2057-3960&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41524-021-00520-w&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-49\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-49\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Li, Jiagen; Lu, Yihua; Xu, Yanheng; Liu, Chongfeng; Tu, Yuxiao; Ye, Shuqian; Liu, Haochen; Xie, Yi <i>et al.<\/i> (21 November 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acs.jpca.8b10680\" target=\"_blank\">\"AIR-Chem: Authentic Intelligent Robotics for Chemistry\"<\/a> (in en). <i>The Journal of Physical Chemistry A<\/i> <b>122<\/b> (46): 9142\u20139148. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1021%2Facs.jpca.8b10680\" target=\"_blank\">10.1021\/acs.jpca.8b10680<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1089-5639\" target=\"_blank\">1089-5639<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acs.jpca.8b10680\" target=\"_blank\">https:\/\/pubs.acs.org\/doi\/10.1021\/acs.jpca.8b10680<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=AIR-Chem%3A+Authentic+Intelligent+Robotics+for+Chemistry&rft.jtitle=The+Journal+of+Physical+Chemistry+A&rft.aulast=Li&rft.aufirst=Jiagen&rft.au=Li%2C%26%2332%3BJiagen&rft.au=Lu%2C%26%2332%3BYihua&rft.au=Xu%2C%26%2332%3BYanheng&rft.au=Liu%2C%26%2332%3BChongfeng&rft.au=Tu%2C%26%2332%3BYuxiao&rft.au=Ye%2C%26%2332%3BShuqian&rft.au=Liu%2C%26%2332%3BHaochen&rft.au=Xie%2C%26%2332%3BYi&rft.au=Qian%2C%26%2332%3BHuihuan&rft.date=21+November+2018&rft.volume=122&rft.issue=46&rft.pages=9142%E2%80%939148&rft_id=info:doi\/10.1021%2Facs.jpca.8b10680&rft.issn=1089-5639&rft_id=https%3A%2F%2Fpubs.acs.org%2Fdoi%2F10.1021%2Facs.jpca.8b10680&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-50\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-50\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Li, Jiagen; Li, Junzi; Liu, Rulin; Tu, Yuxiao; Li, Yiwen; Cheng, Jiaji; He, Tingchao; Zhu, Xi (27 April 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41467-020-15728-5\" target=\"_blank\">\"Autonomous discovery of optically active chiral inorganic perovskite nanocrystals through an intelligent cloud lab\"<\/a> (in en). <i>Nature Communications<\/i> <b>11<\/b> (1): 2046. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41467-020-15728-5\" target=\"_blank\">10.1038\/s41467-020-15728-5<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2041-1723\" target=\"_blank\">2041-1723<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC7184584\/\" target=\"_blank\">PMC7184584<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/32341340\" target=\"_blank\">32341340<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41467-020-15728-5\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41467-020-15728-5<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Autonomous+discovery+of+optically+active+chiral+inorganic+perovskite+nanocrystals+through+an+intelligent+cloud+lab&rft.jtitle=Nature+Communications&rft.aulast=Li&rft.aufirst=Jiagen&rft.au=Li%2C%26%2332%3BJiagen&rft.au=Li%2C%26%2332%3BJunzi&rft.au=Liu%2C%26%2332%3BRulin&rft.au=Tu%2C%26%2332%3BYuxiao&rft.au=Li%2C%26%2332%3BYiwen&rft.au=Cheng%2C%26%2332%3BJiaji&rft.au=He%2C%26%2332%3BTingchao&rft.au=Zhu%2C%26%2332%3BXi&rft.date=27+April+2020&rft.volume=11&rft.issue=1&rft.pages=2046&rft_id=info:doi\/10.1038%2Fs41467-020-15728-5&rft.issn=2041-1723&rft_id=info:pmc\/PMC7184584&rft_id=info:pmid\/32341340&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41467-020-15728-5&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-51\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-51\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">von Drigalski, Felix; Tanaka, Kazutoshi; Hamaya, Masashi; Lee, Robert; Nakashima, Chisato; Shibata, Yoshiya; Ijiri, Yoshihisa (24 October 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9341487\/\" target=\"_blank\">\"A Compact, Cable-driven, Activatable Soft Wrist with Six Degrees of Freedom for Assembly Tasks\"<\/a>. <i>2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)<\/i> (Las Vegas, NV, USA: IEEE): 8752\u20138757. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FIROS45743.2020.9341487\" target=\"_blank\">10.1109\/IROS45743.2020.9341487<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-7281-6212-6<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9341487\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/9341487\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Compact%2C+Cable-driven%2C+Activatable+Soft+Wrist+with+Six+Degrees+of+Freedom+for+Assembly+Tasks&rft.jtitle=2020+IEEE%2FRSJ+International+Conference+on+Intelligent+Robots+and+Systems+%28IROS%29&rft.aulast=von+Drigalski&rft.aufirst=Felix&rft.au=von+Drigalski%2C%26%2332%3BFelix&rft.au=Tanaka%2C%26%2332%3BKazutoshi&rft.au=Hamaya%2C%26%2332%3BMasashi&rft.au=Lee%2C%26%2332%3BRobert&rft.au=Nakashima%2C%26%2332%3BChisato&rft.au=Shibata%2C%26%2332%3BYoshiya&rft.au=Ijiri%2C%26%2332%3BYoshihisa&rft.date=24+October+2020&rft.pages=8752%E2%80%938757&rft.place=Las+Vegas%2C+NV%2C+USA&rft.pub=IEEE&rft_id=info:doi\/10.1109%2FIROS45743.2020.9341487&rft.isbn=978-1-7281-6212-6&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F9341487%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-52\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-52\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Hamaya, Masashi; Lee, Robert; Tanaka, Kazutoshi; von Drigalski, Felix; Nakashima, Chisato; Shibata, Yoshiya; Ijiri, Yoshihisa (1 May 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9197327\/\" target=\"_blank\">\"Learning Robotic Assembly Tasks with Lower Dimensional Systems by Leveraging Physical Softness and Environmental Constraints\"<\/a>. <i>2020 IEEE International Conference on Robotics and Automation (ICRA)<\/i> (Paris, France: IEEE): 7747\u20137753. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FICRA40945.2020.9197327\" target=\"_blank\">10.1109\/ICRA40945.2020.9197327<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-7281-7395-5<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9197327\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/9197327\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Learning+Robotic+Assembly+Tasks+with+Lower+Dimensional+Systems+by+Leveraging+Physical+Softness+and+Environmental+Constraints&rft.jtitle=2020+IEEE+International+Conference+on+Robotics+and+Automation+%28ICRA%29&rft.aulast=Hamaya&rft.aufirst=Masashi&rft.au=Hamaya%2C%26%2332%3BMasashi&rft.au=Lee%2C%26%2332%3BRobert&rft.au=Tanaka%2C%26%2332%3BKazutoshi&rft.au=von+Drigalski%2C%26%2332%3BFelix&rft.au=Nakashima%2C%26%2332%3BChisato&rft.au=Shibata%2C%26%2332%3BYoshiya&rft.au=Ijiri%2C%26%2332%3BYoshihisa&rft.date=1+May+2020&rft.pages=7747%E2%80%937753&rft.place=Paris%2C+France&rft.pub=IEEE&rft_id=info:doi\/10.1109%2FICRA40945.2020.9197327&rft.isbn=978-1-7281-7395-5&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F9197327%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-53\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-53\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ge, M.; Su, F.; Zhao, Z.; Su, D. (1 August 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S258884202030016X\" target=\"_blank\">\"Deep learning analysis on microscopic imaging in materials science\"<\/a> (in en). <i>Materials Today Nano<\/i> <b>11<\/b>: 100087. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.mtnano.2020.100087\" target=\"_blank\">10.1016\/j.mtnano.2020.100087<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S258884202030016X\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S258884202030016X<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Deep+learning+analysis+on+microscopic+imaging+in+materials+science&rft.jtitle=Materials+Today+Nano&rft.aulast=Ge&rft.aufirst=M.&rft.au=Ge%2C%26%2332%3BM.&rft.au=Su%2C%26%2332%3BF.&rft.au=Zhao%2C%26%2332%3BZ.&rft.au=Su%2C%26%2332%3BD.&rft.date=1+August+2020&rft.volume=11&rft.pages=100087&rft_id=info:doi\/10.1016%2Fj.mtnano.2020.100087&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS258884202030016X&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-54\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-54\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Suzuki, Yuta; Hino, Hideitsu; Kotsugi, Masato; Ono, Kanta (29 March 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41524-019-0176-1\" target=\"_blank\">\"Automated estimation of materials parameter from X-ray absorption and electron energy-loss spectra with similarity measures\"<\/a> (in en). <i>npj Computational Materials<\/i> <b>5<\/b> (1): 39. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41524-019-0176-1\" target=\"_blank\">10.1038\/s41524-019-0176-1<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2057-3960\" target=\"_blank\">2057-3960<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41524-019-0176-1\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41524-019-0176-1<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Automated+estimation+of+materials+parameter+from+X-ray+absorption+and+electron+energy-loss+spectra+with+similarity+measures&rft.jtitle=npj+Computational+Materials&rft.aulast=Suzuki&rft.aufirst=Yuta&rft.au=Suzuki%2C%26%2332%3BYuta&rft.au=Hino%2C%26%2332%3BHideitsu&rft.au=Kotsugi%2C%26%2332%3BMasato&rft.au=Ono%2C%26%2332%3BKanta&rft.date=29+March+2019&rft.volume=5&rft.issue=1&rft.pages=39&rft_id=info:doi\/10.1038%2Fs41524-019-0176-1&rft.issn=2057-3960&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41524-019-0176-1&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-55\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-55\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Tshitoyan, Vahe; Dagdelen, John; Weston, Leigh; Dunn, Alexander; Rong, Ziqin; Kononova, Olga; Persson, Kristin A.; Ceder, Gerbrand <i>et al.<\/i> (1 July 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41586-019-1335-8\" target=\"_blank\">\"Unsupervised word embeddings capture latent knowledge from materials science literature\"<\/a> (in en). <i>Nature<\/i> <b>571<\/b> (7763): 95\u201398. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41586-019-1335-8\" target=\"_blank\">10.1038\/s41586-019-1335-8<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0028-0836\" target=\"_blank\">0028-0836<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41586-019-1335-8\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41586-019-1335-8<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Unsupervised+word+embeddings+capture+latent+knowledge+from+materials+science+literature&rft.jtitle=Nature&rft.aulast=Tshitoyan&rft.aufirst=Vahe&rft.au=Tshitoyan%2C%26%2332%3BVahe&rft.au=Dagdelen%2C%26%2332%3BJohn&rft.au=Weston%2C%26%2332%3BLeigh&rft.au=Dunn%2C%26%2332%3BAlexander&rft.au=Rong%2C%26%2332%3BZiqin&rft.au=Kononova%2C%26%2332%3BOlga&rft.au=Persson%2C%26%2332%3BKristin+A.&rft.au=Ceder%2C%26%2332%3BGerbrand&rft.au=Jain%2C%26%2332%3BAnubhav&rft.date=1+July+2019&rft.volume=571&rft.issue=7763&rft.pages=95%E2%80%9398&rft_id=info:doi\/10.1038%2Fs41586-019-1335-8&rft.issn=0028-0836&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41586-019-1335-8&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-56\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-56\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Abbott, Ryan (2016). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ssrn.com\/abstract=2727884\" target=\"_blank\">\"I Think, Therefore I Invent: Creative Computers and the Future of Patent Law\"<\/a> (in en). <i>SSRN Electronic Journal<\/i>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.2139%2Fssrn.2727884\" target=\"_blank\">10.2139\/ssrn.2727884<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1556-5068\" target=\"_blank\">1556-5068<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.ssrn.com\/abstract=2727884\" target=\"_blank\">http:\/\/www.ssrn.com\/abstract=2727884<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=I+Think%2C+Therefore+I+Invent%3A+Creative+Computers+and+the+Future+of+Patent+Law&rft.jtitle=SSRN+Electronic+Journal&rft.aulast=Abbott&rft.aufirst=Ryan&rft.au=Abbott%2C%26%2332%3BRyan&rft.date=2016&rft_id=info:doi\/10.2139%2Fssrn.2727884&rft.issn=1556-5068&rft_id=http%3A%2F%2Fwww.ssrn.com%2Fabstract%3D2727884&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-57\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-57\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Chen, A. (8 January 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.technologyreview.com\/2020\/01\/08\/102298\/ai-inventor-patent-dabus-intellectual-property-uk-european-patent-office-law\/\" target=\"_blank\">\"Can an AI be an inventor? Not yet.\"<\/a>. <i>MIT Technology Review<\/i>. Massachusetts Institute of Technology<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.technologyreview.com\/2020\/01\/08\/102298\/ai-inventor-patent-dabus-intellectual-property-uk-european-patent-office-law\/\" target=\"_blank\">https:\/\/www.technologyreview.com\/2020\/01\/08\/102298\/ai-inventor-patent-dabus-intellectual-property-uk-european-patent-office-law\/<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 19 November 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Can+an+AI+be+an+inventor%3F+Not+yet.&rft.atitle=MIT+Technology+Review&rft.aulast=Chen%2C+A.&rft.au=Chen%2C+A.&rft.date=8+January+2020&rft.pub=Massachusetts+Institute+of+Technology&rft_id=https%3A%2F%2Fwww.technologyreview.com%2F2020%2F01%2F08%2F102298%2Fai-inventor-patent-dabus-intellectual-property-uk-european-patent-office-law%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:28-58\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:28_58-0\">58.0<\/a><\/sup> <sup><a href=\"#cite_ref-:28_58-1\">58.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Carlson, Erika K. (1 November 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S209580992030254X\" target=\"_blank\">\"Artificial Intelligence Can Invent But Not Patent\u2014For Now\"<\/a> (in en). <i>Engineering<\/i> <b>6<\/b> (11): 1212\u20131213. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.eng.2020.09.003\" target=\"_blank\">10.1016\/j.eng.2020.09.003<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S209580992030254X\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S209580992030254X<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Artificial+Intelligence+Can+Invent+But+Not+Patent%E2%80%94For+Now&rft.jtitle=Engineering&rft.aulast=Carlson&rft.aufirst=Erika+K.&rft.au=Carlson%2C%26%2332%3BErika+K.&rft.date=1+November+2020&rft.volume=6&rft.issue=11&rft.pages=1212%E2%80%931213&rft_id=info:doi\/10.1016%2Fj.eng.2020.09.003&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS209580992030254X&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:29-59\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:29_59-0\">59.0<\/a><\/sup> <sup><a href=\"#cite_ref-:29_59-1\">59.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation web\">Abbott, R.; Austin, R.; Davidson, J.R. et al. (2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/artificialinventor.com\/\" target=\"_blank\">\"The Artificial Inventor Project\"<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/artificialinventor.com\/\" target=\"_blank\">https:\/\/artificialinventor.com\/<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 19 November 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=The+Artificial+Inventor+Project&rft.atitle=&rft.aulast=Abbott%2C+R.%3B+Austin%2C+R.%3B+Davidson%2C+J.R.+et+al.&rft.au=Abbott%2C+R.%3B+Austin%2C+R.%3B+Davidson%2C+J.R.+et+al.&rft.date=2022&rft_id=https%3A%2F%2Fartificialinventor.com%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-60\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-60\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">DABUS (2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/iponline.cipc.co.za\/Publications\/PublishedJournals\/E_Journal_July%202021%20Part%202.pdf\" target=\"_blank\">\"ZA 2021\/03242 - Food Container and Devices and Methods for Attracting Enhanced Attention\"<\/a> (PDF). <i>Patent Journal<\/i> <b>54<\/b> (7): 255. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2223-4837\" target=\"_blank\">2223-4837<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/iponline.cipc.co.za\/Publications\/PublishedJournals\/E_Journal_July%202021%20Part%202.pdf\" target=\"_blank\">https:\/\/iponline.cipc.co.za\/Publications\/PublishedJournals\/E_Journal_July%202021%20Part%202.pdf<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=ZA+2021%2F03242+-+Food+Container+and+Devices+and+Methods+for+Attracting+Enhanced+Attention&rft.jtitle=Patent+Journal&rft.aulast=DABUS&rft.au=DABUS&rft.date=2021&rft.volume=54&rft.issue=7&rft.pages=255&rft.issn=2223-4837&rft_id=https%3A%2F%2Fiponline.cipc.co.za%2FPublications%2FPublishedJournals%2FE_Journal_July%25202021%2520Part%25202.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-61\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-61\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.judgments.fedcourt.gov.au\/judgments\/Judgments\/fca\/full\/2022\/2022fcafc0062\" target=\"_blank\">\"Commissioner of Patents v Thaler [2022<\/a> FCAFC 62\"]. Federal Court of Australia. 13 April 2022<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.judgments.fedcourt.gov.au\/judgments\/Judgments\/fca\/full\/2022\/2022fcafc0062\" target=\"_blank\">https:\/\/www.judgments.fedcourt.gov.au\/judgments\/Judgments\/fca\/full\/2022\/2022fcafc0062<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 19 November 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Commissioner+of+Patents+v+Thaler+%5B2022%5D+FCAFC+62&rft.atitle=&rft.date=13+April+2022&rft.pub=Federal+Court+of+Australia&rft_id=https%3A%2F%2Fwww.judgments.fedcourt.gov.au%2Fjudgments%2FJudgments%2Ffca%2Ffull%2F2022%2F2022fcafc0062&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-62\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-62\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.epo.org\/news-events\/news\/2021\/20211221.html\" target=\"_blank\">\"AI cannot be named as inventor on patent applications\"<\/a>. European Patent Office. 21 December 2021<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.epo.org\/news-events\/news\/2021\/20211221.html\" target=\"_blank\">https:\/\/www.epo.org\/news-events\/news\/2021\/20211221.html<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 19 November 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=AI+cannot+be+named+as+inventor+on+patent+applications&rft.atitle=&rft.date=21+December+2021&rft.pub=European+Patent+Office&rft_id=https%3A%2F%2Fwww.epo.org%2Fnews-events%2Fnews%2F2021%2F20211221.html&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-63\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-63\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.jpo.go.jp\/system\/process\/shutugan\/hatsumei.html\" target=\"_blank\">\"\u767a\u660e\u8005\u7b49\u306e\u8868\u793a\u306b\u3064\u3044\u3066\"<\/a>. Japan Patent Office. 30 July 2021<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.jpo.go.jp\/system\/process\/shutugan\/hatsumei.html\" target=\"_blank\">https:\/\/www.jpo.go.jp\/system\/process\/shutugan\/hatsumei.html<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 19 November 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=%E7%99%BA%E6%98%8E%E8%80%85%E7%AD%89%E3%81%AE%E8%A1%A8%E7%A4%BA%E3%81%AB%E3%81%A4%E3%81%84%E3%81%A6&rft.atitle=&rft.date=30+July+2021&rft.pub=Japan+Patent+Office&rft_id=https%3A%2F%2Fwww.jpo.go.jp%2Fsystem%2Fprocess%2Fshutugan%2Fhatsumei.html&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-64\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-64\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.jpo.go.jp\/resources\/shingikai\/sangyo-kouzou\/shousai\/tokkyo_shoi\/document\/seisakubukai-06-shiryou\/paper07_1.pdf\" target=\"_blank\">\"\u65e5\u672c\u306b\u304a\u3051\u308b\u767a\u660e\u8005\u306e\u6c7a\u5b9a\"<\/a> (PDF). Japan Patent Office<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.jpo.go.jp\/resources\/shingikai\/sangyo-kouzou\/shousai\/tokkyo_shoi\/document\/seisakubukai-06-shiryou\/paper07_1.pdf\" target=\"_blank\">https:\/\/www.jpo.go.jp\/resources\/shingikai\/sangyo-kouzou\/shousai\/tokkyo_shoi\/document\/seisakubukai-06-shiryou\/paper07_1.pdf<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 19 November 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=%E6%97%A5%E6%9C%AC%E3%81%AB%E3%81%8A%E3%81%91%E3%82%8B%E7%99%BA%E6%98%8E%E8%80%85%E3%81%AE%E6%B1%BA%E5%AE%9A&rft.atitle=&rft.pub=Japan+Patent+Office&rft_id=https%3A%2F%2Fwww.jpo.go.jp%2Fresources%2Fshingikai%2Fsangyo-kouzou%2Fshousai%2Ftokkyo_shoi%2Fdocument%2Fseisakubukai-06-shiryou%2Fpaper07_1.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-65\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-65\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Kageyama, K. (1 October 2010). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/academic.oup.com\/jiplp\/article-lookup\/doi\/10.1093\/jiplp\/jpq097\" target=\"_blank\">\"Formation of invention\/joint invention and recognition of inventor\/joint inventor\"<\/a> (in en). <i>Journal of Intellectual Property Law & Practice<\/i> <b>5<\/b> (10): 699\u2013712. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1093%2Fjiplp%2Fjpq097\" target=\"_blank\">10.1093\/jiplp\/jpq097<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1747-1532\" target=\"_blank\">1747-1532<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/academic.oup.com\/jiplp\/article-lookup\/doi\/10.1093\/jiplp\/jpq097\" target=\"_blank\">https:\/\/academic.oup.com\/jiplp\/article-lookup\/doi\/10.1093\/jiplp\/jpq097<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Formation+of+invention%2Fjoint+invention+and+recognition+of+inventor%2Fjoint+inventor&rft.jtitle=Journal+of+Intellectual+Property+Law+%26+Practice&rft.aulast=Kageyama&rft.aufirst=K.&rft.au=Kageyama%2C%26%2332%3BK.&rft.date=1+October+2010&rft.volume=5&rft.issue=10&rft.pages=699%E2%80%93712&rft_id=info:doi\/10.1093%2Fjiplp%2Fjpq097&rft.issn=1747-1532&rft_id=https%3A%2F%2Facademic.oup.com%2Fjiplp%2Farticle-lookup%2Fdoi%2F10.1093%2Fjiplp%2Fjpq097&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-66\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-66\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.jpo.go.jp\/e\/system\/laws\/rule\/guideline\/patent\/tukujitu_kijun\/document\/index\/03_0202_e.pdf\" target=\"_blank\">\"Section 2 - Inventive Step\"<\/a> (PDF). Japan Patent Office<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.jpo.go.jp\/e\/system\/laws\/rule\/guideline\/patent\/tukujitu_kijun\/document\/index\/03_0202_e.pdf\" target=\"_blank\">https:\/\/www.jpo.go.jp\/e\/system\/laws\/rule\/guideline\/patent\/tukujitu_kijun\/document\/index\/03_0202_e.pdf<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 19 November 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Section+2+-+Inventive+Step&rft.atitle=&rft.pub=Japan+Patent+Office&rft_id=https%3A%2F%2Fwww.jpo.go.jp%2Fe%2Fsystem%2Flaws%2Frule%2Fguideline%2Fpatent%2Ftukujitu_kijun%2Fdocument%2Findex%2F03_0202_e.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-67\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-67\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Nakayama, I. (2019). \"AI and Inventive Step \u2013 Proposal of Issues\". <i>Patent<\/i> <b>72<\/b> (12): 179\u201399.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=AI+and+Inventive+Step+%E2%80%93+Proposal+of+Issues&rft.jtitle=Patent&rft.aulast=Nakayama%2C+I.&rft.au=Nakayama%2C+I.&rft.date=2019&rft.volume=72&rft.issue=12&rft.pages=179%E2%80%9399&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-68\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-68\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Abbott, Ryan (2017). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.ssrn.com\/abstract=3056915\" target=\"_blank\">\"Everything is Obvious\"<\/a> (in en). <i>SSRN Electronic Journal<\/i>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.2139%2Fssrn.3056915\" target=\"_blank\">10.2139\/ssrn.3056915<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1556-5068\" target=\"_blank\">1556-5068<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.ssrn.com\/abstract=3056915\" target=\"_blank\">https:\/\/www.ssrn.com\/abstract=3056915<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Everything+is+Obvious&rft.jtitle=SSRN+Electronic+Journal&rft.aulast=Abbott&rft.aufirst=Ryan&rft.au=Abbott%2C%26%2332%3BRyan&rft.date=2017&rft_id=info:doi\/10.2139%2Fssrn.3056915&rft.issn=1556-5068&rft_id=https%3A%2F%2Fwww.ssrn.com%2Fabstract%3D3056915&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-69\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-69\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.weforum.org\/whitepapers\/artificial-intelligence-collides-with-patent-law\" target=\"_blank\">\"Artificial Intelligence Collides with Patent Law\"<\/a>. World Economic Forum. 20 April 2018<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.weforum.org\/whitepapers\/artificial-intelligence-collides-with-patent-law\" target=\"_blank\">https:\/\/www.weforum.org\/whitepapers\/artificial-intelligence-collides-with-patent-law<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 19 November 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Artificial+Intelligence+Collides+with+Patent+Law&rft.atitle=&rft.date=20+April+2018&rft.pub=World+Economic+Forum&rft_id=https%3A%2F%2Fwww.weforum.org%2Fwhitepapers%2Fartificial-intelligence-collides-with-patent-law&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-70\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-70\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.wipo.int\/export\/sites\/www\/about-ip\/en\/artificial_intelligence\/call_for_comments\/pdf\/org_ccia.pdf\" target=\"_blank\">\"CCIA Comments on WIPO draft issues paper on IP and AI\"<\/a> (PDF). Computer & Communications Industry Association. 14 February 2020<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.wipo.int\/export\/sites\/www\/about-ip\/en\/artificial_intelligence\/call_for_comments\/pdf\/org_ccia.pdf\" target=\"_blank\">https:\/\/www.wipo.int\/export\/sites\/www\/about-ip\/en\/artificial_intelligence\/call_for_comments\/pdf\/org_ccia.pdf<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 19 November 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=CCIA+Comments+on+WIPO+draft+issues+paper+on+IP+and+AI&rft.atitle=&rft.date=14+February+2020&rft.pub=Computer+%26+Communications+Industry+Association&rft_id=https%3A%2F%2Fwww.wipo.int%2Fexport%2Fsites%2Fwww%2Fabout-ip%2Fen%2Fartificial_intelligence%2Fcall_for_comments%2Fpdf%2Forg_ccia.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-71\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-71\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ramalho, Ana (2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.ssrn.com\/abstract=3168703\" target=\"_blank\">\"Patentability of AI-Generated Inventions: Is a Reform of the Patent System Needed?\"<\/a> (in en). <i>SSRN Electronic Journal<\/i>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.2139%2Fssrn.3168703\" target=\"_blank\">10.2139\/ssrn.3168703<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1556-5068\" target=\"_blank\">1556-5068<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.ssrn.com\/abstract=3168703\" target=\"_blank\">https:\/\/www.ssrn.com\/abstract=3168703<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Patentability+of+AI-Generated+Inventions%3A+Is+a+Reform+of+the+Patent+System+Needed%3F&rft.jtitle=SSRN+Electronic+Journal&rft.aulast=Ramalho&rft.aufirst=Ana&rft.au=Ramalho%2C%26%2332%3BAna&rft.date=2018&rft_id=info:doi\/10.2139%2Fssrn.3168703&rft.issn=1556-5068&rft_id=https%3A%2F%2Fwww.ssrn.com%2Fabstract%3D3168703&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-72\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-72\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\"> <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/publicationethics.org\/node\/34946\" target=\"_blank\"><i>COPE Discussion Document: Authorship<\/i><\/a>. 1 June 2014. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.24318%2Fcope.2019.3.3\" target=\"_blank\">10.24318\/cope.2019.3.3<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/publicationethics.org\/node\/34946\" target=\"_blank\">https:\/\/publicationethics.org\/node\/34946<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=COPE+Discussion+Document%3A+Authorship&rft.date=1+June+2014&rft_id=info:doi\/10.24318%2Fcope.2019.3.3&rft_id=https%3A%2F%2Fpublicationethics.org%2Fnode%2F34946&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-73\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-73\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.icmje.org\/recommendations\/browse\/roles-and-responsibilities\/defining-the-role-of-authors-and-contributors.html\" target=\"_blank\">\"Defining the Role of Authors and Contributors\"<\/a>. International Committee of Medical Journal Editors<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.icmje.org\/recommendations\/browse\/roles-and-responsibilities\/defining-the-role-of-authors-and-contributors.html\" target=\"_blank\">http:\/\/www.icmje.org\/recommendations\/browse\/roles-and-responsibilities\/defining-the-role-of-authors-and-contributors.html<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 19 November 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Defining+the+Role+of+Authors+and+Contributors&rft.atitle=&rft.pub=International+Committee+of+Medical+Journal+Editors&rft_id=http%3A%2F%2Fwww.icmje.org%2Frecommendations%2Fbrowse%2Froles-and-responsibilities%2Fdefining-the-role-of-authors-and-contributors.html&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-74\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-74\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Generative Pre-trained Transformer, ChatGPT; Zhavoronkov, Alex (21 December 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.oncoscience.us\/lookup\/doi\/10.18632\/oncoscience.571\" target=\"_blank\">\"Rapamycin in the context of Pascal\u2019s Wager: generative pre-trained transformer perspective\"<\/a> (in en). <i>Oncoscience<\/i> <b>9<\/b>: 82\u201384. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.18632%2Foncoscience.571\" target=\"_blank\">10.18632\/oncoscience.571<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2331-4737\" target=\"_blank\">2331-4737<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC9796173\/\" target=\"_blank\">PMC9796173<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/36589923\" target=\"_blank\">36589923<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.oncoscience.us\/lookup\/doi\/10.18632\/oncoscience.571\" target=\"_blank\">https:\/\/www.oncoscience.us\/lookup\/doi\/10.18632\/oncoscience.571<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Rapamycin+in+the+context+of+Pascal%E2%80%99s+Wager%3A+generative+pre-trained+transformer+perspective&rft.jtitle=Oncoscience&rft.aulast=Generative+Pre-trained+Transformer&rft.aufirst=ChatGPT&rft.au=Generative+Pre-trained+Transformer%2C%26%2332%3BChatGPT&rft.au=Zhavoronkov%2C%26%2332%3BAlex&rft.date=21+December+2022&rft.volume=9&rft.pages=82%E2%80%9384&rft_id=info:doi\/10.18632%2Foncoscience.571&rft.issn=2331-4737&rft_id=info:pmc\/PMC9796173&rft_id=info:pmid\/36589923&rft_id=https%3A%2F%2Fwww.oncoscience.us%2Flookup%2Fdoi%2F10.18632%2Foncoscience.571&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-75\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-75\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">O\u2019Connor, Siobhan; ChatGPT (1 January 2023). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1471595322002517\" target=\"_blank\">\"Open artificial intelligence platforms in nursing education: Tools for academic progress or abuse?\"<\/a> (in en). <i>Nurse Education in Practice<\/i> <b>66<\/b>: 103537. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.nepr.2022.103537\" target=\"_blank\">10.1016\/j.nepr.2022.103537<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1471595322002517\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1471595322002517<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Open+artificial+intelligence+platforms+in+nursing+education%3A+Tools+for+academic+progress+or+abuse%3F&rft.jtitle=Nurse+Education+in+Practice&rft.aulast=O%E2%80%99Connor&rft.aufirst=Siobhan&rft.au=O%E2%80%99Connor%2C%26%2332%3BSiobhan&rft.au=ChatGPT&rft.date=1+January+2023&rft.volume=66&rft.pages=103537&rft_id=info:doi\/10.1016%2Fj.nepr.2022.103537&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS1471595322002517&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-76\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-76\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Stokel-Walker, Chris (26 January 2023). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/d41586-023-00107-z\" target=\"_blank\">\"ChatGPT listed as author on research papers: many scientists disapprove\"<\/a> (in en). <i>Nature<\/i> <b>613<\/b> (7945): 620\u2013621. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fd41586-023-00107-z\" target=\"_blank\">10.1038\/d41586-023-00107-z<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0028-0836\" target=\"_blank\">0028-0836<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/d41586-023-00107-z\" target=\"_blank\">https:\/\/www.nature.com\/articles\/d41586-023-00107-z<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=ChatGPT+listed+as+author+on+research+papers%3A+many+scientists+disapprove&rft.jtitle=Nature&rft.aulast=Stokel-Walker&rft.aufirst=Chris&rft.au=Stokel-Walker%2C%26%2332%3BChris&rft.date=26+January+2023&rft.volume=613&rft.issue=7945&rft.pages=620%E2%80%93621&rft_id=info:doi\/10.1038%2Fd41586-023-00107-z&rft.issn=0028-0836&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fd41586-023-00107-z&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-77\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-77\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Parviainen, Jaana; Coeckelbergh, Mark (1 September 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/link.springer.com\/10.1007\/s00146-020-01104-w\" target=\"_blank\">\"The political choreography of the Sophia robot: beyond robot rights and citizenship to political performances for the social robotics market\"<\/a> (in en). <i>AI & SOCIETY<\/i> <b>36<\/b> (3): 715\u2013724. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2Fs00146-020-01104-w\" target=\"_blank\">10.1007\/s00146-020-01104-w<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0951-5666\" target=\"_blank\">0951-5666<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/link.springer.com\/10.1007\/s00146-020-01104-w\" target=\"_blank\">https:\/\/link.springer.com\/10.1007\/s00146-020-01104-w<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+political+choreography+of+the+Sophia+robot%3A+beyond+robot+rights+and+citizenship+to+political+performances+for+the+social+robotics+market&rft.jtitle=AI+%26+SOCIETY&rft.aulast=Parviainen&rft.aufirst=Jaana&rft.au=Parviainen%2C%26%2332%3BJaana&rft.au=Coeckelbergh%2C%26%2332%3BMark&rft.date=1+September+2021&rft.volume=36&rft.issue=3&rft.pages=715%E2%80%93724&rft_id=info:doi\/10.1007%2Fs00146-020-01104-w&rft.issn=0951-5666&rft_id=https%3A%2F%2Flink.springer.com%2F10.1007%2Fs00146-020-01104-w&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-78\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-78\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/legalinstruments.oecd.org\/en\/instruments\/OECD-LEGAL-0449\" target=\"_blank\">\"Recommendation of the Council on Artificial Intelligence\"<\/a>. <i>OECD Legal Instruments<\/i>. OECD. 21 May 2019<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/legalinstruments.oecd.org\/en\/instruments\/OECD-LEGAL-0449\" target=\"_blank\">https:\/\/legalinstruments.oecd.org\/en\/instruments\/OECD-LEGAL-0449<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 19 November 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Recommendation+of+the+Council+on+Artificial+Intelligence&rft.atitle=OECD+Legal+Instruments&rft.date=21+May+2019&rft.pub=OECD&rft_id=https%3A%2F%2Flegalinstruments.oecd.org%2Fen%2Finstruments%2FOECD-LEGAL-0449&rfr_id=info:sid\/en.wikipedia.org:Journal:Autonomous_experimental_systems_in_materials_science\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<\/ol><\/div><\/div>\n<h2><span class=\"mw-headline\" id=\"Notes\">Notes<\/span><\/h2>\n<p>This presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added. In the original, there are multiple instances of citing research work using the last name of the last author listed, rather than the last name of the first author listed; this may have been a product of Japanese culture tending to read text from right to left. For this version, the last name of the first author was used to be consistent with research norms.\n<\/p>\n<!-- \nNewPP limit report\nCached time: 20231215050306\nCache expiry: 86400\nDynamic content: false\nComplications: []\nCPU time usage: 1.279 seconds\nReal time usage: 1.580 seconds\nPreprocessor visited node count: 79361\/1000000\nPost\u2010expand include size: 689743\/2097152 bytes\nTemplate argument size: 200817\/2097152 bytes\nHighest expansion depth: 25\/40\nExpensive parser function count: 0\/100\nUnstrip recursion depth: 0\/20\nUnstrip post\u2010expand size: 192478\/5000000 bytes\n-->\n<!--\nTransclusion expansion time report (%,ms,calls,template)\n100.00% 1017.081 1 -total\n 91.84% 934.085 1 Template:Reflist\n 75.46% 767.520 78 Template:Citation\/core\n 71.18% 723.978 67 Template:Cite_journal\n 13.09% 133.142 73 Template:Date\n 9.01% 91.600 11 Template:Cite_web\n 8.17% 83.046 153 Template:Citation\/identifier\n 3.02% 30.675 1 Template:Infobox_journal_article\n 2.75% 27.978 306 Template:Hide_in_print\n 2.59% 26.377 1 Template:Infobox\n-->\n\n<!-- Saved in parser cache with key limswiki:pcache:idhash:14368-0!canonical and timestamp 20231215050304 and revision id 52946. Serialized with JSON.\n -->\n<\/div><\/div><div class=\"printfooter\">Source: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.limswiki.org\/index.php\/Journal:Autonomous_experimental_systems_in_materials_science\">https:\/\/www.limswiki.org\/index.php\/Journal:Autonomous_experimental_systems_in_materials_science<\/a><\/div>\n<!-- end content --><div class=\"visualClear\"><\/div><\/div><\/div><div class=\"visualClear\"><\/div><\/div><!-- end of the left (by default at least) column --><div class=\"visualClear\"><\/div><\/div>\n\n\n\n<\/body>","537370f6a0e7e5345701b0b5a2fb2d41_images":["https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/9\/9b\/GA_Ishizuki_SciTechAdvMatMeth2023_3-1.jpg","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/d\/d6\/Fig1_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/a\/a8\/Fig2_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/e\/e3\/Fig3_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/0\/0a\/Fig4_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/a\/a3\/Fig5_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/9\/9b\/Fig6_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/a\/ae\/Fig7_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/9\/9b\/Fig8_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg"],"537370f6a0e7e5345701b0b5a2fb2d41_timestamp":1702682171,"affac9ff82db9d386600be5eb3d77056_type":"article","affac9ff82db9d386600be5eb3d77056_title":"FAIR Health Informatics: A health informatics framework for verifiable and explainable data analysis (Siddiqi et al. 2023)","affac9ff82db9d386600be5eb3d77056_url":"https:\/\/www.limswiki.org\/index.php\/Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis","affac9ff82db9d386600be5eb3d77056_plaintext":"\n\nJournal:FAIR Health Informatics: A health informatics framework for verifiable and explainable data analysisFrom LIMSWikiJump to navigationJump to searchFull article title\n \nFAIR Health Informatics: A health informatics framework for verifiable and explainable data analysisJournal\n \nHealthcareAuthor(s)\n \nSiddiqi, Muhammad H.; Idris, Muhammad; Alruwaili, MadallahAuthor affiliation(s)\n \nJouf University, Universite Libre de BruxellesPrimary contact\n \nEmail: mhsiddiqi at ju dot edu dot saYear published\n \n2023Volume and issue\n \n11(12)Article #\n \n1713DOI\n \n10.3390\/healthcare11121713ISSN\n \n2227-9032Distribution license\n \nCreative Commons Attribution 4.0 InternationalWebsite\n \nhttps:\/\/www.mdpi.com\/2227-9032\/11\/12\/1713Download\n \nhttps:\/\/www.mdpi.com\/2227-9032\/11\/12\/1713\/pdf?version=1686475395 (PDF)\n\nContents \n\n1 Abstract \n2 Introduction \n\n2.1 Objectives and contributions \n\n\n3 State of the art \n\n3.1 Data management \n3.2 Data interoperability and standardization \n3.3 Data analysis \n3.4 Comparison of the proposed FAIR Health Informatics framework to the state of the art \n\n\n4 Methodology \n\n4.1 Preliminaries \n\n4.1.1 Terms \n4.1.2 Processes \n\n\n4.2 Architecture \n4.3 Processes and methods \n\n4.3.1 Data sources \n4.3.2 Data on-boarding and discovery process \n4.3.3 Data management process \n4.3.4 Data correlation \n4.3.5 Data analysis \n4.3.6 Data services \n4.3.7 Lineage and reproducibility \n\n\n4.4 Use cases and scenario \n\n4.4.1 Use cases \n4.4.2 Scenarios \n\n\n\n\n5 Experiments, results, and evaluation \n\n5.1 Implementation \n5.2 Results and evaluation \n\n\n6 Discussion \n7 Conclusions \n8 Abbreviations, acronyms, and initialisms \n9 Acknowledgements \n\n9.1 Author contributions \n9.2 Funding \n9.3 Conflicts of interest \n\n\n10 References \n11 Notes \n\n\n\nAbstract \nThe recent COVID-19 pandemic has hit humanity very hard in ways rarely observed before. In this digitally connected world, the health informatics and clinical research domains (both public and private) lack a robust framework to enable rapid investigation and cures. Since data in the healthcare domain are highly confidential, any framework in the healthcare domain must work on real data, be verifiable, and support reproducibility for evidence purposes. In this paper, we propose a health informatics framework that supports data acquisition from various sources in real-time, correlates these data from various sources among each other and to the domain-specific terminologies, and supports querying and analyses. Various sources include sensory data from wearable sensors, clinical investigation (for trials and devices) data from private\/public agencies, personal health records, academic publications in the healthcare domain, and semantic information such as clinical ontologies and the Medical Subject Headings (MeSH) ontology. The linking and correlation of various sources include mapping personal wearable data to health records, clinical oncology terms to clinical trials, and so on. The framework is designed such that the data are findable, accessible, interoperable, and reusable (FAIR) with proper identity and access management mechanisms. This practically means tracing and linking each step in the data management lifecycle through discovery, ease of access and exchange, and data reuse. We present a practical use case to correlate a variety of aspects of data relating to a certain medical subject heading from the MeSH ontology and academic publications with clinical investigation data. The proposed architecture supports streaming data acquisition, and servicing and processing changes throughout the lifecycle of the data management process. This is necessary in certain events, such as when the status of a certain clinical or other health-related investigation needs to be updated. In such cases, it is required to track and view the outline of those events for the analysis and traceability of the clinical investigation and to define interventions if necessary.\nKeywords: data correlation, data linking, verifiable data, data analysis, explainable decisions, clinical trials, COVID, clinical investigation, semantic mapping, smart health\n\nIntroduction \nPandemics are not new to this world or humanity. There have been pandemics in the past, and they may happen again in the future. The recent COVID-19 pandemic is different from the previous ones in that the virus is more infectious without being known, symptoms are ambiguous, and the detection methods require a lot of time and resources. It has caused more deaths than ever in the history of humankind, and the impact it has had on the world economy and human lives (whether affected or not) is grave and is posing questions about the future of diseases and pandemics. At the same time, while humans advance knowledge and technology, there is a need to investigate and put effort into overcoming the challenges posed by these kinds of serious threats. This not only requires us to deal with the current pandemic but also to look into the future, predict and presume the possibilities, investigate, and develop solutions at a large scale so that there is a reduction in the risk of losing lives and danger on a large scale.\nThe majority of the existing software systems and solutions in the healthcare domain are proprietary and limited in their capacity to a specific domain, such as only processing clinical trial data without integrating state-of-the-art investigation and wearable sensor data. Hence, they lack the ability to present a scalable analytical and technical solution, while having limitations and lacking the ability to trace back the analysis and results to the origin of the data. Because of these limitations, any robust and practical solution does not only have to account for clinical data but also present a practical and broader overview of these catastrophic events from clinical investigations, their results and treatments that are up-to-date, and combine them with medical history records as well as academic and other related datasets available. In other words, the recent advancements and investigations in the clinical domain specific to a particular disease are published in research articles and journals, and they also need to be correlated to real human subjects that undergo clinical trials so that up-to-date analysis can be carried out and proper guidelines and interventions can be suggested.\nMoreover, the majority of the record-keeping bodies maintain electronic health records (EHRs) for patients and, recently, records of COVID vaccinations. However, they lack the ability to link the data to individuals\u2019 activities and clinical outcomes. There is also a lack of fusing data related to the investigation of a particular disease from various data providers, such as private clinical trials, public clinical trials[1][2][3], and public-private clinical trials. The lack of these services is not only because there is less literature on fusing these multiple forms of data but also because of security and privacy concerns related to the confidentiality of healthcare data. The current advanced and robust privacy and security infrastructure available is more than enough to ensure personal and organizational interests. On the one hand, there are governments and other organizations that publish their clinical research data to public repositories to be available for clinical research using defined standards. On the other hand, there are pharma companies, which mostly hold the analytics driven by these and the respective algorithms and methods private. Furthermore, clinical trial data are not enough since they only provide measurements for different subjects who underwent a trial, while other data, from sources such as EHRs, contain real investigative cases and the histories of patients. Therefore, fusing these data from a variety of sources is helpful to determine the effects of a particular drug or treatment plan in combination with other vaccines, treatments, etc.\nGiven the above brief overview of the capabilities of the state-of-the-art, most of these systems either tackle static or dynamic, relational or non-relational, noisy or cleaned data without fusing, integrating, or semantically linking it. This paper proposes a solution to the above problems by presenting a healthcare framework that supports the ability to acquire, manage, and process static and dynamic (real-time) data. Our proposed framework is a data format that focuses on fusing information from various sources. In a nutshell, the proposed framework ideally targets a strategy that makes data findable, accessible, interoperable, and reusable (FAIR).[4]\n\nObjectives and contributions \nIn particular, we define the objectives and contributions of this research work as proposing a framework that is designed to be able to provide the following basic and essential capabilities for healthcare data:\n\nA clinical \"data lake\" that stores data in a unified format where the raw data can be in any format loaded from a raw storage or streamed.\nPipelines that support static and incremental data collection from raw storage to a clinical data lake and maintain a record of any change to the structure at the data level and at the schema level all the way to the raw data and raw data schema.\nA schema repository that versions the data when it changes its structure and enables forward and backward compatibility of the data throughout.\nUniversal clinical schemas that can incorporate any clinically related concepts (such as clinical trials from any provider) and support flexibility.\nProcesses to perform change data capture (CDC) as the data proceed down the pipeline towards the applications, i.e., incremental algorithms and methods to enable incremental processing of incoming data and keep a log of only the data of interest.\nTimelines for changing data for a specific clinical investigation from a clinical trial data element such as a trial, site, investigator, etc., and include the ability to stream the data to the application, while providing semantic linking and profiling of subjects in the data.\nThese capabilities are meant to provide evidence of data analysis (i.e., where does the analysis go back in terms of data), traces of changes, and a holistic view of various clinical investigations running simultaneously at different places related to a certain specific clinical investigation. An example of that is when the COVID-19 vaccine was being developed; it was necessary to be able to trace the investigation of various efforts by independent bodies in a single place where one could see the phases of clinical trials of vaccines, their outcomes, treatments, and even the investigation sites and investigators. This could not only result in better decision-making but also in putting resources in suitable places and better planning for future similar scenarios.\n\nState of the art \nMany works exist in the literature that deal with various aspects of clinical data, ranging from data management and analysis to interoperability, the meaning of big data in healthcare and its future course, the standardization of clinical data (especially clinical trials), and the correlation and analysis of data from various sources. The following sections present a brief overview of existing works in these various aspects.\n\nData management \nMuch work exists in data management in the healthcare domain. These include the design and analysis of clinical trials[5], big data in healthcare[6], and others.[7] A detailed survey of big data in healthcare by Bahri et al.[8] provides further insights. The crux of all the work in data management is to store data on a large scale and then be able to process it efficiently and quickly. However, this is not the only scope of this paper, since this paper does not only communicate and present information about data management (where we resemble the existing work) but also presents additional key features that distinguish our work from the existing work. Those distinguishing features are presented as contributions in the introduction section.\n\nData interoperability and standardization \nData interoperability and standardization are two other key aspects of healthcare data management and analysis. Unlike other traditional data management systems, healthcare data\u2014in particular clinical data\u2014require more robust, universally known, and recognized interoperable methods and standards because the data are critical to healthcare and the healthcare investigations and diagnoses that come with it. For example, Health Level 7 (HL7) standards focus on standardizing clinical trial terminologies across various stakeholders in the world. Multiple investigation research centers need to exchange results, outcomes, treatments, etc. to come to a common conclusion for certain diseases and treatments. Therefore, a high quality of standards, as explained by Schulz et al.[9] and Hussain et al.[10], exists, and ontological[11] representations have been defined to represent the data in an interoperable manner universally. There are a wide range of resources in this domain, and the readers can see further details in the survey by Brundage et al.[12] Our research scope goes beyond this topic of standardization and interoperability and focuses on a more abstract level where all data from all types of providers in all standards can be brought together for analysis and be able to incorporate changes and evolve as the timelines of investigations evolve.\n\nData analysis \nLike any other field of data analysis, clinical healthcare data has also been widely studied for analysis and preparation. This includes strategies for dealing with missing data[9], correlating data from various sources and generating recommendations for better healthcare[13], and the use of machine learning (ML) approaches for investigating the diagnosis of vaccines, e.g., COVID-19 vaccines.[14] Some examples of works related to this specific field are presented by Brundage et al.[12] and Majumder and Minko.[15] Moreover, we also find efforts that investigate supporting clinical decisions by clinicians in various fields, and these types of systems are generally referred to as clinical decision support systems (CDSSs).[16] These types of systems generally focus on generating recommendations using artificial intelligence (AI) and ML techniques. However, they lack the ability to discover, link, and provide an analytical view of the process of clinical trials[2] under investigation. These trials may still be under investigation and not yet complete. On the other hand, this research work leaves the part of analysis for a specific disease, diagnosis, treatment, etc. to the user of this solution and focuses on presenting a unified system where data from a variety of sources can be obtained at one place and be able to perform any kind of analysis such as ML, data preparation tasks, profiling, and recommendations[13] as listed in the contributions and uniqueness section of this paper. This research does provide a real-time and robust view of the processes to support fast and reliable clinical investigation while the data are not yet complete or an investigation has been completed.\n\nComparison of the proposed FAIR Health Informatics framework to the state of the art \nThe proposed FAIR Health Informatics framework differs from and supersedes the existing state of the art across several perspectives. Firstly, existing approaches either focus on data standardization for interoperability and exchange of clinical data specifically or act on the data in silos. Secondly, most of the analytical frameworks\u2019 usage is only with specialized datasets, such as clinical trials only, EHRs only, or sensory data only. Thirdly, all the above-discussed systems do not capture the building timeline of events (changes) and mostly work with static data by loading periodic batches. The proposed framework in this paper addresses these problems by fusing data from various sources, building profiles for entities, maintaining changes in events over time for the entities of interest, and avoiding silos of analysis and computation. Moreover, the solution is designed to support streaming, batch, and static data.\n\nMethodology \nIn this section, the terms and symbols used throughout this paper will be introduced first as preliminaries, and then the overall architecture of the framework will be presented.\n\nPreliminaries \nThe terms used in this paper are of two types: those that describe entities or subjects for which a dataset is produced by a data provider, and those that describe or represent a process that relates to the steps a dataset undergoes. A process may involve subjects as its input or output, but a vice versa approach is not possible.\n\nTerms \nEntity: An entity, E, is a clinical concept, disease, treatment, or a human being related to data that can be collected, correlated to, and analyzed in combination with other entities. For example, a vaccine for COVID-19 under investigation is considered an entity. Humans under monitoring for a vaccine trial comprise an entity. Clinical concepts, diseases, treatments, devices, etc. are the types of entities that will be referred to as \u201centity being investigated,\u201d whereas human beings on which the entities are observed are referred to as \u201centity being observed.\u201d\nSubject: A subject, S, is an entity (i.e., SE) for which a data item or a measurement is recorded. For example, a person is a human entity, and a vaccine is a clinical concept entity.\nSubject types: A subject type is a subject for which data items can be recorded and can either be an entity being investigated or an entity being observed.\nDomain: A domain, D, is a contextual entity and can be combined with a specific \u201centity being investigated\u201d subject-type. For example, \u201cbreast cancer\u201d is a domain, and \"viral drug\" is a domain. Moreover, in clinical terms, each high-level concept in Medical Subject Headings (MeSH) ontology[17] is a domain. Similarly, \u201celectronic health record\u201d (or \"EHR\") is a domain.\nSub-domain: Just like a subject with sub-types, a domain has sub-domains, e.g., \u201ccancer\u201d is a domain and \u201cbreast cancer\u201d is a sub-domain. Furthermore, sub-domains can have further sub-domains.\nSchema: A schema, Sch, is the template in which measurements or real values of a subject are recorded. A schema specifies the types, names, hierarchy, and arity of values in a measurement or a record. Schema is also referred to as type-level records.\nInstance: An instance, I, is an actual record or a measurement that corresponds to a schema for the subjects of a particular domain. An instance of an EHR record belongs to a subject \u201cperson\u201d of entity \u201chuman,\u201d with entity type as an \u201centity being observed.\u201d Similarly, a clinical trial record for \u201cbreast cancer\u201d investigation with all its essential data (as described in coming sections) is an instance of entity \u201ctrial\u201d in the domain \u201ccancer\u201d with sub-domain \u201cbreast cancer\u201d and is an entity of type \u201centity being investigated.\u201d Moreover, a set {I} of instances is referred to by a dataset, Dat, such that Dat has a schema, Sch.\nStakeholder: A stakeholder is a person, an organization, or any other such entity that needs to either onboard their data in the framework for analysis, use the framework with existing data for insights, or do both.\nScenario: A scenario, Sce, is a representation of a query that defines the parameters, domain, context, and scope of the intended use of the framework. For example, a possible scenario is when a stakeholder wants to visualize the timeline of investigations\/trials for a certain \u201cdomain\u201d (i.e., \u201cbreast cancer\u201d) in the last two years by a particular investigation agency\/organization. The scenario is then the encapsulation of all such parameters and context.\nUse case: A use case, UC, is a practical scenario represented by steps and actions in a flow from the start of raw data until the point of analytical\/processed data intended to show the result of the scenario, Sce.\nTimeline: A timeline is normally the sequence of data-changing events in a particular entity being monitored. For example, in the case of a clinical trial, it is every change to any of its features, such as the number of registered patients, the addition\/removal of investigation sites and the investigators, and\/or the methods of investigation. These types of changing events need to be captured and linked with the timestamp they were captured on. These data (a type of time-series) are crucial for building time-series analysis of clinical entities and investigations. An example of time-series analysis may be determining the evolution of a vaccine over a certain time or between dates, or determining the role of certain investigators with a particular background in a clinical trial over a certain period linked with the trials\u2019 stages or phases.\n\nProcesses \nData on-boarding: This is the process to identify and declare (if needed) the raw data schema Sch for a dataset D, identify and declare domain-specific terms (e.g., an ontology term), and declare limitations, risks, and use cases.\nData management: This is the process to bring the raw data under management that is already \"on-boarded,\" as discussed above. It should declare and process data and type lineage[18] and be able to represent each data item with a timeline, hence treat every dataset as a time-series dataset. There is more on this in the following sections.\nData correlation: The framework is designed to on-board and manage the interrelated data coming from various resources. This process, which runs across all other processes, is meant to declare the correlations at each step to link technical terms to business\/domain terms; for example, for a medical treatment for people with obesity, it should find all datasets that provide insights into these clinical terms. In each of the processes, this process is carried out at various levels with different semantics, as shown in the following sections.\nData analysis: This is the process to perform on-demand data processing based on pre-configured and orchestrated pipelines. When a scenario, Sce, is provided, the pipeline is triggered, and various orchestrated queries are processed by the framework generating derived results that can be streamed\/sent to the user and additionally stored in the framework (particularly in the data lake) for future use.\nReproducibility: This is a process to reproduce[19] the result of a scenario in case of failure, doubt, or correctness checking of the framework system.\n\nArchitecture \nIn this section, we introduce the overall architecture of the framework. Figure 1 presents an abstract overview of the generic architecture with different components that will address the above-described objectives. A detailed flow of the architecture follows in the following paragraphs; however, firstly, a layer-wise functioning is presented in the framework. This architecture consists of various components, such as raw data collection at the bottom that acquires data that can be regularly scraped from data sources at the bottom. Next is the data-cleaning pipeline powered by Apache Spark; it is the component that is based on defined schemas and transforms all datasets to a unified data format (such as Parquet) before storing them in the data lake. Next are the layers of semantic profiling and predictions. These layers are meant to process the data written to the data lake and perform analytical tasks such as fusing data from various sources (e.g., academic clinical articles, clinical trials, etc.) and predicting profiles from clinical trial data. Finally, the application layer at the top represents dashboards and external applications requesting data from the bottom layers through some services.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 1. General architecture depicting different phases of data processing and analytics related to clinical data and publications.\n\n\n\nBefore any dataset is brought into the system, the raw data schema and any vocabularies or ontologies must be declared. This initial step is called data \u201con-boarding,\u201d and it will become more visible in the following section when processes are discussed. After this on-boarding process, the data flows through the layers discussed above as follows. Storage of raw data to the data lake (right top in Figure 1), performing quality checking and mappings declared in the on-boarding process, and storing to the clinical data lake. The clinical data lake is also a file storage system with the ability to store each piece of data in a time-series format. For that purpose, we propose the usage of Apache Hudi[20], which allows each piece of data written to the data lake to be stamped\/committed with a timestamp, thereby providing the ability to travel back in time. This data movement from the raw data to the data lake is optional in the sense that the stakeholders who want to store raw data can store it in the blob storage, whereas for others, the data can be directly brought to the data lake. In either case, the framework can stream the data either from an external source or from the raw data storage. For the streaming of the data, we propose the use of \u201cKafka\u201d along with akka-streaming technologies such as Cloudflow.[21]\nOnce the data are acquired and ingested into the data lake, queries can be performed to do analysis and perform data correlation between various data sources. The analysis and correlation can be carried out on both static and inflight data, i.e., data ingested in a streaming fashion, and hence requires the ability to merge\/fuse with other streams.\nWhen the data are related to a particular \u201centity being investigated,\u201d it requires the correlation of these data to the appropriate academic research articles so that various stakeholders can relate the investigation to the state of the art in academia. The correlation methods and techniques are out of the scope of this paper; however, we do want to emphasize that this correlation and the mapping are critical to this framework. The results of the processing layer (the middle layer in Figure 1) are stored back in the data lake for re-usability and availability. The querying engine that can be employed for this purpose is Apache Spark.[22]\nFinally, the data from the data lake and the results of semantic analysis are meant to be streamed to applications of various types by various stakeholders, thus requiring the contracts to be defined before making such service requests. We emphasize the contracts here since the contracts are key to managing data lineage and explainability, as detailed in the following sections, and hence it is a prerequisite for an explainable system to have control over the data being moved and have a clear understanding of at what time what data in what format is being moved.\nThe framework is designed to be multi-tenant, having the flexibility to provide services and processing capabilities for private and\/or public data management services. Those kinds of differences are made possible through proper access controls.\n\nProcesses and methods \nIn this section, first an overview of clinical data sources will be presented, then some specific processes and methods (used interchangeably) that acquire, manage, fuse, and process these data sources to produce quality analytics will be presented. Next, all the components of the architecture that include data analysis modes servicing data to external analytical applications are presented, and finally, the concepts of lineage and reproducibility are presented since these concepts are at the core of the FAIR principal.\nIn the following subsections, each service is part of a layer in Figure 1. The layers in Figure 1 are abstract and hence overlook these services. In each process explained below, the respective layer of the architecture in Figure 1 is referred to.\n\nData sources \nAs visible from the framework, this framework is designed to on-board data that are related to real entities. These include various sources:\n\nPublic sources: First, there were data from clinical trial investigations conducted by public agencies that were published in their publicly available repositories. In this case, the data can be on-boarded by the published (as the stakeholder) to the framework so that it is not just available to the public (because it already is in the form of a public repository), but it is also available and meaningful because this framework makes the data integrated with other sources and provides an intelligent and smart overview and analysis. The clinical trial data mostly contain a wider range of information related to the sites of investigation, the investigators, their affiliations, treatments, and outcomes, among others. See Ali et al.[23] and Lipscomb[24] for a detailed overview of what a trial can contain. Moreover, publicly standardized clinical ontologies are also one of the key sources of data that would be on-boarded and used in multiple phases; they are used in the on-boarding process to map the raw data to clinical entities and concepts and map the clinical trials, investigators, and sites to clinical concepts.\nPrivate sources: A second source of the data is a private agency that would like to analyze its own clinical investigation in the same domain as public data providers, hence on-board data from its private repository and get the analysis results in the examples of use cases and scenarios as described in sections below. Note that this framework is multi-tenant with proper access control mechanisms; therefore, the on-boarding of private data can be \u201ctimed\u201d (e.g., deleted or archived after a certain time limit).\nPersonal\/Electronic health records: These are data that can be on-boarded from private and public data providers. However, these data might include sensitive and personal information about valid and real human beings, who can be reluctant to share their data for privacy reasons. In that case, proper consent must be obtained before data are on-boarded, and the data must be anonymized using known industry-best anonymization practices.[25]\nAcademic data: \u201cEntities being observed\u201d are not only appearing in the data from clinical trials as discussed above, but also occur as the latest research in these domains is published in the form of academic journals and conferences such as biomedical engineering and other journals. We consider these datasets (journals and conferences) as one of the key datasets that are publicly available and brought to the framework in increments and updates. As explained in the following sections, these datasets are correlated with clinical concepts in ontologies, such as MeSH and others, and with clinical trials. The usefulness of these datasets and their integration is of the utmost importance for clinicians and the pharmaceutical industry to correlate their findings with state-of-the-art academic advances.\nDerived datasets: These are the datasets that result when the data are managed and correlation is performed and\/or when there are updates and it is needed to pre-compute some results so that these results can be provided when requested without computing them on the fly, for obvious reasons such as the complexity of the analysis\/query and the latency it takes to compute those results. Examples of these include correlating academic journals to clinical concepts, building trials and other timelines, generating time-series of certain data, and resolving entities to concepts, among others. In other words, these derived datasets could result in both semantic data (stored in ontologies) and non-semantic data stored according to a particular schema, Sch. Since it is only allowed for a dataset to be persisted in the data lake if and only if it has a schema, therefore each such derived dataset must be declared, annotated, and searchable. These are described in the following section of the \"on-boarding\" datasets, and on-boarding can be for both internal and external datasets.\nData on-boarding and discovery process \nThis subsection details how the on-boarding service is designed to operate. This service is external to the framework as a separate entity since it needs to be independent of whatever process the framework follows. It is critical to the whole framework as this is the first point of entry for stakeholders that will likely be storing the data and requires knowledge about the domain of the data. Figure 2 explains this with an example. The data on-boarding process typically involves four entities\/components. The sequence of steps presented in Figure 2 is complemented by the business process presented in Figure 3. We assume one wants to on-board a dataset, Dat, that contains or will contain the set {I} of the instance of a domain, D, for an \u201centity being investigated\u201d. In that case, to identify the meaning of each field in Dat, e.g., if a field\/variable has a certain value, it is needed\/required to identify the type, domain, and its business or conceptual meaning (as shown in Figure 3). Therefore, in this process, defining both the technical schema, Sch, of the D as well as a conceptual schema are required. The technical schema is used during the data acquisition and processing, whereas the conceptual schema is used to construct the semantic meaning of the assets, such as for discoverability of datasets by other users.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 2. Data on-boarding activity sequence.\n\n\n\n\n\n\n\n\n\n\n\n\nFigure 3. Data asset on-boarding business process.\n\n\n\nFigure 4 presents the sequence of steps that showcase the discovery of assets after the technical and conceptual schemas in Figure 2 and Figure 3 have been defined. In this process, an external user can be interested in using a particular dataset for the training of ML algorithms or other analytical algorithms if the data provider has consented to it.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 4. Data asset discovery process.\n\n\n\nData management process \nThe data management process cannot be instantiated prior to or in parallel with the on-boarding process. This is intended to make sure that each dataset that lands in the management is able to be queried, understood well by the framework, and of use to the stakeholders. This process relates to the two bottom layers in Figure 1. Once a dataset, Dat, is on-boarded (See the Section 3.3.2), the technical schema, Sch, is version controlled (we suggest using Git VCS for this) and stored in the schema repository. Figure 5 presents the flow of events, essentially depicting the pipeline through which the data flow (both static and dynamic). The technical schema, Sch, is first used by validating that the raw dataset (read from Kafka), Dat, conforms to the schema, Sch. Then, the Ingestor component performs necessary transformations such as validation on certain fields, extracting information from file names, etc. Then, the validator validates real data values against the schema. Lastly, the transformer components perform the transformation to match the table in the data lake, wherein all types and formats of data are stored irrespective of their formats. In all these steps, the distributed framework Spark is leveraged, and all the steps can be traced using the lineage tracking application programming interfaces (APIs). \nNote that the transformer supports additional mappings (format changes\/updates, time-zone information, among others) and creates a schema, Sch, for the data under management (i.e., in the data lake). However, it is strictly monitored that the schema, Sch\u2019, does not alter the meaning, names, and bounds set in the initial technical schema, Sch, in the on-boarding process. As visible in the diagram, the data can be streamed directly from an external source through the means of a streaming framework such as Apache Kafka[26], Akka Streams[27], or any other such framework. The goal here is not to impose any technology but rather the capabilities of lineage and tracking. Note that by having a proper schema and the conformance of the data to that schema, one can trace back from the end to the start of the whole process to see where the data comes from and how it comes.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 5. Data collection and transformation sequence.\n\n\n\nData correlation \nData correlation (layer semantic linking in Figure 1) is about mapping data from various sources to one another such that it can easily be queried for understanding and mapping. As an example, when data for entities being observed in some domain (i.e., cancer) are on-boarded, this will include updates related to investigators, sites, treatments, medication, and outcomes. The first step is to map the personal data (if available) from health records to the clinical trial investigation study. Moreover, another aspect is that each update to a dataset or to a clinical trial can contain a reference to a single instance of a person or investigation site, say an investigator, but with changes in the name or the spelling, among others. Similar is the case with sites, diseases, and medical conditions (components of clinical trial data). In all such cases, the mapping of all such individual occurrences from the data to real records in the system is carried out, and the ontology has to be built.\nMoreover, academic data include references to clinical concepts (entities being observed) and sub-types and domains. Whenever an academic dataset is brought to be managed, the data points are correlated with the \u201cdata correlation\u201d component to perform semantic search and produce records. These kinds of semantic results are useful for clinicians and pharmaceutical agencies to relate their outcomes to academic investigations. Therefore, data correlation is a key process that takes place each time there is an update or a new dataset in the system. Note that the clinical terms can also be referenced from clinical trials, and in fact, each clinical trial always relates to at least one clinical term, so the mapping must occur at the time of ingestion of each dataset. For this purpose, semantic mappers can be developed that will use clustering and text-based techniques to map incoming updates to certain entities to existing entities in the system, e.g., mapping updated clinical sites to existing sites, investigators to existing investigators, etc. Note that the clinical data available in public repositories or with private owners do not necessarily contain unique identifiers for these entities, and if they do, it is not synchronized across various repositories and data providers.\n\nData analysis \nThis subsection describes two types of analyses, and those can always be extended to other types. This process is part of the layers of semantic linking and correlation processing in Figure 1.\n\nOffline\/Periodic: As a first type of analysis, the vision is that data analyses are required to be performed on updates to data or periodically. These include analyses that are heavy with high latency or analyses that are potentially going to be requested frequently in scenarios by various stakeholders\/user of the framework. As an example, building the timeline of a clinical trial is performed each time there are updates to the datasets for that trial. Similarly, building timelines for investigators, treatments, and sites is also offline but actively performed when data arrive in the system. These kinds of datasets are referred to as \"projections\" or \"derived datasets,\" as described in the data sources section.\nFeature store: Moreover, another example of such analysis is building a feature store of clinical trial results. For example, when an outcome of a trial is obtained, a feature is made out of it so that it can be used as input for statistical learning in ML for predictions and for further analysis.\nData services \nThe architecture is presented in Figure 1. Data services serve two types of data: (1) data about data (i.e., metadata) and (2) data itself. In the case of metadata, the framework offers the ability to search and discover what types of data exist in the system. For example, a data user may want to know what data providers provide the data, which clinical trial repositories\u2019 data exist in the system, what are the schemas of these data, how to obtain the data, and so on. As mentioned earlier, for this purpose, datahub[28] is one of the best examples. This service is closely coupled with a data on-boarding service; once data are on-boarded, they can be searched as explained above. For this kind of service, we propose using an external system called datahub, which is mature enough to perform this function. The separation is mainly because metadata is a slowly changing dimension of the data. Hence, it does not require streaming capabilities now, and redoing it is not wise either. Secondly, when the data are in the system, the framework offers the ability to query and stream the data to the user\/end system. This service complements the metadata service since this cannot stream data that cannot be discovered by the former. Here, we insist on streaming services since, on the one hand, streams can be generalized to streaming larger chunks to support non-streaming application endpoints and streaming data to support transporting large datasets continuously and in mini chunks. The second service of serving data from the platform is available for each layer in Figure 1 since all the data end up in the data lake.\n\nLineage and reproducibility \nThe ability to reproduce results retrieved from certain data using a set of algorithms and techniques is a key requirement for explainable and trustable systems. To do so, the framework supports techniques and methods to keep track of data lineage at the meta level, and then the framework is designed in such a way that, if needed, one can reproduce the expected result from the same data using the same algorithms. At the meta-data level, for each change (i.e., update) in the dataset that is being ingested into the system, if there is a certain modification (e.g., technical modifications such as a schema change), it is required to create a schema and deal with it internally as a separate dataset, as earlier described as the derived dataset or projected dataset. In this way, one can track the lineage at each point of each computation or calculation, and all these schemas for these datasets are available in the meta-data discovery service. Moreover, ontology is built of related concepts with the system to make sure that if an entity is related to, for example, a trial, then it can be tracked. Secondly, the data can be queried using the scenarios as explained below; these scenarios in fact define the various parameters such as time window and dataset, among others. With these parameters, one can always reproduce an expected result from a certain point in the pipeline since the windows and the scenarios are recorded.\n\nUse cases and scenario \nThe framework supports a variety of use cases (UC), and we will explain some of the preliminary and easily understandable ones in this section. We relate use cases to stakeholders (i.e., users of the framework), and hence the design of the framework is intended to be specific and address the objectives. Moreover, each use case is assumed to include some steps, involving multiple parts of the framework in a certain order. In other words, a use case is related to a business process. Therefore, with each use case, we also show a tentative business process diagram. This can also be interpreted as an orchestration scenario, where the steps in the process are orchestrated. An orchestration scenario is different than the scenario described above in the sense that the orchestration scenario is about the steps of a process, whereas a general scenario is analogous to defining the scope of what data to retrieve as described above.\n\nUse cases \nUC1: \"As a data provider, I want to bring the data into the framework so that they are available to other stakeholders for research and investigation.\" - In this use case, a potential stakeholder can be a government organization that wishes to make a dataset, Dat, that it owns for a domain, D, and\/or continuously performs investigations related to a particular \u201centity being investigated.\u201d Therefore, with the help of the \u201con-boarding\u201d process, the stake-holder along with the technical support should be able to declare related \u201cconcepts\u201d domain terms, define a technical schema, Sch, for Dat, a conceptual schema, Sch\u2019, and provide an API to access the data by the framework or place it in a raw data storage (again making it accessible by the framework).\nUC2: \"As a data provider, I want to bring the data into the framework so that I can use them in combination with other publicly available datasets, but I want my dataset to be private.\" -\nIn this case, the process is similar to above; however, this time, the data are private (i.e., tenant support) and can be queried through the service layer in combination with other publicly available datasets based on scenarios.\n\nUC3: \"As a stakeholder, I want to know the available datasets related to the domain breast cancer[28] and the timelines of trials in the last three years of investigation in this domain for the entities being investigated correlated with 'entities being observed.'\" - This particular use case deals with querying data that is correlated. The process starts by first using the on-boarding service to query existing available data using business\/conceptual terms. This will enable the stakeholder to choose which datasets he\/she is interested in. Then, a scenario is created based on the use case definition and forwarded to the service layer for execution. The service layer, with pre-executed correlation and mapping, answers the query in a streaming fashion.\nScenarios \nS1: \"As a user (i.e., stakeholder), I want to get the set of instances, {I} (i.e., data), for the last two months for the 'entities being observed,' E, in the 'viral disease' domain, D, with a sub-domain 'COVID-19.'\"\nS2: \"As a user, I want a timeline (i.e., an analysis) of the entities, E, being observed in the 'breast cancer' sub-domain for data providers from the Americas (or for a specific clinical trial data provider).\"\nS3: \"I want to get the analysis of successful trials in the last two years in the 'intestinal disease' domain.\"\nS4: \"I want to get all the instances, {I}, for all the entities, {E}, in the 'cancer' domain, D, with trial outcomes[23][24] to train a machine learning algorithm.\"\nS5: \"As a pharmaceutical company analyst (i.e., stakeholder), I want to see the duration of affiliation of all investigators from sites in Europe associated with clinical trials in the domain of 'cancer'.\"\nOnline scenarios represent the analyses that will be executed by the \u201cdata service\u201d and might result in two options: (1) when the analysis (query) completes, stream\/send the result back to the requested; (2) store the result in the data lake and stream\/send the result back. The second case needs to be implemented such that one can cache the analysis result in memory; if the request is repeated after a certain threshold (or another such metric can be defined), one can persist the analysis result to the data lake.\nAll such analytical results at the time of deciding to persist to the data lake will first go through the process of on-boarding. As described in the on-boarding process, each dataset that lands in the data lake must have a schema, Sch. In this case, a schema is registered, semantically annotated, and then the dataset is written to the data lake for reasons described in the framework description and the \"derived\" data sources description.\nFigure 6 shows the sequence of activities that are performed to execute a scenario by interacting with the platform. Here, the internals of the platform are hidden, and only the interactions between components are shown.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 6. Sequence of activities in a scenario.\n\n\n\nSimilarly, Figure 7 shows the process of a scenario based on parameters provided in the scenario. For example, if the scenario asks for the analysis of sites of clinical trials, then the data service invokes those particular components to perform those functions seamlessly, and so on.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 7. Business process of scenario execution..\n\n\n\n Experiments, results, and evaluation \nThis section presents implementation details, followed by results and the evaluation of results and computational advantages.\n\nImplementation \nA prototype version of the proposed framework is implemented with each component, as presented in Figure 1, to reflect the use cases and provide a minimal viable prototype (MVP). More specifically, we took the case of managing \u201cclinical\u201d trials as a case study for this prototype. Clinical trial data are obtained from the aforementioned repositories available publicly, where one can obtain a history of trials as well. The specific clinical trial repositories used are BSMO trials and CityGov trials, and for academic research manuscripts, we used research articles from well-known research journals such as The Journal of Clinical Investigation[29] and Sage Journals' Clinical Trials.[30] We used Apache Hudi as the data storage format (using parquet file format), which is compact, provides time-series data checkpointed by any update\/upserts, and provides efficient query processing using the query engine Spark. Apache Hudi was used since it supports parquet, which is format-agnostic, succinct, and fast.\nFor storage of data, Azure blob storage is used as the clinical data lake, where the raw data are stored, and it has also been used to store the ingested data in the Hudi format as Hudi tables (which are time-seriesed and checkpointed), which is also called the data lake. This storage selection is purely optional, and it can be any file system storage. Scala is used as a programming language, with Apache Kafka as a message-passing and data streaming framework, and Apache Spark as a distributed data processing framework. Moreover, Lightbend Cloudflow is integrated with these technologies and hence used in the replacement of microservices.\nThe process of data acquisition, management, and processing takes place as follows: First, we scrape data from available public repositories provided by providers and log it to two places: Kafka messages and the data lake. Then, we ingest the data from Kafka (each Kafka message conforms to a schema defined in the Avro format specific for that dataset) in a streaming pipeline integrating Kafka and write to the data lake (bottom layer in Figure 1). This step brings any format of data into a single format, and we can use our processing framework Spark with ease to perform any type of analytics (next layer from bottom in Figure 1).\nWe implemented a data analysis service (3rd layer from bottom in Figure 1) to correlate information from profiles, clinical trials, and trial subjects to identify the clinical profiles of people. This included various forms of information extraction and knowledge creation, such as the creation of the ontology to construct the relationship between these various datasets. Similarly, we also implemented a front-end to register assets (datasets and providers) so that we could later query and prove that such a concept does exist. We leveraged the datahub\u2019s data model[28] for that purpose and found that it is one of the best metadata models for registering and managing assets linked with glossary terms and business vocabularies, as well as technical vocabularies, and defining roles and ownerships.\nMoreover, the clinical trial data contain textual data wherein personal and investigation site profiles repeat. This type of data needs correlational and contextual analysis to deduce the real person from it. That is mainly because the person profiles are manually filled in by various people in the trial data or because the people are active in various sites and organizations, and hence a single profile can have a lot of variations. For example, a person\u2019s name \u201cKasper Sukre\u201d can be written as \u201cK. Sukre,\u201d \u201cKasper S,\u201d \u201cK. S,\u201d etc. We therefore needed to resolve all those names to a single person entity. This entity is essentially a person entity in the relational schema of the clinical trial data that we take as a case study.\n\nResults and evaluation \nWe implemented textual person profile prediction using clustering algorithms (2nd layer from the top in Figure 1). We compare the results of two clustering algorithms with two different types of distance algorithms. In the first distance algorithm, since a person profile contains various features such as name, address, cell, email, association, and similarly, address and association can have further sub-fields; therefore, we compute the edit distance between each sub-field separately, which is a nested distance. Finally, we take an average of all distances. In the second distance algorithm, we apply a generic edit distance to the whole person profile as a string and then apply the clustering algorithms. Table 1 shows the results of the two algorithms for the two variants separately.\n\n\n\n\n\n\n\nTable 1. The results of the two clustering algorithms for two different variants of distance algorithms.\n\n\nClustering algorithm\n\nGeneral edit distance\n\nFeature-wise edit distance\n\n\nK-Means\n\n79.43%\n\n85.4%\n\n\nEM\n\n83.54%\n\n88.9%\n\n\n\nAs can be seen in Table 1, the algorithm EM consistently performs better than K-Means on both variants of the distance algorithm. Since the algorithms are not 100% accurate, and it makes sense, we therefore employ a quality check on the results to verify the final person recommendation that is also required since we cannot completely rely on algorithms for personal data.\nNext, we must also indicate that we were able to stream data from the scraper that scrapes the data from various repositories simultaneously in parallel and were able to write to the same table (the Apache Hudi table) time-series data at around five times faster than normal upserts to a relational database. Since the framework is designed to poll data sources regularly and as soon as data are available, it is seamlessly brought into the system using the automated pipeline, and the algorithms to link and personalize are performed on the fly on the incoming data. Note here that, when any new data arrives in the system, the algorithms implemented above (analytical and predictive) are also part of the pipeline orchestration. Therefore, this avoids any cron jobs, periodic, and\/or manual jobs to perform these tasks, and if necessary, data can already be streamed or made available to applications such as Kafka messages. This kind of automation, fusion, and parallelism makes this framework unique in that it speeds up the manual process used by the existing systems that follow the process of scraping data, ingesting it periodically, and then running cron jobs to perform analytics. Apart from the automation of the process, existing systems, as explained in the state-of-the-art, also lack data fusion of trial data, academic articles, Medical Subject Heading ontologies, and linking profiles.\n\nDiscussion \nIn this paper, we present a healthcare framework that enables us to handle health-related data. When clinical trials are under investigation and need an advanced and automated system to speed up the trial process and monitor the treatments, investigations at sites, their outcomes, and other related aspects of trial management, we presented a framework that streams data from various sources, fuses the data semantically, applies algorithms to resolve duplicates, and handles the history of events. This does not only speed up the trial analysis process compared to the existing methods; it also enriches the trial analysis and gives a complete context with the latest research from academia, semantic ontologies, and a holistic view of events. The framework is designed to support the declaration of schemas (type-level information) and metadata related to various data that comes into the framework. This enables exploring any analysis being made over the data and making it accessible to third parties as well as government and other agencies to use data from data providers, express interests, and\/or bring their own data to the system to be used by other such users of the system.\nWe have presented scenarios in which this framework can be utilized. Not limited to these scenarios, the framework can be extended to support a variety of specialized data systems; for example, they include complex event processing, weather monitoring, and traffic and air management, among others. This is all because we have leveraged the data lake technology and provided abstractions that can be used for any kind of querying of the data. Moreover, the framework is designed to support data lineage, which helps in achieving data accuracy and provenance. This is mainly obtained by keeping track of each transformation and operation performed within the framework, and thus it can be reproduced either at the operation level, data level, or type level. All these help in evidence-based decision-making and thus prove that the framework conforms to and fulfills the critical needs of a system needed for healthcare-related sensitive data.\nFurthermore, the framework is designed to be extensible, and it envisions the inclusion of automated feature engineering required to generate recommendations from historical data based on statistical models and methods. This can further be used to plug and play models in the framework, which leads to using data from one model provided by a model provider or a third party using the model and data from separate users. Hence, this framework in this paper is limited to clinical data processing, lineage tracking, and governance and applies and extends to other areas of research as well.\n\nConclusions \nIn this research article, we presented a detailed overview of the need for a framework for clinical investigation to speed up the process of clinical trials for medical equipment, vaccines, and other such products, especially in the event of pandemics such as COVID-19.[4][31] This is necessary since a delay in such matters causes the death of humans, and saving a single life is analogous to saving humanity. Moreover, the presented framework has the potential to provide evidence-based data analysis through lineage tracking; data governance capability to explore, visualize semantics and links, and possibly make decisions among various data sets available in the framework for use; and process data at state and in motion. We have presented use cases that showcase the usage of the framework in different scenarios and how it can be used by both private, public, and private-public organizations. The framework is extendable, encapsulates the abilities to include other domains, provides support for semantic querying at the governance level (i.e., at the data assets and type assets level), and is designed towards a data and computation economic model. Finally, it supports evidence-based decision tracking, and hence nothing is lost or unknown in the process of evaluation and analysis, which follows the FAIR principle. Future work includes a detailed investigation of the specifics of each part of the framework in the domains of semantics, learning models, and the incremental evaluation of queries that need to be evaluated on updates in real-time data processing.\n\n Abbreviations, acronyms, and initialisms \nAI: artificial intelligence\nAPI: application programming interface\nCDC: change data capture\nCDSS: clinical decision support system\nEHR: electronic health record\nFAIR: findable, accessible, interoperable, and reusable\nHL7: Health Level 7\nMeSH: Medical Subject Headings\nML: machine learning\nMVP: minimal viable prototype\nUC: use case\nAcknowledgements \nAuthor contributions \nConceptualization, methodology, and validation, M.H.S. and M.I.; formal analysis, M.H.S.; resources, M.A.; data curation, M.A. and M.I.; writing\u2014review and editing, M.H.S. and M.I.; funding acquisition, M.A. All authors have read and agreed to the published version of the manuscript.\n\nFunding \nThis work was funded by the Deanship of Scientific Research at Jouf University under Grant number DSR-2022-RG-0101.\n\nConflicts of interest \nThe authors declare no conflict of interest.\n\nReferences \n\n\n\u2191 Friedman, Lawrence M.; Furberg, Curt D.; DeMets, David L.; Reboussin, David M.; Granger, Christopher B. (2015) (in en). Fundamentals of Clinical Trials. Cham: Springer International Publishing. doi:10.1007\/978-3-319-18539-2. ISBN 978-3-319-18538-5. https:\/\/link.springer.com\/10.1007\/978-3-319-18539-2 .   \n \n\n\u2191 2.0 2.1 \"ClinicalTrials.gov\". National Library of Medicine. https:\/\/www.clinicaltrials.gov\/ . Retrieved 29 August 2022 .   \n \n\n\u2191 \"Clinical Trials\". Belgian Society for Medical Oncology. https:\/\/www.bsmo.be\/clinical\/clinical-trials\/ . Retrieved 29 August 2022 .   \n \n\n\u2191 4.0 4.1 Plug, R.; Liang, Y.; Basajja, M. et al. (2022). \"FAIR and GDPR Compliant Population Health Data Generation, Processing and Analytics\" (PDF). Proceedings of the 13th International Conference on Semantic Web Applications and Tools for Health Care and Life Sciences: 1\u201310. https:\/\/ceur-ws.org\/Vol-3127\/paper-7.pdf .   \n \n\n\u2191 Meinert, Curtis L. (2012). Clinical trials: design, conduct, and analysis. Monographs in epidemiology and biostatistics. 39 (2nd ed ed.). New York: Oxford University Press. ISBN 978-0-19-538788-9. OCLC 756044508. https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/756044508 .   \n \n\n\u2191 Dash, Sabyasachi; Shakyawar, Sushil Kumar; Sharma, Mohit; Kaushik, Sandeep (1 December 2019). \"Big data in healthcare: management, analysis and future prospects\" (in en). Journal of Big Data 6 (1): 54. doi:10.1186\/s40537-019-0217-0. ISSN 2196-1115. https:\/\/journalofbigdata.springeropen.com\/articles\/10.1186\/s40537-019-0217-0 .   \n \n\n\u2191 Dziura, James D.; Post, Lori A.; Zhao, Qing; Fu, Zhixuan; Peduzzi, Peter (1 September 2013). \"Strategies for dealing with missing data in clinical trials: from design to analysis\". The Yale Journal of Biology and Medicine 86 (3): 343\u2013358. ISSN 1551-4056. PMC 3767219. PMID 24058309. https:\/\/pubmed.ncbi.nlm.nih.gov\/24058309 .   \n \n\n\u2191 Bahri, Safa; Zoghlami, Nesrine; Abed, Mourad; Tavares, Joao Manuel R. S. (2019). \"BIG DATA for Healthcare: A Survey\". IEEE Access 7: 7397\u20137408. doi:10.1109\/ACCESS.2018.2889180. ISSN 2169-3536. https:\/\/ieeexplore.ieee.org\/document\/8585021\/ .   \n \n\n\u2191 9.0 9.1 Schulz, Stefan; Stegwee, Robert; Chronaki, Catherine (2019), Kubben, Pieter; Dumontier, Michel; Dekker, Andre, eds., \"Standards in Healthcare Data\" (in en), Fundamentals of Clinical Data Science (Cham: Springer International Publishing): 19\u201336, doi:10.1007\/978-3-319-99713-1_3, ISBN 978-3-319-99712-4, http:\/\/link.springer.com\/10.1007\/978-3-319-99713-1_3 . Retrieved 2023-07-20   \n \n\n\u2191 Hussain, Maqbool; Afzal, Muhammad; Ali, Taqdir; Ali, Rahman; Khan, Wajahat Ali; Jamshed, Arif; Lee, Sungyoung; Kang, Byeong Ho et al. (1 November 2018). \"Data-driven knowledge acquisition, validation, and transformation into HL7 Arden Syntax\" (in en). Artificial Intelligence in Medicine 92: 51\u201370. doi:10.1016\/j.artmed.2015.09.008. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0933365715001256 .   \n \n\n\u2191 Ethier, J.-F.; Curcin, V.; Barton, A.; McGilchrist, M. M.; Bastiaens, H.; Andreasson, A.; Rossiter, J.; Zhao, L. et al. (2015). \"Clinical data integration model. Core interoperability ontology for research using primary care data\". Methods of Information in Medicine 54 (1): 16\u201323. doi:10.3414\/ME13-02-0024. ISSN 2511-705X. PMID 24954896. https:\/\/pubmed.ncbi.nlm.nih.gov\/24954896 .   \n \n\n\u2191 12.0 12.1 Brundage, Michael; Blazeby, Jane; Revicki, Dennis; Bass, Brenda; de Vet, Henrica; Duffy, Helen; Efficace, Fabio; King, Madeleine et al. (1 August 2013). \"Patient-reported outcomes in randomized clinical trials: development of ISOQOL reporting standards\" (in en). Quality of Life Research 22 (6): 1161\u20131175. doi:10.1007\/s11136-012-0252-1. ISSN 0962-9343. PMC PMC3731511. PMID 22987144. http:\/\/link.springer.com\/10.1007\/s11136-012-0252-1 .   \n \n\n\u2191 13.0 13.1 Banos, Oresti; Bilal Amin, Muhammad; Ali Khan, Wajahat; Afzal, Muhammad; Hussain, Maqbool; Kang, Byeong Ho; Lee, Sungyong (1 July 2016). \"The Mining Minds digital health and wellness framework\" (in en). BioMedical Engineering OnLine 15 (S1): 76. doi:10.1186\/s12938-016-0179-9. ISSN 1475-925X. PMC PMC4959395. PMID 27454608. http:\/\/biomedical-engineering-online.biomedcentral.com\/articles\/10.1186\/s12938-016-0179-9 .   \n \n\n\u2191 Alballa, Norah; Al-Turaiki, Isra (2021). \"Machine learning approaches in COVID-19 diagnosis, mortality, and severity risk prediction: A review\" (in en). Informatics in Medicine Unlocked 24: 100564. doi:10.1016\/j.imu.2021.100564. PMC PMC8018906. PMID 33842685. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S235291482100054X .   \n \n\n\u2191 Majumder, Joydeb; Minko, Tamara (1 January 2021). \"Recent Developments on Therapeutic and Diagnostic Approaches for COVID-19\" (in en). The AAPS Journal 23 (1): 14. doi:10.1208\/s12248-020-00532-2. ISSN 1550-7416. PMC PMC7784226. PMID 33400058. http:\/\/link.springer.com\/10.1208\/s12248-020-00532-2 .   \n \n\n\u2191 Hussain, M.; Khattak, A. M.; Khan, W. A.; Fatima, I.; Amin, M. B.; Pervez, Z.; Batool, R.; Saleem, M. A. et al. (1 June 2013). \"Cloud-based Smart CDSS for chronic diseases\" (in en). Health and Technology 3 (2): 153\u2013175. doi:10.1007\/s12553-013-0051-x. ISSN 2190-7188. http:\/\/link.springer.com\/10.1007\/s12553-013-0051-x .   \n \n\n\u2191 Zu, D.; Li, D.; Carterette, B. et al. (2013). \"An Incremental Approach for MEDLINE MeSH Indexing\" (PDF). Proceedings of the first Workshop on Bio-Medical Semantic Indexing and Question Answering, a Post-Conference Workshop of Conference and Labs of the Evaluation Forum 2013 (CLEF 2013) 1094. https:\/\/ceur-ws.org\/Vol-1094\/bioasq2013_submission_7.pdf .   \n \n\n\u2191 Ikeda, R.; Widom, J. (2009). \"Data Lineage: A Survey\". Stanford University. https:\/\/citeseerx.ist.psu.edu\/document?repid=rep1&type=pdf&doi=af8988cb698ce8641cb5fa239c149295716ebc76 .   \n \n\n\u2191 Mobley, Aaron; Linder, Suzanne K.; Braeuer, Russell; Ellis, Lee M.; Zwelling, Leonard (15 May 2013). Arakawa, Hirofumi. ed. \"A Survey on Data Reproducibility in Cancer Research Provides Insights into Our Limited Ability to Translate Findings from the Laboratory to the Clinic\" (in en). PLoS ONE 8 (5): e63221. doi:10.1371\/journal.pone.0063221. ISSN 1932-6203. PMC PMC3655010. PMID 23691000. https:\/\/dx.plos.org\/10.1371\/journal.pone.0063221 .   \n \n\n\u2191 \"Apache Hudi\". Apache Software Foundation. 2021. https:\/\/hudi.apache.org\/ . Retrieved 20 August 2022 .   \n \n\n\u2191 de Lucia, A.; Xhelo, E. (2020). \"6. Data Science Pipeline Containerization\". In Smedinga, R.; Biehl, M.. 17th SC@RUG proceedings 2019-2020. Bibliotheek der R.U.. pp. 39\u201344. ISBN 978-94-034-2766-9. https:\/\/pure.rug.nl\/ws\/portalfiles\/portal\/124739867\/proceedings_2020.pdf .   \n \n\n\u2191 Zaharia, Matei; Xin, Reynold S.; Wendell, Patrick; Das, Tathagata; Armbrust, Michael; Dave, Ankur; Meng, Xiangrui; Rosen, Josh et al. (28 October 2016). \"Apache Spark: a unified engine for big data processing\" (in en). Communications of the ACM 59 (11): 56\u201365. doi:10.1145\/2934664. ISSN 0001-0782. https:\/\/dl.acm.org\/doi\/10.1145\/2934664 .   \n \n\n\u2191 23.0 23.1 Ali, Rahman; Afzal, Muhammad; Hussain, Maqbool; Ali, Maqbool; Siddiqi, Muhammad Hameed; Lee, Sungyoung; Ho Kang, Byeong (1 February 2016). \"Multimodal hybrid reasoning methodology for personalized wellbeing services\" (in en). Computers in Biology and Medicine 69: 10\u201328. doi:10.1016\/j.compbiomed.2015.11.013. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0010482515003819 .   \n \n\n\u2191 24.0 24.1 Lipscomb, C. E. (1 July 2000). \"Medical Subject Headings (MeSH)\". Bulletin of the Medical Library Association 88 (3): 265\u2013266. ISSN 0025-7338. PMC PMC35238. PMID 10928714. https:\/\/pubmed.ncbi.nlm.nih.gov\/10928714 .   \n \n\n\u2191 Murthy, Suntherasvaran; Abu Bakar, Asmidar; Abdul Rahim, Fiza; Ramli, Ramona (1 May 2019). \"A Comparative Study of Data Anonymization Techniques\". 2019 IEEE 5th Intl Conference on Big Data Security on Cloud (BigDataSecurity), IEEE Intl Conference on High Performance and Smart Computing, (HPSC) and IEEE Intl Conference on Intelligent Data and Security (IDS) (Washington, DC, USA: IEEE): 306\u2013309. doi:10.1109\/BigDataSecurity-HPSC-IDS.2019.00063. ISBN 978-1-7281-0006-7. https:\/\/ieeexplore.ieee.org\/document\/8819477\/ .   \n \n\n\u2191 Kreps, J.; Narkhede, N.; Rao, J. (2011). \"Kafka: A Distributed Messaging System for Log Processing\". Proceedings of the NetDB. 11. pp. 1\u20137. ISBN 978-1-4503-0652-2\/11\/06. https:\/\/cs.uwaterloo.ca\/~ssalihog\/courses\/papers\/netdb11-final12.pdf .   \n \n\n\u2191 Davis, Adam L. (2019), \"Akka Streams\" (in en), Reactive Streams in Java (Berkeley, CA: Apress): 57\u201370, doi:10.1007\/978-1-4842-4176-9_6, ISBN 978-1-4842-4175-2, http:\/\/link.springer.com\/10.1007\/978-1-4842-4176-9_6   \n \n\n\u2191 28.0 28.1 28.2 Mamounas, Eleftherios P. (1 October 2003). \"NSABP Breast Cancer Clinical Trials: Recent Results and Future Directions\" (in en). Clinical Medicine & Research 1 (4): 309\u2013326. doi:10.3121\/cmr.1.4.309. ISSN 1539-4182. PMC PMC1069061. PMID 15931325. http:\/\/www.clinmedres.org\/content\/1\/4\/309 .   \n \n\n\u2191 \"The Journal of Clinical Investigation\". American Society for Clinical Investigation. ISSN 1558-8238. https:\/\/www.jci.org\/ . Retrieved 28 May 2023 .   \n \n\n\u2191 \"Clinical Trials\". The Society for Clinical Trials. ISSN 1740-7753. https:\/\/journals.sagepub.com\/home\/ctj . Retrieved 28 May 2023 .   \n \n\n\u2191 Platto, Sara; Xue, Tongtong; Carafoli, Ernesto (24 September 2020). \"COVID19: an announced pandemic\" (in en). Cell Death & Disease 11 (9): 799. doi:10.1038\/s41419-020-02995-9. ISSN 2041-4889. PMC PMC7513903. PMID 32973152. https:\/\/www.nature.com\/articles\/s41419-020-02995-9 .   \n \n\n\nNotes \nThis presentation is faithful to the original, with only a few minor changes to presentation, though grammar and word usage was substantially updated for improved readability. In some cases important information was missing from the references, and that information was added.\n\n\n\n\n\nSource: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.limswiki.org\/index.php\/Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\">https:\/\/www.limswiki.org\/index.php\/Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis<\/a>\nCategories: LIMSwiki journal articles (added in 2023)LIMSwiki journal articles (all)LIMSwiki journal articles on clinical researchLIMSwiki journal articles on data analysisLIMSwiki journal articles on health informaticsNavigation menuPage actionsJournalDiscussionView sourceHistoryPage actionsJournalDiscussionMoreToolsIn other languagesPersonal toolsLog inNavigationMain pageEncyclopedic articlesRecent changesRandom pageHelp about MediaWikiSearch\u00a0 ToolsWhat links hereRelated changesSpecial pagesPermanent linkPage informationPopular publications\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\nPrint\/exportCreate a bookDownload as PDFDownload as PDFDownload as Plain textPrintable version This page was last edited on 20 July 2023, at 22:27.Content is available under a Creative Commons Attribution-ShareAlike 4.0 International License unless otherwise noted.This page has been accessed 588 times.Privacy policyAbout LIMSWikiDisclaimers\n\n\n\n","affac9ff82db9d386600be5eb3d77056_html":"<body class=\"mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-206 ns-subject page-Journal_FAIR_Health_Informatics_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis rootpage-Journal_FAIR_Health_Informatics_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis skin-monobook action-view skin--responsive\"><div id=\"rdp-ebb-globalWrapper\"><div id=\"rdp-ebb-column-content\"><div id=\"rdp-ebb-content\" class=\"mw-body\" role=\"main\"><a id=\"rdp-ebb-top\"><\/a>\n<h1 id=\"rdp-ebb-firstHeading\" class=\"firstHeading\" lang=\"en\">Journal:FAIR Health Informatics: A health informatics framework for verifiable and explainable data analysis<\/h1><div id=\"rdp-ebb-bodyContent\" class=\"mw-body-content\"><!-- start content --><div id=\"rdp-ebb-mw-content-text\" lang=\"en\" dir=\"ltr\" class=\"mw-content-ltr\"><div class=\"mw-parser-output\">\n\n\n<h2><span class=\"mw-headline\" id=\"Abstract\">Abstract<\/span><\/h2>\n<p>The recent <a href=\"https:\/\/www.limswiki.org\/index.php\/COVID-19\" class=\"mw-redirect wiki-link\" title=\"COVID-19\" data-key=\"da9bd20c492b2a17074ad66c2fe25652\">COVID-19<\/a> <a href=\"https:\/\/www.limswiki.org\/index.php\/Pandemic\" title=\"Pandemic\" class=\"wiki-link\" data-key=\"bd9a48e6c6e41b6d603ee703836b01f1\">pandemic<\/a> has hit humanity very hard in ways rarely observed before. In this digitally connected world, the <a href=\"https:\/\/www.limswiki.org\/index.php\/Health_informatics\" title=\"Health informatics\" class=\"wiki-link\" data-key=\"055eb51f53cfdbacc08ed150b266c9f4\">health informatics<\/a> and <a href=\"https:\/\/www.limswiki.org\/index.php\/Medical_research\" title=\"Medical research\" class=\"wiki-link\" data-key=\"0ee7e4e2a32a422d78fe6bd1ab0d1cbc\">clinical research<\/a> domains (both public and private) lack a robust framework to enable rapid investigation and cures. Since data in the healthcare domain are highly confidential, any framework in the healthcare domain must work on real data, be verifiable, and support reproducibility for evidence purposes. In this paper, we propose a health informatics framework that supports data acquisition from various sources in real-time, correlates these data from various sources among each other and to the domain-specific terminologies, and supports querying and <a href=\"https:\/\/www.limswiki.org\/index.php\/Data_analysis\" title=\"Data analysis\" class=\"wiki-link\" data-key=\"545c95e40ca67c9e63cd0a16042a5bd1\">analyses<\/a>. Various sources include sensory data from wearable sensors, clinical investigation (for trials and <a href=\"https:\/\/www.limswiki.org\/index.php\/Medical_device\" title=\"Medical device\" class=\"wiki-link\" data-key=\"8e821122daa731f0fa8782fae57831fa\">devices<\/a>) data from private\/public agencies, personal health records, academic publications in the healthcare domain, and semantic information such as clinical <a href=\"https:\/\/www.limswiki.org\/index.php\/Ontology_(information_science)\" title=\"Ontology (information science)\" class=\"wiki-link\" data-key=\"52d0664bde4b458e81fbc128b911a4a6\">ontologies<\/a> and the <a href=\"https:\/\/www.limswiki.org\/index.php\/Medical_Subject_Headings_(MeSH)\" title=\"Medical Subject Headings (MeSH)\" class=\"wiki-link\" data-key=\"5a78d7a7cdb189a093388e284b249efc\">Medical Subject Headings<\/a> (MeSH) ontology. The linking and correlation of various sources include mapping personal wearable data to <a href=\"https:\/\/www.limswiki.org\/index.php\/Electronic_health_record\" title=\"Electronic health record\" class=\"wiki-link\" data-key=\"f2e31a73217185bb01389404c1fd5255\">health records<\/a>, clinical oncology terms to clinical trials, and so on. The framework is designed such that the data are <a href=\"https:\/\/www.limswiki.org\/index.php\/Journal:The_FAIR_Guiding_Principles_for_scientific_data_management_and_stewardship\" title=\"Journal:The FAIR Guiding Principles for scientific data management and stewardship\" class=\"wiki-link\" data-key=\"e5903ddcc7734415af1d91fcd258da90\">findable, accessible, interoperable, and reusable<\/a> (FAIR) with proper <a href=\"https:\/\/www.limswiki.org\/index.php\/Identity_management\" title=\"Identity management\" class=\"wiki-link\" data-key=\"48c80fc5994d9aaac22e580e91fe75d3\">identity and access management<\/a> mechanisms. This practically means tracing and linking each step in the <a href=\"https:\/\/www.limswiki.org\/index.php\/Information_management\" title=\"Information management\" class=\"wiki-link\" data-key=\"f8672d270c0750a858ed940158ca0a73\">data management<\/a> lifecycle through discovery, ease of access and exchange, and data reuse. We present a practical use case to correlate a variety of aspects of data relating to a certain medical subject heading from the MeSH ontology and academic publications with clinical investigation data. The proposed architecture supports streaming data acquisition, and servicing and processing changes throughout the lifecycle of the data management process. This is necessary in certain events, such as when the status of a certain clinical or other health-related investigation needs to be updated. In such cases, it is required to track and view the outline of those events for the analysis and traceability of the clinical investigation and to define interventions if necessary.\n<\/p><p><b>Keywords<\/b>: data correlation, data linking, verifiable data, data analysis, explainable decisions, clinical trials, COVID, clinical investigation, semantic mapping, smart health\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Introduction\">Introduction<\/span><\/h2>\n<p><a href=\"https:\/\/www.limswiki.org\/index.php\/Pandemic\" title=\"Pandemic\" class=\"wiki-link\" data-key=\"bd9a48e6c6e41b6d603ee703836b01f1\">Pandemics<\/a> are not new to this world or humanity. There have been pandemics in the past, and they may happen again in the future. The recent <a href=\"https:\/\/www.limswiki.org\/index.php\/COVID-19\" class=\"mw-redirect wiki-link\" title=\"COVID-19\" data-key=\"da9bd20c492b2a17074ad66c2fe25652\">COVID-19<\/a> pandemic is different from the previous ones in that the virus is more infectious without being known, symptoms are ambiguous, and the detection methods require a lot of time and resources. It has caused more deaths than ever in the history of humankind, and the impact it has had on the world economy and human lives (whether affected or not) is grave and is posing questions about the future of diseases and pandemics. At the same time, while humans advance knowledge and technology, there is a need to investigate and put effort into overcoming the challenges posed by these kinds of serious threats. This not only requires us to deal with the current pandemic but also to look into the future, predict and presume the possibilities, investigate, and develop solutions at a large scale so that there is a reduction in the risk of losing lives and danger on a large scale.\n<\/p><p>The majority of the existing software systems and solutions in the healthcare domain are proprietary and limited in their capacity to a specific domain, such as only processing clinical trial data without integrating state-of-the-art investigation and wearable sensor data. Hence, they lack the ability to present a scalable analytical and technical solution, while having limitations and lacking the ability to trace back the analysis and results to the origin of the data. Because of these limitations, any robust and practical solution does not only have to account for clinical data but also present a practical and broader overview of these catastrophic events from clinical investigations, their results and treatments that are up-to-date, and combine them with medical history records as well as academic and other related datasets available. In other words, the recent advancements and investigations in the clinical domain specific to a particular disease are published in research articles and journals, and they also need to be correlated to real human subjects that undergo clinical trials so that up-to-date analysis can be carried out and proper guidelines and interventions can be suggested.\n<\/p><p>Moreover, the majority of the record-keeping bodies maintain <a href=\"https:\/\/www.limswiki.org\/index.php\/Electronic_health_record\" title=\"Electronic health record\" class=\"wiki-link\" data-key=\"f2e31a73217185bb01389404c1fd5255\">electronic health records<\/a> (EHRs) for patients and, recently, records of COVID vaccinations. However, they lack the ability to link the data to individuals\u2019 activities and clinical outcomes. There is also a lack of fusing data related to the investigation of a particular disease from various data providers, such as private clinical trials, public clinical trials<sup id=\"rdp-ebb-cite_ref-1\" class=\"reference\"><a href=\"#cite_note-1\">[1]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:0_2-0\" class=\"reference\"><a href=\"#cite_note-:0-2\">[2]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-3\" class=\"reference\"><a href=\"#cite_note-3\">[3]<\/a><\/sup>, and public-private clinical trials. The lack of these services is not only because there is less literature on fusing these multiple forms of data but also because of <a href=\"https:\/\/www.limswiki.org\/index.php\/Information_security\" title=\"Information security\" class=\"wiki-link\" data-key=\"9eff362d944224ff1d4ffe3a149d7cff\">security<\/a> and <a href=\"https:\/\/www.limswiki.org\/index.php\/Information_privacy\" title=\"Information privacy\" class=\"wiki-link\" data-key=\"185f6d9f874e48914b5789317408f782\">privacy<\/a> concerns related to the confidentiality of healthcare data. The current advanced and robust privacy and security infrastructure available is more than enough to ensure personal and organizational interests. On the one hand, there are governments and other organizations that publish their <a href=\"https:\/\/www.limswiki.org\/index.php\/Medical_research\" title=\"Medical research\" class=\"wiki-link\" data-key=\"0ee7e4e2a32a422d78fe6bd1ab0d1cbc\">clinical research<\/a> data to public repositories to be available for clinical research using defined standards. On the other hand, there are pharma companies, which mostly hold the analytics driven by these and the respective algorithms and methods private. Furthermore, clinical trial data are not enough since they only provide measurements for different subjects who underwent a trial, while other data, from sources such as EHRs, contain real investigative cases and the histories of patients. Therefore, fusing these data from a variety of sources is helpful to determine the effects of a particular drug or treatment plan in combination with other vaccines, treatments, etc.\n<\/p><p>Given the above brief overview of the capabilities of the state-of-the-art, most of these systems either tackle static or dynamic, relational or non-relational, noisy or cleaned data without fusing, <a href=\"https:\/\/www.limswiki.org\/index.php\/Data_integration\" title=\"Data integration\" class=\"wiki-link\" data-key=\"fd01c635859e1d5b9583e43e31ef6718\">integrating<\/a>, or semantically linking it. This paper proposes a solution to the above problems by presenting a healthcare framework that supports the ability to acquire, manage, and process static and dynamic (real-time) data. Our proposed framework is a data format that focuses on fusing information from various sources. In a nutshell, the proposed framework ideally targets a strategy that makes data <a href=\"https:\/\/www.limswiki.org\/index.php\/Journal:The_FAIR_Guiding_Principles_for_scientific_data_management_and_stewardship\" title=\"Journal:The FAIR Guiding Principles for scientific data management and stewardship\" class=\"wiki-link\" data-key=\"e5903ddcc7734415af1d91fcd258da90\">findable, accessible, interoperable, and reusable<\/a> (FAIR).<sup id=\"rdp-ebb-cite_ref-:1_4-0\" class=\"reference\"><a href=\"#cite_note-:1-4\">[4]<\/a><\/sup>\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Objectives_and_contributions\">Objectives and contributions<\/span><\/h3>\n<p>In particular, we define the objectives and contributions of this research work as proposing a framework that is designed to be able to provide the following basic and essential capabilities for healthcare data:\n<\/p>\n<ul><li>A clinical \"<a href=\"https:\/\/www.limswiki.org\/index.php\/Data_lake\" title=\"Data lake\" class=\"wiki-link\" data-key=\"cbe28db47d4d3ce56b947c2959cc9eea\">data lake<\/a>\" that stores data in a unified format where the raw data can be in any format loaded from a raw storage or streamed.<\/li>\n<li>Pipelines that support static and incremental data collection from raw storage to a clinical data lake and maintain a record of any change to the structure at the data level and at the schema level all the way to the raw data and raw data schema.<\/li>\n<li>A schema repository that <a href=\"https:\/\/www.limswiki.org\/index.php\/Version_control\" title=\"Version control\" class=\"wiki-link\" data-key=\"81823f6b21d385f8db9ac0a17b571cc1\">versions<\/a> the data when it changes its structure and enables forward and backward compatibility of the data throughout.<\/li>\n<li>Universal clinical schemas that can incorporate any clinically related concepts (such as clinical trials from any provider) and support flexibility.<\/li>\n<li>Processes to perform change data capture (CDC) as the data proceed down the pipeline towards the applications, i.e., incremental algorithms and methods to enable incremental processing of incoming data and keep a log of only the data of interest.<\/li>\n<li>Timelines for changing data for a specific clinical investigation from a clinical trial data element such as a trial, site, investigator, etc., and include the ability to stream the data to the application, while providing semantic linking and profiling of subjects in the data.<\/li><\/ul>\n<p>These capabilities are meant to provide evidence of <a href=\"https:\/\/www.limswiki.org\/index.php\/Data_analysis\" title=\"Data analysis\" class=\"wiki-link\" data-key=\"545c95e40ca67c9e63cd0a16042a5bd1\">data analysis<\/a> (i.e., where does the analysis go back in terms of data), traces of changes, and a holistic view of various clinical investigations running simultaneously at different places related to a certain specific clinical investigation. An example of that is when the COVID-19 vaccine was being developed; it was necessary to be able to trace the investigation of various efforts by independent bodies in a single place where one could see the phases of clinical trials of vaccines, their outcomes, treatments, and even the investigation sites and investigators. This could not only result in better decision-making but also in putting resources in suitable places and better planning for future similar scenarios.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"State_of_the_art\">State of the art<\/span><\/h2>\n<p>Many works exist in the literature that deal with various aspects of clinical data, ranging from <a href=\"https:\/\/www.limswiki.org\/index.php\/Information_management\" title=\"Information management\" class=\"wiki-link\" data-key=\"f8672d270c0750a858ed940158ca0a73\">data management<\/a> and analysis to interoperability, the meaning of big data in healthcare and its future course, the standardization of clinical data (especially clinical trials), and the correlation and analysis of data from various sources. The following sections present a brief overview of existing works in these various aspects.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Data_management\">Data management<\/span><\/h3>\n<p>Much work exists in data management in the healthcare domain. These include the design and analysis of clinical trials<sup id=\"rdp-ebb-cite_ref-5\" class=\"reference\"><a href=\"#cite_note-5\">[5]<\/a><\/sup>, big data in healthcare<sup id=\"rdp-ebb-cite_ref-6\" class=\"reference\"><a href=\"#cite_note-6\">[6]<\/a><\/sup>, and others.<sup id=\"rdp-ebb-cite_ref-7\" class=\"reference\"><a href=\"#cite_note-7\">[7]<\/a><\/sup> A detailed survey of big data in healthcare by Bahri <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-8\" class=\"reference\"><a href=\"#cite_note-8\">[8]<\/a><\/sup> provides further insights. The crux of all the work in data management is to store data on a large scale and then be able to process it efficiently and quickly. However, this is not the only scope of this paper, since this paper does not only communicate and present information about data management (where we resemble the existing work) but also presents additional key features that distinguish our work from the existing work. Those distinguishing features are presented as contributions in the introduction section.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Data_interoperability_and_standardization\">Data interoperability and standardization<\/span><\/h3>\n<p>Data interoperability and standardization are two other key aspects of healthcare data management and analysis. Unlike other traditional data management systems, healthcare data\u2014in particular clinical data\u2014require more robust, universally known, and recognized interoperable methods and standards because the data are critical to healthcare and the healthcare investigations and diagnoses that come with it. For example, <a href=\"https:\/\/www.limswiki.org\/index.php\/Health_Level_7\" title=\"Health Level 7\" class=\"wiki-link\" data-key=\"e0bf845fb58d2bae05a846b47629e86f\">Health Level 7<\/a> (HL7) standards focus on standardizing clinical trial terminologies across various stakeholders in the world. Multiple investigation research centers need to exchange results, outcomes, treatments, etc. to come to a common conclusion for certain diseases and treatments. Therefore, a high quality of standards, as explained by Schulz <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-:2_9-0\" class=\"reference\"><a href=\"#cite_note-:2-9\">[9]<\/a><\/sup> and Hussain <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-10\" class=\"reference\"><a href=\"#cite_note-10\">[10]<\/a><\/sup>, exists, and <a href=\"https:\/\/www.limswiki.org\/index.php\/Ontology_(information_science)\" title=\"Ontology (information science)\" class=\"wiki-link\" data-key=\"52d0664bde4b458e81fbc128b911a4a6\">ontological<\/a><sup id=\"rdp-ebb-cite_ref-11\" class=\"reference\"><a href=\"#cite_note-11\">[11]<\/a><\/sup> representations have been defined to represent the data in an interoperable manner universally. There are a wide range of resources in this domain, and the readers can see further details in the survey by Brundage <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-:3_12-0\" class=\"reference\"><a href=\"#cite_note-:3-12\">[12]<\/a><\/sup> Our research scope goes beyond this topic of standardization and interoperability and focuses on a more abstract level where all data from all types of providers in all standards can be brought together for analysis and be able to incorporate changes and evolve as the timelines of investigations evolve.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Data_analysis\">Data analysis<\/span><\/h3>\n<p>Like any other field of data analysis, clinical healthcare data has also been widely studied for analysis and preparation. This includes strategies for dealing with missing data<sup id=\"rdp-ebb-cite_ref-:2_9-1\" class=\"reference\"><a href=\"#cite_note-:2-9\">[9]<\/a><\/sup>, correlating data from various sources and generating recommendations for better healthcare<sup id=\"rdp-ebb-cite_ref-:4_13-0\" class=\"reference\"><a href=\"#cite_note-:4-13\">[13]<\/a><\/sup>, and the use of <a href=\"https:\/\/www.limswiki.org\/index.php\/Machine_learning\" title=\"Machine learning\" class=\"wiki-link\" data-key=\"79aab39cfa124c958cd1dbcab3dde122\">machine learning<\/a> (ML) approaches for investigating the diagnosis of vaccines, e.g., COVID-19 vaccines.<sup id=\"rdp-ebb-cite_ref-14\" class=\"reference\"><a href=\"#cite_note-14\">[14]<\/a><\/sup> Some examples of works related to this specific field are presented by Brundage <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-:3_12-1\" class=\"reference\"><a href=\"#cite_note-:3-12\">[12]<\/a><\/sup> and Majumder and Minko.<sup id=\"rdp-ebb-cite_ref-15\" class=\"reference\"><a href=\"#cite_note-15\">[15]<\/a><\/sup> Moreover, we also find efforts that investigate supporting clinical decisions by clinicians in various fields, and these types of systems are generally referred to as <a href=\"https:\/\/www.limswiki.org\/index.php\/Clinical_decision_support_system\" title=\"Clinical decision support system\" class=\"wiki-link\" data-key=\"095141425468d057aa977016869ca37d\">clinical decision support systems<\/a> (CDSSs).<sup id=\"rdp-ebb-cite_ref-16\" class=\"reference\"><a href=\"#cite_note-16\">[16]<\/a><\/sup> These types of systems generally focus on generating recommendations using <a href=\"https:\/\/www.limswiki.org\/index.php\/Artificial_intelligence\" title=\"Artificial intelligence\" class=\"wiki-link\" data-key=\"0c45a597361ca47e1cd8112af676276e\">artificial intelligence<\/a> (AI) and ML techniques. However, they lack the ability to discover, link, and provide an analytical view of the process of clinical trials<sup id=\"rdp-ebb-cite_ref-:0_2-1\" class=\"reference\"><a href=\"#cite_note-:0-2\">[2]<\/a><\/sup> under investigation. These trials may still be under investigation and not yet complete. On the other hand, this research work leaves the part of analysis for a specific disease, diagnosis, treatment, etc. to the user of this solution and focuses on presenting a unified system where data from a variety of sources can be obtained at one place and be able to perform any kind of analysis such as ML, data preparation tasks, profiling, and recommendations<sup id=\"rdp-ebb-cite_ref-:4_13-1\" class=\"reference\"><a href=\"#cite_note-:4-13\">[13]<\/a><\/sup> as listed in the contributions and uniqueness section of this paper. This research does provide a real-time and robust view of the processes to support fast and reliable clinical investigation while the data are not yet complete or an investigation has been completed.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Comparison_of_the_proposed_FAIR_Health_Informatics_framework_to_the_state_of_the_art\">Comparison of the proposed FAIR Health Informatics framework to the state of the art<\/span><\/h3>\n<p>The proposed FAIR Health Informatics framework differs from and supersedes the existing state of the art across several perspectives. Firstly, existing approaches either focus on data standardization for interoperability and exchange of clinical data specifically or act on the data in silos. Secondly, most of the analytical frameworks\u2019 usage is only with specialized datasets, such as clinical trials only, EHRs only, or sensory data only. Thirdly, all the above-discussed systems do not capture the building timeline of events (changes) and mostly work with static data by loading periodic batches. The proposed framework in this paper addresses these problems by fusing data from various sources, building profiles for entities, maintaining changes in events over time for the entities of interest, and avoiding silos of analysis and computation. Moreover, the solution is designed to support streaming, batch, and static data.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Methodology\">Methodology<\/span><\/h2>\n<p>In this section, the terms and symbols used throughout this paper will be introduced first as preliminaries, and then the overall architecture of the framework will be presented.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Preliminaries\">Preliminaries<\/span><\/h3>\n<p>The terms used in this paper are of two types: those that describe entities or subjects for which a dataset is produced by a data provider, and those that describe or represent a process that relates to the steps a dataset undergoes. A process may involve subjects as its input or output, but a vice versa approach is not possible.\n<\/p>\n<h4><span class=\"mw-headline\" id=\"Terms\">Terms<\/span><\/h4>\n<p><i>Entity<\/i>: An entity, E, is a clinical concept, disease, treatment, or a human being related to data that can be collected, correlated to, and analyzed in combination with other entities. For example, a vaccine for COVID-19 under investigation is considered an entity. Humans under monitoring for a vaccine trial comprise an entity. Clinical concepts, diseases, treatments, devices, etc. are the types of entities that will be referred to as \u201centity being investigated,\u201d whereas human beings on which the entities are observed are referred to as \u201centity being observed.\u201d\n<\/p><p><i>Subject<\/i>: A subject, S, is an entity (i.e., SE) for which a data item or a measurement is recorded. For example, a person is a human entity, and a vaccine is a clinical concept entity.\n<\/p><p><i>Subject types<\/i>: A subject type is a subject for which data items can be recorded and can either be an entity being investigated or an entity being observed.\n<\/p><p><i>Domain<\/i>: A domain, D, is a contextual entity and can be combined with a specific \u201centity being investigated\u201d subject-type. For example, \u201cbreast cancer\u201d is a domain, and \"viral drug\" is a domain. Moreover, in clinical terms, each high-level concept in <a href=\"https:\/\/www.limswiki.org\/index.php\/Medical_Subject_Headings_(MeSH)\" title=\"Medical Subject Headings (MeSH)\" class=\"wiki-link\" data-key=\"5a78d7a7cdb189a093388e284b249efc\">Medical Subject Headings<\/a> (MeSH) ontology<sup id=\"rdp-ebb-cite_ref-17\" class=\"reference\"><a href=\"#cite_note-17\">[17]<\/a><\/sup> is a domain. Similarly, \u201celectronic health record\u201d (or \"EHR\") is a domain.\n<\/p><p><i>Sub-domain<\/i>: Just like a subject with sub-types, a domain has sub-domains, e.g., \u201ccancer\u201d is a domain and \u201cbreast cancer\u201d is a sub-domain. Furthermore, sub-domains can have further sub-domains.\n<\/p><p><i>Schema<\/i>: A schema, Sch, is the template in which measurements or real values of a subject are recorded. A schema specifies the types, names, hierarchy, and arity of values in a measurement or a record. Schema is also referred to as type-level records.\n<\/p><p><i>Instance<\/i>: An instance, I, is an actual record or a measurement that corresponds to a schema for the subjects of a particular domain. An instance of an EHR record belongs to a subject \u201cperson\u201d of entity \u201chuman,\u201d with entity type as an \u201centity being observed.\u201d Similarly, a clinical trial record for \u201cbreast cancer\u201d investigation with all its essential data (as described in coming sections) is an instance of entity \u201ctrial\u201d in the domain \u201ccancer\u201d with sub-domain \u201cbreast cancer\u201d and is an entity of type \u201centity being investigated.\u201d Moreover, a set {I} of instances is referred to by a dataset, Dat, such that Dat has a schema, Sch.\n<\/p><p><i>Stakeholder<\/i>: A stakeholder is a person, an organization, or any other such entity that needs to either onboard their data in the framework for analysis, use the framework with existing data for insights, or do both.\n<\/p><p><i>Scenario<\/i>: A scenario, Sce, is a representation of a query that defines the parameters, domain, context, and scope of the intended use of the framework. For example, a possible scenario is when a stakeholder wants to visualize the timeline of investigations\/trials for a certain \u201cdomain\u201d (i.e., \u201cbreast cancer\u201d) in the last two years by a particular investigation agency\/organization. The scenario is then the encapsulation of all such parameters and context.\n<\/p><p><i>Use case<\/i>: A use case, UC, is a practical scenario represented by steps and actions in a flow from the start of raw data until the point of analytical\/processed data intended to show the result of the scenario, Sce.\n<\/p><p><i>Timeline<\/i>: A timeline is normally the sequence of data-changing events in a particular entity being monitored. For example, in the case of a clinical trial, it is every change to any of its features, such as the number of registered patients, the addition\/removal of investigation sites and the investigators, and\/or the methods of investigation. These types of changing events need to be captured and linked with the timestamp they were captured on. These data (a type of time-series) are crucial for building time-series analysis of clinical entities and investigations. An example of time-series analysis may be determining the evolution of a vaccine over a certain time or between dates, or determining the role of certain investigators with a particular background in a clinical trial over a certain period linked with the trials\u2019 stages or phases.\n<\/p>\n<h4><span class=\"mw-headline\" id=\"Processes\">Processes<\/span><\/h4>\n<p><i>Data on-boarding<\/i>: This is the process to identify and declare (if needed) the raw data schema Sch for a dataset D, identify and declare domain-specific terms (e.g., an ontology term), and declare limitations, risks, and use cases.\n<\/p><p><i>Data management<\/i>: This is the process to bring the raw data under management that is already \"on-boarded,\" as discussed above. It should declare and process data and type lineage<sup id=\"rdp-ebb-cite_ref-18\" class=\"reference\"><a href=\"#cite_note-18\">[18]<\/a><\/sup> and be able to represent each data item with a timeline, hence treat every dataset as a time-series dataset. There is more on this in the following sections.\n<\/p><p><i>Data correlation<\/i>: The framework is designed to on-board and manage the interrelated data coming from various resources. This process, which runs across all other processes, is meant to declare the correlations at each step to link technical terms to business\/domain terms; for example, for a medical treatment for people with obesity, it should find all datasets that provide insights into these clinical terms. In each of the processes, this process is carried out at various levels with different semantics, as shown in the following sections.\n<\/p><p><i>Data analysis<\/i>: This is the process to perform on-demand data processing based on pre-configured and orchestrated pipelines. When a scenario, Sce, is provided, the pipeline is triggered, and various orchestrated queries are processed by the framework generating derived results that can be streamed\/sent to the user and additionally stored in the framework (particularly in the data lake) for future use.\n<\/p><p><i>Reproducibility<\/i>: This is a process to reproduce<sup id=\"rdp-ebb-cite_ref-19\" class=\"reference\"><a href=\"#cite_note-19\">[19]<\/a><\/sup> the result of a scenario in case of failure, doubt, or correctness checking of the framework system.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Architecture\">Architecture<\/span><\/h3>\n<p>In this section, we introduce the overall architecture of the framework. Figure 1 presents an abstract overview of the generic architecture with different components that will address the above-described objectives. A detailed flow of the architecture follows in the following paragraphs; however, firstly, a layer-wise functioning is presented in the framework. This architecture consists of various components, such as raw data collection at the bottom that acquires data that can be regularly scraped from data sources at the bottom. Next is the data-cleaning pipeline powered by Apache Spark; it is the component that is based on defined schemas and transforms all datasets to a unified data format (such as Parquet) before storing them in the data lake. Next are the layers of semantic profiling and predictions. These layers are meant to process the data written to the data lake and perform analytical tasks such as fusing data from various sources (e.g., academic clinical articles, clinical trials, etc.) and predicting profiles from clinical trial data. Finally, the application layer at the top represents dashboards and external applications requesting data from the bottom layers through some services.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig1_Siddiqi_Healthcare23_11-12.png\" class=\"image wiki-link\" data-key=\"28e17bdca81bc1c7ba3e0c7449d36702\"><img alt=\"Fig1 Siddiqi Healthcare23 11-12.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/2\/27\/Fig1_Siddiqi_Healthcare23_11-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 1.<\/b> General architecture depicting different phases of data processing and analytics related to clinical data and publications.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>Before any dataset is brought into the system, the raw data schema and any vocabularies or ontologies must be declared. This initial step is called data \u201con-boarding,\u201d and it will become more visible in the following section when processes are discussed. After this on-boarding process, the data flows through the layers discussed above as follows. Storage of raw data to the data lake (right top in Figure 1), performing quality checking and mappings declared in the on-boarding process, and storing to the clinical data lake. The clinical data lake is also a file storage system with the ability to store each piece of data in a time-series format. For that purpose, we propose the usage of Apache Hudi<sup id=\"rdp-ebb-cite_ref-20\" class=\"reference\"><a href=\"#cite_note-20\">[20]<\/a><\/sup>, which allows each piece of data written to the data lake to be stamped\/committed with a timestamp, thereby providing the ability to travel back in time. This data movement from the raw data to the data lake is optional in the sense that the stakeholders who want to store raw data can store it in the blob storage, whereas for others, the data can be directly brought to the data lake. In either case, the framework can stream the data either from an external source or from the raw data storage. For the streaming of the data, we propose the use of \u201cKafka\u201d along with akka-streaming technologies such as Cloudflow.<sup id=\"rdp-ebb-cite_ref-21\" class=\"reference\"><a href=\"#cite_note-21\">[21]<\/a><\/sup>\n<\/p><p>Once the data are acquired and ingested into the data lake, queries can be performed to do analysis and perform data correlation between various data sources. The analysis and correlation can be carried out on both static and inflight data, i.e., data ingested in a streaming fashion, and hence requires the ability to merge\/fuse with other streams.\n<\/p><p>When the data are related to a particular \u201centity being investigated,\u201d it requires the correlation of these data to the appropriate academic research articles so that various stakeholders can relate the investigation to the state of the art in academia. The correlation methods and techniques are out of the scope of this paper; however, we do want to emphasize that this correlation and the mapping are critical to this framework. The results of the processing layer (the middle layer in Figure 1) are stored back in the data lake for re-usability and availability. The querying engine that can be employed for this purpose is Apache Spark.<sup id=\"rdp-ebb-cite_ref-22\" class=\"reference\"><a href=\"#cite_note-22\">[22]<\/a><\/sup>\n<\/p><p>Finally, the data from the data lake and the results of semantic analysis are meant to be streamed to applications of various types by various stakeholders, thus requiring the contracts to be defined before making such service requests. We emphasize the contracts here since the contracts are key to managing data lineage and explainability, as detailed in the following sections, and hence it is a prerequisite for an explainable system to have control over the data being moved and have a clear understanding of at what time what data in what format is being moved.\n<\/p><p>The framework is designed to be multi-tenant, having the flexibility to provide services and processing capabilities for private and\/or public data management services. Those kinds of differences are made possible through proper access controls.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Processes_and_methods\">Processes and methods<\/span><\/h3>\n<p>In this section, first an overview of clinical data sources will be presented, then some specific processes and methods (used interchangeably) that acquire, manage, fuse, and process these data sources to produce quality analytics will be presented. Next, all the components of the architecture that include data analysis modes servicing data to external analytical applications are presented, and finally, the concepts of lineage and reproducibility are presented since these concepts are at the core of the FAIR principal.\n<\/p><p>In the following subsections, each service is part of a layer in Figure 1. The layers in Figure 1 are abstract and hence overlook these services. In each process explained below, the respective layer of the architecture in Figure 1 is referred to.\n<\/p>\n<h4><span class=\"mw-headline\" id=\"Data_sources\">Data sources<\/span><\/h4>\n<p>As visible from the framework, this framework is designed to on-board data that are related to real entities. These include various sources:\n<\/p>\n<ul><li><i>Public sources<\/i>: First, there were data from clinical trial investigations conducted by public agencies that were published in their publicly available repositories. In this case, the data can be on-boarded by the published (as the stakeholder) to the framework so that it is not just available to the public (because it already is in the form of a public repository), but it is also available and meaningful because this framework makes the data integrated with other sources and provides an intelligent and smart overview and analysis. The clinical trial data mostly contain a wider range of information related to the sites of investigation, the investigators, their affiliations, treatments, and outcomes, among others. See Ali <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-:5_23-0\" class=\"reference\"><a href=\"#cite_note-:5-23\">[23]<\/a><\/sup> and Lipscomb<sup id=\"rdp-ebb-cite_ref-:6_24-0\" class=\"reference\"><a href=\"#cite_note-:6-24\">[24]<\/a><\/sup> for a detailed overview of what a trial can contain. Moreover, publicly standardized clinical ontologies are also one of the key sources of data that would be on-boarded and used in multiple phases; they are used in the on-boarding process to map the raw data to clinical entities and concepts and map the clinical trials, investigators, and sites to clinical concepts.<\/li><\/ul>\n<ul><li><i>Private sources<\/i>: A second source of the data is a private agency that would like to analyze its own clinical investigation in the same domain as public data providers, hence on-board data from its private repository and get the analysis results in the examples of use cases and scenarios as described in sections below. Note that this framework is multi-tenant with proper access control mechanisms; therefore, the on-boarding of private data can be \u201ctimed\u201d (e.g., deleted or archived after a certain time limit).<\/li><\/ul>\n<ul><li><i>Personal\/Electronic health records<\/i>: These are data that can be on-boarded from private and public data providers. However, these data might include sensitive and personal information about valid and real human beings, who can be reluctant to share their data for privacy reasons. In that case, proper consent must be obtained before data are on-boarded, and the data must be anonymized using known industry-best anonymization practices.<sup id=\"rdp-ebb-cite_ref-25\" class=\"reference\"><a href=\"#cite_note-25\">[25]<\/a><\/sup><\/li><\/ul>\n<ul><li><i>Academic data<\/i>: \u201cEntities being observed\u201d are not only appearing in the data from clinical trials as discussed above, but also occur as the latest research in these domains is published in the form of academic journals and conferences such as biomedical engineering and other journals. We consider these datasets (journals and conferences) as one of the key datasets that are publicly available and brought to the framework in increments and updates. As explained in the following sections, these datasets are correlated with clinical concepts in ontologies, such as MeSH and others, and with clinical trials. The usefulness of these datasets and their integration is of the utmost importance for clinicians and the pharmaceutical industry to correlate their findings with state-of-the-art academic advances.<\/li><\/ul>\n<ul><li><i>Derived datasets<\/i>: These are the datasets that result when the data are managed and correlation is performed and\/or when there are updates and it is needed to pre-compute some results so that these results can be provided when requested without computing them on the fly, for obvious reasons such as the complexity of the analysis\/query and the latency it takes to compute those results. Examples of these include correlating academic journals to clinical concepts, building trials and other timelines, generating time-series of certain data, and resolving entities to concepts, among others. In other words, these derived datasets could result in both semantic data (stored in ontologies) and non-semantic data stored according to a particular schema, Sch. Since it is only allowed for a dataset to be persisted in the data lake if and only if it has a schema, therefore each such derived dataset must be declared, annotated, and searchable. These are described in the following section of the \"on-boarding\" datasets, and on-boarding can be for both internal and external datasets.<\/li><\/ul>\n<h4><span class=\"mw-headline\" id=\"Data_on-boarding_and_discovery_process\">Data on-boarding and discovery process<\/span><\/h4>\n<p>This subsection details how the on-boarding service is designed to operate. This service is external to the framework as a separate entity since it needs to be independent of whatever process the framework follows. It is critical to the whole framework as this is the first point of entry for stakeholders that will likely be storing the data and requires knowledge about the domain of the data. Figure 2 explains this with an example. The data on-boarding process typically involves four entities\/components. The sequence of steps presented in Figure 2 is complemented by the business process presented in Figure 3. We assume one wants to on-board a dataset, Dat, that contains or will contain the set {I} of the instance of a domain, D, for an \u201centity being investigated\u201d. In that case, to identify the meaning of each field in Dat, e.g., if a field\/variable has a certain value, it is needed\/required to identify the type, domain, and its business or conceptual meaning (as shown in Figure 3). Therefore, in this process, defining both the technical schema, Sch, of the D as well as a conceptual schema are required. The technical schema is used during the data acquisition and processing, whereas the conceptual schema is used to construct the semantic meaning of the assets, such as for discoverability of datasets by other users.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig2_Siddiqi_Healthcare23_11-12.png\" class=\"image wiki-link\" data-key=\"b4667dd1ee28a0c53e03c9a5a15017ab\"><img alt=\"Fig2 Siddiqi Healthcare23 11-12.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/2\/2b\/Fig2_Siddiqi_Healthcare23_11-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 2.<\/b> Data on-boarding activity sequence.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p><a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig3_Siddiqi_Healthcare23_11-12.png\" class=\"image wiki-link\" data-key=\"4d09a5f685ef4e5653c254773ecdac6f\"><img alt=\"Fig3 Siddiqi Healthcare23 11-12.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/9\/91\/Fig3_Siddiqi_Healthcare23_11-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 3.<\/b> Data asset on-boarding business process.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>Figure 4 presents the sequence of steps that showcase the discovery of assets after the technical and conceptual schemas in Figure 2 and Figure 3 have been defined. In this process, an external user can be interested in using a particular dataset for the training of ML algorithms or other analytical algorithms if the data provider has consented to it.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig4_Siddiqi_Healthcare23_11-12.png\" class=\"image wiki-link\" data-key=\"203dc5efbdc1e3b7573c8c610506f9b0\"><img alt=\"Fig4 Siddiqi Healthcare23 11-12.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/4\/47\/Fig4_Siddiqi_Healthcare23_11-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 4.<\/b> Data asset discovery process.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h4><span class=\"mw-headline\" id=\"Data_management_process\">Data management process<\/span><\/h4>\n<p>The data management process cannot be instantiated prior to or in parallel with the on-boarding process. This is intended to make sure that each dataset that lands in the management is able to be queried, understood well by the framework, and of use to the stakeholders. This process relates to the two bottom layers in Figure 1. Once a dataset, Dat, is on-boarded (See the Section 3.3.2), the technical schema, Sch, is version controlled (we suggest using Git VCS for this) and stored in the schema repository. Figure 5 presents the flow of events, essentially depicting the pipeline through which the data flow (both static and dynamic). The technical schema, Sch, is first used by validating that the raw dataset (read from Kafka), Dat, conforms to the schema, Sch. Then, the Ingestor component performs necessary transformations such as validation on certain fields, extracting information from file names, etc. Then, the validator validates real data values against the schema. Lastly, the transformer components perform the transformation to match the table in the data lake, wherein all types and formats of data are stored irrespective of their formats. In all these steps, the distributed framework Spark is leveraged, and all the steps can be traced using the lineage tracking <a href=\"https:\/\/www.limswiki.org\/index.php\/Application_programming_interface\" title=\"Application programming interface\" class=\"wiki-link\" data-key=\"36fc319869eba4613cb0854b421b0934\">application programming interfaces<\/a> (APIs). \n<\/p><p>Note that the transformer supports additional mappings (format changes\/updates, time-zone information, among others) and creates a schema, Sch, for the data under management (i.e., in the data lake). However, it is strictly monitored that the schema, Sch\u2019, does not alter the meaning, names, and bounds set in the initial technical schema, Sch, in the on-boarding process. As visible in the diagram, the data can be streamed directly from an external source through the means of a streaming framework such as Apache Kafka<sup id=\"rdp-ebb-cite_ref-26\" class=\"reference\"><a href=\"#cite_note-26\">[26]<\/a><\/sup>, Akka Streams<sup id=\"rdp-ebb-cite_ref-27\" class=\"reference\"><a href=\"#cite_note-27\">[27]<\/a><\/sup>, or any other such framework. The goal here is not to impose any technology but rather the capabilities of lineage and tracking. Note that by having a proper schema and the conformance of the data to that schema, one can trace back from the end to the start of the whole process to see where the data comes from and how it comes.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig5_Siddiqi_Healthcare23_11-12.png\" class=\"image wiki-link\" data-key=\"56a59766a7f0bde2ddc0227503850969\"><img alt=\"Fig5 Siddiqi Healthcare23 11-12.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/3\/3d\/Fig5_Siddiqi_Healthcare23_11-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 5.<\/b> Data collection and transformation sequence.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h4><span class=\"mw-headline\" id=\"Data_correlation\">Data correlation<\/span><\/h4>\n<p>Data correlation (layer semantic linking in Figure 1) is about mapping data from various sources to one another such that it can easily be queried for understanding and mapping. As an example, when data for entities being observed in some domain (i.e., cancer) are on-boarded, this will include updates related to investigators, sites, treatments, medication, and outcomes. The first step is to map the personal data (if available) from health records to the clinical trial investigation study. Moreover, another aspect is that each update to a dataset or to a clinical trial can contain a reference to a single instance of a person or investigation site, say an investigator, but with changes in the name or the spelling, among others. Similar is the case with sites, diseases, and medical conditions (components of clinical trial data). In all such cases, the mapping of all such individual occurrences from the data to real records in the system is carried out, and the ontology has to be built.\n<\/p><p>Moreover, academic data include references to clinical concepts (entities being observed) and sub-types and domains. Whenever an academic dataset is brought to be managed, the data points are correlated with the \u201cdata correlation\u201d component to perform semantic search and produce records. These kinds of semantic results are useful for clinicians and pharmaceutical agencies to relate their outcomes to academic investigations. Therefore, data correlation is a key process that takes place each time there is an update or a new dataset in the system. Note that the clinical terms can also be referenced from clinical trials, and in fact, each clinical trial always relates to at least one clinical term, so the mapping must occur at the time of ingestion of each dataset. For this purpose, semantic mappers can be developed that will use clustering and text-based techniques to map incoming updates to certain entities to existing entities in the system, e.g., mapping updated clinical sites to existing sites, investigators to existing investigators, etc. Note that the clinical data available in public repositories or with private owners do not necessarily contain unique identifiers for these entities, and if they do, it is not synchronized across various repositories and data providers.\n<\/p>\n<h4><span class=\"mw-headline\" id=\"Data_analysis_2\">Data analysis<\/span><\/h4>\n<p>This subsection describes two types of analyses, and those can always be extended to other types. This process is part of the layers of semantic linking and correlation processing in Figure 1.\n<\/p>\n<ul><li><i>Offline\/Periodic<\/i>: As a first type of analysis, the vision is that data analyses are required to be performed on updates to data or periodically. These include analyses that are heavy with high latency or analyses that are potentially going to be requested frequently in scenarios by various stakeholders\/user of the framework. As an example, building the timeline of a clinical trial is performed each time there are updates to the datasets for that trial. Similarly, building timelines for investigators, treatments, and sites is also offline but actively performed when data arrive in the system. These kinds of datasets are referred to as \"projections\" or \"derived datasets,\" as described in the data sources section.<\/li><\/ul>\n<ul><li><i>Feature store<\/i>: Moreover, another example of such analysis is building a feature store of clinical trial results. For example, when an outcome of a trial is obtained, a feature is made out of it so that it can be used as input for statistical learning in ML for predictions and for further analysis.<\/li><\/ul>\n<h4><span class=\"mw-headline\" id=\"Data_services\">Data services<\/span><\/h4>\n<p>The architecture is presented in Figure 1. Data services serve two types of data: (1) data about data (i.e., <a href=\"https:\/\/www.limswiki.org\/index.php\/Metadata\" title=\"Metadata\" class=\"wiki-link\" data-key=\"f872d4d6272811392bafe802f3edf2d8\">metadata<\/a>) and (2) data itself. In the case of metadata, the framework offers the ability to search and discover what types of data exist in the system. For example, a data user may want to know what data providers provide the data, which clinical trial repositories\u2019 data exist in the system, what are the schemas of these data, how to obtain the data, and so on. As mentioned earlier, for this purpose, datahub<sup id=\"rdp-ebb-cite_ref-:7_28-0\" class=\"reference\"><a href=\"#cite_note-:7-28\">[28]<\/a><\/sup> is one of the best examples. This service is closely coupled with a data on-boarding service; once data are on-boarded, they can be searched as explained above. For this kind of service, we propose using an external system called datahub, which is mature enough to perform this function. The separation is mainly because metadata is a slowly changing dimension of the data. Hence, it does not require streaming capabilities now, and redoing it is not wise either. Secondly, when the data are in the system, the framework offers the ability to query and stream the data to the user\/end system. This service complements the metadata service since this cannot stream data that cannot be discovered by the former. Here, we insist on streaming services since, on the one hand, streams can be generalized to streaming larger chunks to support non-streaming application endpoints and streaming data to support transporting large datasets continuously and in mini chunks. The second service of serving data from the platform is available for each layer in Figure 1 since all the data end up in the data lake.\n<\/p>\n<h4><span class=\"mw-headline\" id=\"Lineage_and_reproducibility\">Lineage and reproducibility<\/span><\/h4>\n<p>The ability to reproduce results retrieved from certain data using a set of algorithms and techniques is a key requirement for explainable and trustable systems. To do so, the framework supports techniques and methods to keep track of data lineage at the meta level, and then the framework is designed in such a way that, if needed, one can reproduce the expected result from the same data using the same algorithms. At the meta-data level, for each change (i.e., update) in the dataset that is being ingested into the system, if there is a certain modification (e.g., technical modifications such as a schema change), it is required to create a schema and deal with it internally as a separate dataset, as earlier described as the derived dataset or projected dataset. In this way, one can track the lineage at each point of each computation or calculation, and all these schemas for these datasets are available in the meta-data discovery service. Moreover, ontology is built of related concepts with the system to make sure that if an entity is related to, for example, a trial, then it can be tracked. Secondly, the data can be queried using the scenarios as explained below; these scenarios in fact define the various parameters such as time window and dataset, among others. With these parameters, one can always reproduce an expected result from a certain point in the pipeline since the windows and the scenarios are recorded.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Use_cases_and_scenario\">Use cases and scenario<\/span><\/h3>\n<p>The framework supports a variety of use cases (UC), and we will explain some of the preliminary and easily understandable ones in this section. We relate use cases to stakeholders (i.e., users of the framework), and hence the design of the framework is intended to be specific and address the objectives. Moreover, each use case is assumed to include some steps, involving multiple parts of the framework in a certain order. In other words, a use case is related to a business process. Therefore, with each use case, we also show a tentative business process diagram. This can also be interpreted as an orchestration scenario, where the steps in the process are orchestrated. An orchestration scenario is different than the scenario described above in the sense that the orchestration scenario is about the steps of a process, whereas a general scenario is analogous to defining the scope of what data to retrieve as described above.\n<\/p>\n<h4><span class=\"mw-headline\" id=\"Use_cases\">Use cases<\/span><\/h4>\n<ul><li><i>UC<sub>1<\/sub><\/i>: \"As a data provider, I want to bring the data into the framework so that they are available to other stakeholders for research and investigation.\" - In this use case, a potential stakeholder can be a government organization that wishes to make a dataset, Dat, that it owns for a domain, D, and\/or continuously performs investigations related to a particular \u201centity being investigated.\u201d Therefore, with the help of the \u201con-boarding\u201d process, the stake-holder along with the technical support should be able to declare related \u201cconcepts\u201d domain terms, define a technical schema, Sch, for Dat, a conceptual schema, Sch\u2019, and provide an API to access the data by the framework or place it in a raw data storage (again making it accessible by the framework).<\/li><\/ul>\n<ul><li><i>UC<sub>2<\/sub><\/i>: \"As a data provider, I want to bring the data into the framework so that I can use them in combination with other publicly available datasets, but I want my dataset to be private.\" -<\/li><\/ul>\n<p>In this case, the process is similar to above; however, this time, the data are private (i.e., tenant support) and can be queried through the service layer in combination with other publicly available datasets based on scenarios.\n<\/p>\n<ul><li><i>UC<sub>3<\/sub><\/i>: \"As a stakeholder, I want to know the available datasets related to the domain breast cancer<sup id=\"rdp-ebb-cite_ref-:7_28-1\" class=\"reference\"><a href=\"#cite_note-:7-28\">[28]<\/a><\/sup> and the timelines of trials in the last three years of investigation in this domain for the entities being investigated correlated with 'entities being observed.'\" - This particular use case deals with querying data that is correlated. The process starts by first using the on-boarding service to query existing available data using business\/conceptual terms. This will enable the stakeholder to choose which datasets he\/she is interested in. Then, a scenario is created based on the use case definition and forwarded to the service layer for execution. The service layer, with pre-executed correlation and mapping, answers the query in a streaming fashion.<\/li><\/ul>\n<h4><span class=\"mw-headline\" id=\"Scenarios\">Scenarios<\/span><\/h4>\n<p>S<sub>1<\/sub>: \"As a user (i.e., stakeholder), I want to get the set of instances, {I} (i.e., data), for the last two months for the 'entities being observed,' E, in the 'viral disease' domain, D, with a sub-domain 'COVID-19.'\"\nS<sub>2<\/sub>: \"As a user, I want a timeline (i.e., an analysis) of the entities, E, being observed in the 'breast cancer' sub-domain for data providers from the Americas (or for a specific clinical trial data provider).\"\nS<sub>3<\/sub>: \"I want to get the analysis of successful trials in the last two years in the 'intestinal disease' domain.\"\nS<sub>4<\/sub>: \"I want to get all the instances, {I}, for all the entities, {E}, in the 'cancer' domain, D, with trial outcomes<sup id=\"rdp-ebb-cite_ref-:5_23-1\" class=\"reference\"><a href=\"#cite_note-:5-23\">[23]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:6_24-1\" class=\"reference\"><a href=\"#cite_note-:6-24\">[24]<\/a><\/sup> to train a machine learning algorithm.\"\nS<sub>5<\/sub>: \"As a pharmaceutical company analyst (i.e., stakeholder), I want to see the duration of affiliation of all investigators from sites in Europe associated with clinical trials in the domain of 'cancer'.\"\n<\/p><p>Online scenarios represent the analyses that will be executed by the \u201cdata service\u201d and might result in two options: (1) when the analysis (query) completes, stream\/send the result back to the requested; (2) store the result in the data lake and stream\/send the result back. The second case needs to be implemented such that one can cache the analysis result in memory; if the request is repeated after a certain threshold (or another such metric can be defined), one can persist the analysis result to the data lake.\n<\/p><p>All such analytical results at the time of deciding to persist to the data lake will first go through the process of on-boarding. As described in the on-boarding process, each dataset that lands in the data lake must have a schema, Sch. In this case, a schema is registered, semantically annotated, and then the dataset is written to the data lake for reasons described in the framework description and the \"derived\" data sources description.\n<\/p><p>Figure 6 shows the sequence of activities that are performed to execute a scenario by interacting with the platform. Here, the internals of the platform are hidden, and only the interactions between components are shown.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig6_Siddiqi_Healthcare23_11-12.png\" class=\"image wiki-link\" data-key=\"389d4dd6c283b8d16b1791d7fc3b41b2\"><img alt=\"Fig6 Siddiqi Healthcare23 11-12.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/b\/b4\/Fig6_Siddiqi_Healthcare23_11-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 6.<\/b> Sequence of activities in a scenario.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>Similarly, Figure 7 shows the process of a scenario based on parameters provided in the scenario. For example, if the scenario asks for the analysis of sites of clinical trials, then the data service invokes those particular components to perform those functions seamlessly, and so on.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig7_Siddiqi_Healthcare23_11-12.png\" class=\"image wiki-link\" data-key=\"d654ae140b49f73fe091b05fa6670e2a\"><img alt=\"Fig7 Siddiqi Healthcare23 11-12.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/1\/19\/Fig7_Siddiqi_Healthcare23_11-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 7.<\/b> Business process of scenario execution..<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h2><span id=\"rdp-ebb-Experiments,_results,_and_evaluation\"><\/span><span class=\"mw-headline\" id=\"Experiments.2C_results.2C_and_evaluation\">Experiments, results, and evaluation<\/span><\/h2>\n<p>This section presents implementation details, followed by results and the evaluation of results and computational advantages.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Implementation\">Implementation<\/span><\/h3>\n<p>A prototype version of the proposed framework is implemented with each component, as presented in Figure 1, to reflect the use cases and provide a minimal viable prototype (MVP). More specifically, we took the case of managing \u201cclinical\u201d trials as a case study for this prototype. Clinical trial data are obtained from the aforementioned repositories available publicly, where one can obtain a history of trials as well. The specific clinical trial repositories used are BSMO trials and CityGov trials, and for academic research manuscripts, we used research articles from well-known research journals such as <i>The Journal of Clinical Investigation<\/i><sup id=\"rdp-ebb-cite_ref-29\" class=\"reference\"><a href=\"#cite_note-29\">[29]<\/a><\/sup> and Sage Journals' <i>Clinical Trials<\/i>.<sup id=\"rdp-ebb-cite_ref-30\" class=\"reference\"><a href=\"#cite_note-30\">[30]<\/a><\/sup> We used Apache Hudi as the data storage format (using parquet file format), which is compact, provides time-series data checkpointed by any update\/upserts, and provides efficient query processing using the query engine Spark. Apache Hudi was used since it supports parquet, which is format-agnostic, succinct, and fast.\n<\/p><p>For storage of data, Azure blob storage is used as the clinical data lake, where the raw data are stored, and it has also been used to store the ingested data in the Hudi format as Hudi tables (which are time-seriesed and checkpointed), which is also called the data lake. This storage selection is purely optional, and it can be any file system storage. Scala is used as a programming language, with Apache Kafka as a message-passing and data streaming framework, and Apache Spark as a distributed data processing framework. Moreover, Lightbend Cloudflow is integrated with these technologies and hence used in the replacement of microservices.\n<\/p><p>The process of data acquisition, management, and processing takes place as follows: First, we scrape data from available public repositories provided by providers and log it to two places: Kafka messages and the data lake. Then, we ingest the data from Kafka (each Kafka message conforms to a schema defined in the Avro format specific for that dataset) in a streaming pipeline integrating Kafka and write to the data lake (bottom layer in Figure 1). This step brings any format of data into a single format, and we can use our processing framework Spark with ease to perform any type of analytics (next layer from bottom in Figure 1).\n<\/p><p>We implemented a data analysis service (3rd layer from bottom in Figure 1) to correlate information from profiles, clinical trials, and trial subjects to identify the clinical profiles of people. This included various forms of information extraction and knowledge creation, such as the creation of the ontology to construct the relationship between these various datasets. Similarly, we also implemented a front-end to register assets (datasets and providers) so that we could later query and prove that such a concept does exist. We leveraged the datahub\u2019s data model<sup id=\"rdp-ebb-cite_ref-:7_28-2\" class=\"reference\"><a href=\"#cite_note-:7-28\">[28]<\/a><\/sup> for that purpose and found that it is one of the best metadata models for registering and managing assets linked with glossary terms and business vocabularies, as well as technical vocabularies, and defining roles and ownerships.\n<\/p><p>Moreover, the clinical trial data contain textual data wherein personal and investigation site profiles repeat. This type of data needs correlational and contextual analysis to deduce the real person from it. That is mainly because the person profiles are manually filled in by various people in the trial data or because the people are active in various sites and organizations, and hence a single profile can have a lot of variations. For example, a person\u2019s name \u201cKasper Sukre\u201d can be written as \u201cK. Sukre,\u201d \u201cKasper S,\u201d \u201cK. S,\u201d etc. We therefore needed to resolve all those names to a single person entity. This entity is essentially a person entity in the relational schema of the clinical trial data that we take as a case study.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Results_and_evaluation\">Results and evaluation<\/span><\/h3>\n<p>We implemented textual person profile prediction using clustering algorithms (2nd layer from the top in Figure 1). We compare the results of two clustering algorithms with two different types of distance algorithms. In the first distance algorithm, since a person profile contains various features such as name, address, cell, email, association, and similarly, address and association can have further sub-fields; therefore, we compute the edit distance between each sub-field separately, which is a nested distance. Finally, we take an average of all distances. In the second distance algorithm, we apply a generic edit distance to the whole person profile as a string and then apply the clustering algorithms. Table 1 shows the results of the two algorithms for the two variants separately.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"3\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 1.<\/b> The results of the two clustering algorithms for two different variants of distance algorithms.\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Clustering algorithm\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">General edit distance\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Feature-wise edit distance\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">K-Means\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">79.43%\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">85.4%\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">EM\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">83.54%\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">88.9%\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>As can be seen in Table 1, the algorithm EM consistently performs better than K-Means on both variants of the distance algorithm. Since the algorithms are not 100% accurate, and it makes sense, we therefore employ a quality check on the results to verify the final person recommendation that is also required since we cannot completely rely on algorithms for personal data.\n<\/p><p>Next, we must also indicate that we were able to stream data from the scraper that scrapes the data from various repositories simultaneously in parallel and were able to write to the same table (the Apache Hudi table) time-series data at around five times faster than normal upserts to a relational database. Since the framework is designed to poll data sources regularly and as soon as data are available, it is seamlessly brought into the system using the automated pipeline, and the algorithms to link and personalize are performed on the fly on the incoming data. Note here that, when any new data arrives in the system, the algorithms implemented above (analytical and predictive) are also part of the pipeline orchestration. Therefore, this avoids any cron jobs, periodic, and\/or manual jobs to perform these tasks, and if necessary, data can already be streamed or made available to applications such as Kafka messages. This kind of automation, fusion, and parallelism makes this framework unique in that it speeds up the manual process used by the existing systems that follow the process of scraping data, ingesting it periodically, and then running cron jobs to perform analytics. Apart from the automation of the process, existing systems, as explained in the state-of-the-art, also lack data fusion of trial data, academic articles, Medical Subject Heading ontologies, and linking profiles.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Discussion\">Discussion<\/span><\/h2>\n<p>In this paper, we present a healthcare framework that enables us to handle health-related data. When clinical trials are under investigation and need an advanced and automated system to speed up the trial process and monitor the treatments, investigations at sites, their outcomes, and other related aspects of trial management, we presented a framework that streams data from various sources, fuses the data semantically, applies algorithms to resolve duplicates, and handles the history of events. This does not only speed up the trial analysis process compared to the existing methods; it also enriches the trial analysis and gives a complete context with the latest research from academia, semantic ontologies, and a holistic view of events. The framework is designed to support the declaration of schemas (type-level information) and metadata related to various data that comes into the framework. This enables exploring any analysis being made over the data and making it accessible to third parties as well as government and other agencies to use data from data providers, express interests, and\/or bring their own data to the system to be used by other such users of the system.\n<\/p><p>We have presented scenarios in which this framework can be utilized. Not limited to these scenarios, the framework can be extended to support a variety of specialized data systems; for example, they include complex event processing, weather monitoring, and traffic and air management, among others. This is all because we have leveraged the data lake technology and provided abstractions that can be used for any kind of querying of the data. Moreover, the framework is designed to support data lineage, which helps in achieving data accuracy and provenance. This is mainly obtained by keeping track of each transformation and operation performed within the framework, and thus it can be reproduced either at the operation level, data level, or type level. All these help in evidence-based decision-making and thus prove that the framework conforms to and fulfills the critical needs of a system needed for healthcare-related sensitive data.\n<\/p><p>Furthermore, the framework is designed to be extensible, and it envisions the inclusion of automated feature engineering required to generate recommendations from historical data based on statistical models and methods. This can further be used to plug and play models in the framework, which leads to using data from one model provided by a model provider or a third party using the model and data from separate users. Hence, this framework in this paper is limited to clinical data processing, lineage tracking, and governance and applies and extends to other areas of research as well.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Conclusions\">Conclusions<\/span><\/h2>\n<p>In this research article, we presented a detailed overview of the need for a framework for clinical investigation to speed up the process of clinical trials for medical equipment, vaccines, and other such products, especially in the event of pandemics such as COVID-19.<sup id=\"rdp-ebb-cite_ref-:1_4-1\" class=\"reference\"><a href=\"#cite_note-:1-4\">[4]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-31\" class=\"reference\"><a href=\"#cite_note-31\">[31]<\/a><\/sup> This is necessary since a delay in such matters causes the death of humans, and saving a single life is analogous to saving humanity. Moreover, the presented framework has the potential to provide evidence-based data analysis through lineage tracking; data governance capability to explore, visualize semantics and links, and possibly make decisions among various data sets available in the framework for use; and process data at state and in motion. We have presented use cases that showcase the usage of the framework in different scenarios and how it can be used by both private, public, and private-public organizations. The framework is extendable, encapsulates the abilities to include other domains, provides support for semantic querying at the governance level (i.e., at the data assets and type assets level), and is designed towards a data and computation economic model. Finally, it supports evidence-based decision tracking, and hence nothing is lost or unknown in the process of evaluation and analysis, which follows the FAIR principle. Future work includes a detailed investigation of the specifics of each part of the framework in the domains of semantics, learning models, and the incremental evaluation of queries that need to be evaluated on updates in real-time data processing.\n<\/p>\n<h2><span id=\"rdp-ebb-Abbreviations,_acronyms,_and_initialisms\"><\/span><span class=\"mw-headline\" id=\"Abbreviations.2C_acronyms.2C_and_initialisms\">Abbreviations, acronyms, and initialisms<\/span><\/h2>\n<ul><li><b>AI<\/b>: artificial intelligence<\/li>\n<li><b>API<\/b>: application programming interface<\/li>\n<li><b>CDC<\/b>: change data capture<\/li>\n<li><b>CDSS<\/b>: clinical decision support system<\/li>\n<li><b>EHR<\/b>: electronic health record<\/li>\n<li><b>FAIR<\/b>: findable, accessible, interoperable, and reusable<\/li>\n<li><b>HL7<\/b>: Health Level 7<\/li>\n<li><b>MeSH<\/b>: Medical Subject Headings<\/li>\n<li><b>ML<\/b>: machine learning<\/li>\n<li><b>MVP<\/b>: minimal viable prototype<\/li>\n<li><b>UC<\/b>: use case<\/li><\/ul>\n<h2><span class=\"mw-headline\" id=\"Acknowledgements\">Acknowledgements<\/span><\/h2>\n<h3><span class=\"mw-headline\" id=\"Author_contributions\">Author contributions<\/span><\/h3>\n<p>Conceptualization, methodology, and validation, M.H.S. and M.I.; formal analysis, M.H.S.; resources, M.A.; data curation, M.A. and M.I.; writing\u2014review and editing, M.H.S. and M.I.; funding acquisition, M.A. All authors have read and agreed to the published version of the manuscript.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Funding\">Funding<\/span><\/h3>\n<p>This work was funded by the Deanship of Scientific Research at Jouf University under Grant number DSR-2022-RG-0101.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Conflicts_of_interest\">Conflicts of interest<\/span><\/h3>\n<p>The authors declare no conflict of interest.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"References\">References<\/span><\/h2>\n<div class=\"reflist references-column-width\" style=\"-moz-column-width: 30em; -webkit-column-width: 30em; column-width: 30em; list-style-type: decimal;\">\n<div class=\"mw-references-wrap mw-references-columns\"><ol class=\"references\">\n<li id=\"cite_note-1\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-1\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation book\">Friedman, Lawrence M.; Furberg, Curt D.; DeMets, David L.; Reboussin, David M.; Granger, Christopher B. (2015) (in en). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/link.springer.com\/10.1007\/978-3-319-18539-2\" target=\"_blank\"><i>Fundamentals of Clinical Trials<\/i><\/a>. Cham: Springer International Publishing. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2F978-3-319-18539-2\" target=\"_blank\">10.1007\/978-3-319-18539-2<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-3-319-18538-5<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/link.springer.com\/10.1007\/978-3-319-18539-2\" target=\"_blank\">https:\/\/link.springer.com\/10.1007\/978-3-319-18539-2<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=Fundamentals+of+Clinical+Trials&rft.aulast=Friedman&rft.aufirst=Lawrence+M.&rft.au=Friedman%2C%26%2332%3BLawrence+M.&rft.au=Furberg%2C%26%2332%3BCurt+D.&rft.au=DeMets%2C%26%2332%3BDavid+L.&rft.au=Reboussin%2C%26%2332%3BDavid+M.&rft.au=Granger%2C%26%2332%3BChristopher+B.&rft.date=2015&rft.place=Cham&rft.pub=Springer+International+Publishing&rft_id=info:doi\/10.1007%2F978-3-319-18539-2&rft.isbn=978-3-319-18538-5&rft_id=https%3A%2F%2Flink.springer.com%2F10.1007%2F978-3-319-18539-2&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:0-2\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:0_2-0\">2.0<\/a><\/sup> <sup><a href=\"#cite_ref-:0_2-1\">2.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.clinicaltrials.gov\/\" target=\"_blank\">\"ClinicalTrials.gov\"<\/a>. National Library of Medicine<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.clinicaltrials.gov\/\" target=\"_blank\">https:\/\/www.clinicaltrials.gov\/<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 29 August 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=ClinicalTrials.gov&rft.atitle=&rft.pub=National+Library+of+Medicine&rft_id=https%3A%2F%2Fwww.clinicaltrials.gov%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-3\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-3\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.bsmo.be\/clinical\/clinical-trials\/\" target=\"_blank\">\"Clinical Trials\"<\/a>. Belgian Society for Medical Oncology<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.bsmo.be\/clinical\/clinical-trials\/\" target=\"_blank\">https:\/\/www.bsmo.be\/clinical\/clinical-trials\/<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 29 August 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Clinical+Trials&rft.atitle=&rft.pub=Belgian+Society+for+Medical+Oncology&rft_id=https%3A%2F%2Fwww.bsmo.be%2Fclinical%2Fclinical-trials%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:1-4\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:1_4-0\">4.0<\/a><\/sup> <sup><a href=\"#cite_ref-:1_4-1\">4.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Plug, R.; Liang, Y.; Basajja, M. et al. (2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ceur-ws.org\/Vol-3127\/paper-7.pdf\" target=\"_blank\">\"FAIR and GDPR Compliant Population Health Data Generation, Processing and Analytics\"<\/a> (PDF). <i>Proceedings of the 13th International Conference on Semantic Web Applications and Tools for Health Care and Life Sciences<\/i>: 1\u201310<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ceur-ws.org\/Vol-3127\/paper-7.pdf\" target=\"_blank\">https:\/\/ceur-ws.org\/Vol-3127\/paper-7.pdf<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=FAIR+and+GDPR+Compliant+Population+Health+Data+Generation%2C+Processing+and+Analytics&rft.jtitle=Proceedings+of+the+13th+International+Conference+on+Semantic+Web+Applications+and+Tools+for+Health+Care+and+Life+Sciences&rft.aulast=Plug%2C+R.%3B+Liang%2C+Y.%3B+Basajja%2C+M.+et+al.&rft.au=Plug%2C+R.%3B+Liang%2C+Y.%3B+Basajja%2C+M.+et+al.&rft.date=2022&rft.pages=1%E2%80%9310&rft_id=https%3A%2F%2Fceur-ws.org%2FVol-3127%2Fpaper-7.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-5\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-5\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation book\">Meinert, Curtis L. (2012). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/756044508\" target=\"_blank\"><i>Clinical trials: design, conduct, and analysis<\/i><\/a>. Monographs in epidemiology and biostatistics. <b>39<\/b> (2nd ed ed.). New York: Oxford University Press. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-0-19-538788-9. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Online_Computer_Library_Center\" data-key=\"b53206e2204c7e657858a88b56c8ac4a\">OCLC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/oclc\/756044508\" target=\"_blank\">756044508<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/756044508\" target=\"_blank\">https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/756044508<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=Clinical+trials%3A+design%2C+conduct%2C+and+analysis&rft.aulast=Meinert&rft.aufirst=Curtis+L.&rft.au=Meinert%2C%26%2332%3BCurtis+L.&rft.date=2012&rft.series=Monographs+in+epidemiology+and+biostatistics&rft.volume=39&rft.edition=2nd+ed&rft.place=New+York&rft.pub=Oxford+University+Press&rft.isbn=978-0-19-538788-9&rft_id=info:oclcnum\/756044508&rft_id=https%3A%2F%2Fwww.worldcat.org%2Ftitle%2Fmediawiki%2Foclc%2F756044508&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-6\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-6\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Dash, Sabyasachi; Shakyawar, Sushil Kumar; Sharma, Mohit; Kaushik, Sandeep (1 December 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/journalofbigdata.springeropen.com\/articles\/10.1186\/s40537-019-0217-0\" target=\"_blank\">\"Big data in healthcare: management, analysis and future prospects\"<\/a> (in en). <i>Journal of Big Data<\/i> <b>6<\/b> (1): 54. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1186%2Fs40537-019-0217-0\" target=\"_blank\">10.1186\/s40537-019-0217-0<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2196-1115\" target=\"_blank\">2196-1115<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/journalofbigdata.springeropen.com\/articles\/10.1186\/s40537-019-0217-0\" target=\"_blank\">https:\/\/journalofbigdata.springeropen.com\/articles\/10.1186\/s40537-019-0217-0<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Big+data+in+healthcare%3A+management%2C+analysis+and+future+prospects&rft.jtitle=Journal+of+Big+Data&rft.aulast=Dash&rft.aufirst=Sabyasachi&rft.au=Dash%2C%26%2332%3BSabyasachi&rft.au=Shakyawar%2C%26%2332%3BSushil+Kumar&rft.au=Sharma%2C%26%2332%3BMohit&rft.au=Kaushik%2C%26%2332%3BSandeep&rft.date=1+December+2019&rft.volume=6&rft.issue=1&rft.pages=54&rft_id=info:doi\/10.1186%2Fs40537-019-0217-0&rft.issn=2196-1115&rft_id=https%3A%2F%2Fjournalofbigdata.springeropen.com%2Farticles%2F10.1186%2Fs40537-019-0217-0&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-7\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-7\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Dziura, James D.; Post, Lori A.; Zhao, Qing; Fu, Zhixuan; Peduzzi, Peter (1 September 2013). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/24058309\" target=\"_blank\">\"Strategies for dealing with missing data in clinical trials: from design to analysis\"<\/a>. <i>The Yale Journal of Biology and Medicine<\/i> <b>86<\/b> (3): 343\u2013358. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1551-4056\" target=\"_blank\">1551-4056<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/3767219\/\" target=\"_blank\">3767219<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/24058309\" target=\"_blank\">24058309<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/24058309\" target=\"_blank\">https:\/\/pubmed.ncbi.nlm.nih.gov\/24058309<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Strategies+for+dealing+with+missing+data+in+clinical+trials%3A+from+design+to+analysis&rft.jtitle=The+Yale+Journal+of+Biology+and+Medicine&rft.aulast=Dziura&rft.aufirst=James+D.&rft.au=Dziura%2C%26%2332%3BJames+D.&rft.au=Post%2C%26%2332%3BLori+A.&rft.au=Zhao%2C%26%2332%3BQing&rft.au=Fu%2C%26%2332%3BZhixuan&rft.au=Peduzzi%2C%26%2332%3BPeter&rft.date=1+September+2013&rft.volume=86&rft.issue=3&rft.pages=343%E2%80%93358&rft.issn=1551-4056&rft_id=info:pmc\/3767219&rft_id=info:pmid\/24058309&rft_id=https%3A%2F%2Fpubmed.ncbi.nlm.nih.gov%2F24058309&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-8\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-8\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Bahri, Safa; Zoghlami, Nesrine; Abed, Mourad; Tavares, Joao Manuel R. S. (2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/8585021\/\" target=\"_blank\">\"BIG DATA for Healthcare: A Survey\"<\/a>. <i>IEEE Access<\/i> <b>7<\/b>: 7397\u20137408. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FACCESS.2018.2889180\" target=\"_blank\">10.1109\/ACCESS.2018.2889180<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2169-3536\" target=\"_blank\">2169-3536<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/8585021\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/8585021\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=BIG+DATA+for+Healthcare%3A+A+Survey&rft.jtitle=IEEE+Access&rft.aulast=Bahri&rft.aufirst=Safa&rft.au=Bahri%2C%26%2332%3BSafa&rft.au=Zoghlami%2C%26%2332%3BNesrine&rft.au=Abed%2C%26%2332%3BMourad&rft.au=Tavares%2C%26%2332%3BJoao+Manuel+R.+S.&rft.date=2019&rft.volume=7&rft.pages=7397%E2%80%937408&rft_id=info:doi\/10.1109%2FACCESS.2018.2889180&rft.issn=2169-3536&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F8585021%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:2-9\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:2_9-0\">9.0<\/a><\/sup> <sup><a href=\"#cite_ref-:2_9-1\">9.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation\" id=\"rdp-ebb-CITEREFSchulzStegweeChronaki2019\">Schulz, Stefan; Stegwee, Robert; Chronaki, Catherine (2019), Kubben, Pieter; Dumontier, Michel; Dekker, Andre, eds., <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/978-3-319-99713-1_3\" target=\"_blank\">\"Standards in Healthcare Data\"<\/a> (in en), <i>Fundamentals of Clinical Data Science<\/i> (Cham: Springer International Publishing): 19\u201336, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2F978-3-319-99713-1_3\" target=\"_blank\">10.1007\/978-3-319-99713-1_3<\/a>, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-3-319-99712-4<span class=\"printonly\">, <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/978-3-319-99713-1_3\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/978-3-319-99713-1_3<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 2023-07-20<\/span><\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Standards+in+Healthcare+Data&rft.jtitle=Fundamentals+of+Clinical+Data+Science&rft.aulast=Schulz&rft.aufirst=Stefan&rft.au=Schulz%2C%26%2332%3BStefan&rft.au=Stegwee%2C%26%2332%3BRobert&rft.au=Chronaki%2C%26%2332%3BCatherine&rft.date=2019&rft.pages=19%E2%80%9336&rft.place=Cham&rft.pub=Springer+International+Publishing&rft_id=info:doi\/10.1007%2F978-3-319-99713-1_3&rft.isbn=978-3-319-99712-4&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2F978-3-319-99713-1_3&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-10\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-10\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Hussain, Maqbool; Afzal, Muhammad; Ali, Taqdir; Ali, Rahman; Khan, Wajahat Ali; Jamshed, Arif; Lee, Sungyoung; Kang, Byeong Ho <i>et al.<\/i> (1 November 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0933365715001256\" target=\"_blank\">\"Data-driven knowledge acquisition, validation, and transformation into HL7 Arden Syntax\"<\/a> (in en). <i>Artificial Intelligence in Medicine<\/i> <b>92<\/b>: 51\u201370. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.artmed.2015.09.008\" target=\"_blank\">10.1016\/j.artmed.2015.09.008<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0933365715001256\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0933365715001256<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Data-driven+knowledge+acquisition%2C+validation%2C+and+transformation+into+HL7+Arden+Syntax&rft.jtitle=Artificial+Intelligence+in+Medicine&rft.aulast=Hussain&rft.aufirst=Maqbool&rft.au=Hussain%2C%26%2332%3BMaqbool&rft.au=Afzal%2C%26%2332%3BMuhammad&rft.au=Ali%2C%26%2332%3BTaqdir&rft.au=Ali%2C%26%2332%3BRahman&rft.au=Khan%2C%26%2332%3BWajahat+Ali&rft.au=Jamshed%2C%26%2332%3BArif&rft.au=Lee%2C%26%2332%3BSungyoung&rft.au=Kang%2C%26%2332%3BByeong+Ho&rft.au=Latif%2C%26%2332%3BKhalid&rft.date=1+November+2018&rft.volume=92&rft.pages=51%E2%80%9370&rft_id=info:doi\/10.1016%2Fj.artmed.2015.09.008&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0933365715001256&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-11\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-11\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ethier, J.-F.; Curcin, V.; Barton, A.; McGilchrist, M. M.; Bastiaens, H.; Andreasson, A.; Rossiter, J.; Zhao, L. <i>et al.<\/i> (2015). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/24954896\" target=\"_blank\">\"Clinical data integration model. Core interoperability ontology for research using primary care data\"<\/a>. <i>Methods of Information in Medicine<\/i> <b>54<\/b> (1): 16\u201323. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3414%2FME13-02-0024\" target=\"_blank\">10.3414\/ME13-02-0024<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2511-705X\" target=\"_blank\">2511-705X<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/24954896\" target=\"_blank\">24954896<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/24954896\" target=\"_blank\">https:\/\/pubmed.ncbi.nlm.nih.gov\/24954896<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Clinical+data+integration+model.+Core+interoperability+ontology+for+research+using+primary+care+data&rft.jtitle=Methods+of+Information+in+Medicine&rft.aulast=Ethier&rft.aufirst=J.-F.&rft.au=Ethier%2C%26%2332%3BJ.-F.&rft.au=Curcin%2C%26%2332%3BV.&rft.au=Barton%2C%26%2332%3BA.&rft.au=McGilchrist%2C%26%2332%3BM.+M.&rft.au=Bastiaens%2C%26%2332%3BH.&rft.au=Andreasson%2C%26%2332%3BA.&rft.au=Rossiter%2C%26%2332%3BJ.&rft.au=Zhao%2C%26%2332%3BL.&rft.au=Arvanitis%2C%26%2332%3BT.+N.&rft.date=2015&rft.volume=54&rft.issue=1&rft.pages=16%E2%80%9323&rft_id=info:doi\/10.3414%2FME13-02-0024&rft.issn=2511-705X&rft_id=info:pmid\/24954896&rft_id=https%3A%2F%2Fpubmed.ncbi.nlm.nih.gov%2F24954896&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:3-12\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:3_12-0\">12.0<\/a><\/sup> <sup><a href=\"#cite_ref-:3_12-1\">12.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Brundage, Michael; Blazeby, Jane; Revicki, Dennis; Bass, Brenda; de Vet, Henrica; Duffy, Helen; Efficace, Fabio; King, Madeleine <i>et al.<\/i> (1 August 2013). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/s11136-012-0252-1\" target=\"_blank\">\"Patient-reported outcomes in randomized clinical trials: development of ISOQOL reporting standards\"<\/a> (in en). <i>Quality of Life Research<\/i> <b>22<\/b> (6): 1161\u20131175. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2Fs11136-012-0252-1\" target=\"_blank\">10.1007\/s11136-012-0252-1<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0962-9343\" target=\"_blank\">0962-9343<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC3731511\/\" target=\"_blank\">PMC3731511<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/22987144\" target=\"_blank\">22987144<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/s11136-012-0252-1\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/s11136-012-0252-1<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Patient-reported+outcomes+in+randomized+clinical+trials%3A+development+of+ISOQOL+reporting+standards&rft.jtitle=Quality+of+Life+Research&rft.aulast=Brundage&rft.aufirst=Michael&rft.au=Brundage%2C%26%2332%3BMichael&rft.au=Blazeby%2C%26%2332%3BJane&rft.au=Revicki%2C%26%2332%3BDennis&rft.au=Bass%2C%26%2332%3BBrenda&rft.au=de+Vet%2C%26%2332%3BHenrica&rft.au=Duffy%2C%26%2332%3BHelen&rft.au=Efficace%2C%26%2332%3BFabio&rft.au=King%2C%26%2332%3BMadeleine&rft.au=Lam%2C%26%2332%3BCindy+L.+K.&rft.date=1+August+2013&rft.volume=22&rft.issue=6&rft.pages=1161%E2%80%931175&rft_id=info:doi\/10.1007%2Fs11136-012-0252-1&rft.issn=0962-9343&rft_id=info:pmc\/PMC3731511&rft_id=info:pmid\/22987144&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2Fs11136-012-0252-1&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:4-13\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:4_13-0\">13.0<\/a><\/sup> <sup><a href=\"#cite_ref-:4_13-1\">13.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Banos, Oresti; Bilal Amin, Muhammad; Ali Khan, Wajahat; Afzal, Muhammad; Hussain, Maqbool; Kang, Byeong Ho; Lee, Sungyong (1 July 2016). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/biomedical-engineering-online.biomedcentral.com\/articles\/10.1186\/s12938-016-0179-9\" target=\"_blank\">\"The Mining Minds digital health and wellness framework\"<\/a> (in en). <i>BioMedical Engineering OnLine<\/i> <b>15<\/b> (S1): 76. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1186%2Fs12938-016-0179-9\" target=\"_blank\">10.1186\/s12938-016-0179-9<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1475-925X\" target=\"_blank\">1475-925X<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC4959395\/\" target=\"_blank\">PMC4959395<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/27454608\" target=\"_blank\">27454608<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/biomedical-engineering-online.biomedcentral.com\/articles\/10.1186\/s12938-016-0179-9\" target=\"_blank\">http:\/\/biomedical-engineering-online.biomedcentral.com\/articles\/10.1186\/s12938-016-0179-9<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+Mining+Minds+digital+health+and+wellness+framework&rft.jtitle=BioMedical+Engineering+OnLine&rft.aulast=Banos&rft.aufirst=Oresti&rft.au=Banos%2C%26%2332%3BOresti&rft.au=Bilal+Amin%2C%26%2332%3BMuhammad&rft.au=Ali+Khan%2C%26%2332%3BWajahat&rft.au=Afzal%2C%26%2332%3BMuhammad&rft.au=Hussain%2C%26%2332%3BMaqbool&rft.au=Kang%2C%26%2332%3BByeong+Ho&rft.au=Lee%2C%26%2332%3BSungyong&rft.date=1+July+2016&rft.volume=15&rft.issue=S1&rft.pages=76&rft_id=info:doi\/10.1186%2Fs12938-016-0179-9&rft.issn=1475-925X&rft_id=info:pmc\/PMC4959395&rft_id=info:pmid\/27454608&rft_id=http%3A%2F%2Fbiomedical-engineering-online.biomedcentral.com%2Farticles%2F10.1186%2Fs12938-016-0179-9&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-14\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-14\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Alballa, Norah; Al-Turaiki, Isra (2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S235291482100054X\" target=\"_blank\">\"Machine learning approaches in COVID-19 diagnosis, mortality, and severity risk prediction: A review\"<\/a> (in en). <i>Informatics in Medicine Unlocked<\/i> <b>24<\/b>: 100564. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.imu.2021.100564\" target=\"_blank\">10.1016\/j.imu.2021.100564<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC8018906\/\" target=\"_blank\">PMC8018906<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/33842685\" target=\"_blank\">33842685<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S235291482100054X\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S235291482100054X<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Machine+learning+approaches+in+COVID-19+diagnosis%2C+mortality%2C+and+severity+risk+prediction%3A+A+review&rft.jtitle=Informatics+in+Medicine+Unlocked&rft.aulast=Alballa&rft.aufirst=Norah&rft.au=Alballa%2C%26%2332%3BNorah&rft.au=Al-Turaiki%2C%26%2332%3BIsra&rft.date=2021&rft.volume=24&rft.pages=100564&rft_id=info:doi\/10.1016%2Fj.imu.2021.100564&rft_id=info:pmc\/PMC8018906&rft_id=info:pmid\/33842685&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS235291482100054X&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-15\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-15\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Majumder, Joydeb; Minko, Tamara (1 January 2021). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1208\/s12248-020-00532-2\" target=\"_blank\">\"Recent Developments on Therapeutic and Diagnostic Approaches for COVID-19\"<\/a> (in en). <i>The AAPS Journal<\/i> <b>23<\/b> (1): 14. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1208%2Fs12248-020-00532-2\" target=\"_blank\">10.1208\/s12248-020-00532-2<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1550-7416\" target=\"_blank\">1550-7416<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC7784226\/\" target=\"_blank\">PMC7784226<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/33400058\" target=\"_blank\">33400058<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1208\/s12248-020-00532-2\" target=\"_blank\">http:\/\/link.springer.com\/10.1208\/s12248-020-00532-2<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Recent+Developments+on+Therapeutic+and+Diagnostic+Approaches+for+COVID-19&rft.jtitle=The+AAPS+Journal&rft.aulast=Majumder&rft.aufirst=Joydeb&rft.au=Majumder%2C%26%2332%3BJoydeb&rft.au=Minko%2C%26%2332%3BTamara&rft.date=1+January+2021&rft.volume=23&rft.issue=1&rft.pages=14&rft_id=info:doi\/10.1208%2Fs12248-020-00532-2&rft.issn=1550-7416&rft_id=info:pmc\/PMC7784226&rft_id=info:pmid\/33400058&rft_id=http%3A%2F%2Flink.springer.com%2F10.1208%2Fs12248-020-00532-2&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-16\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-16\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Hussain, M.; Khattak, A. M.; Khan, W. A.; Fatima, I.; Amin, M. B.; Pervez, Z.; Batool, R.; Saleem, M. A. <i>et al.<\/i> (1 June 2013). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/s12553-013-0051-x\" target=\"_blank\">\"Cloud-based Smart CDSS for chronic diseases\"<\/a> (in en). <i>Health and Technology<\/i> <b>3<\/b> (2): 153\u2013175. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2Fs12553-013-0051-x\" target=\"_blank\">10.1007\/s12553-013-0051-x<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2190-7188\" target=\"_blank\">2190-7188<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/s12553-013-0051-x\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/s12553-013-0051-x<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Cloud-based+Smart+CDSS+for+chronic+diseases&rft.jtitle=Health+and+Technology&rft.aulast=Hussain&rft.aufirst=M.&rft.au=Hussain%2C%26%2332%3BM.&rft.au=Khattak%2C%26%2332%3BA.+M.&rft.au=Khan%2C%26%2332%3BW.+A.&rft.au=Fatima%2C%26%2332%3BI.&rft.au=Amin%2C%26%2332%3BM.+B.&rft.au=Pervez%2C%26%2332%3BZ.&rft.au=Batool%2C%26%2332%3BR.&rft.au=Saleem%2C%26%2332%3BM.+A.&rft.au=Afzal%2C%26%2332%3BM.&rft.date=1+June+2013&rft.volume=3&rft.issue=2&rft.pages=153%E2%80%93175&rft_id=info:doi\/10.1007%2Fs12553-013-0051-x&rft.issn=2190-7188&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2Fs12553-013-0051-x&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-17\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-17\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Zu, D.; Li, D.; Carterette, B. et al. (2013). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ceur-ws.org\/Vol-1094\/bioasq2013_submission_7.pdf\" target=\"_blank\">\"An Incremental Approach for MEDLINE MeSH Indexing\"<\/a> (PDF). <i>Proceedings of the first Workshop on Bio-Medical Semantic Indexing and Question Answering, a Post-Conference Workshop of Conference and Labs of the Evaluation Forum 2013 (CLEF 2013)<\/i> <b>1094<\/b><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ceur-ws.org\/Vol-1094\/bioasq2013_submission_7.pdf\" target=\"_blank\">https:\/\/ceur-ws.org\/Vol-1094\/bioasq2013_submission_7.pdf<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=An+Incremental+Approach+for+MEDLINE+MeSH+Indexing&rft.jtitle=Proceedings+of+the+first+Workshop+on+Bio-Medical+Semantic+Indexing+and+Question+Answering%2C+a+Post-Conference+Workshop+of+Conference+and+Labs+of+the+Evaluation+Forum+2013+%28CLEF+2013%29&rft.aulast=Zu%2C+D.%3B+Li%2C+D.%3B+Carterette%2C+B.+et+al.&rft.au=Zu%2C+D.%3B+Li%2C+D.%3B+Carterette%2C+B.+et+al.&rft.date=2013&rft.volume=1094&rft_id=https%3A%2F%2Fceur-ws.org%2FVol-1094%2Fbioasq2013_submission_7.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-18\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-18\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Ikeda, R.; Widom, J. (2009). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/citeseerx.ist.psu.edu\/document?repid=rep1&type=pdf&doi=af8988cb698ce8641cb5fa239c149295716ebc76\" target=\"_blank\">\"Data Lineage: A Survey\"<\/a>. Stanford University<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/citeseerx.ist.psu.edu\/document?repid=rep1&type=pdf&doi=af8988cb698ce8641cb5fa239c149295716ebc76\" target=\"_blank\">https:\/\/citeseerx.ist.psu.edu\/document?repid=rep1&type=pdf&doi=af8988cb698ce8641cb5fa239c149295716ebc76<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Data+Lineage%3A+A+Survey&rft.atitle=&rft.aulast=Ikeda%2C+R.%3B+Widom%2C+J.&rft.au=Ikeda%2C+R.%3B+Widom%2C+J.&rft.date=2009&rft.pub=Stanford+University&rft_id=https%3A%2F%2Fciteseerx.ist.psu.edu%2Fdocument%3Frepid%3Drep1%26type%3Dpdf%26doi%3Daf8988cb698ce8641cb5fa239c149295716ebc76&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-19\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-19\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Mobley, Aaron; Linder, Suzanne K.; Braeuer, Russell; Ellis, Lee M.; Zwelling, Leonard (15 May 2013). Arakawa, Hirofumi. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/dx.plos.org\/10.1371\/journal.pone.0063221\" target=\"_blank\">\"A Survey on Data Reproducibility in Cancer Research Provides Insights into Our Limited Ability to Translate Findings from the Laboratory to the Clinic\"<\/a> (in en). <i>PLoS ONE<\/i> <b>8<\/b> (5): e63221. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1371%2Fjournal.pone.0063221\" target=\"_blank\">10.1371\/journal.pone.0063221<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1932-6203\" target=\"_blank\">1932-6203<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC3655010\/\" target=\"_blank\">PMC3655010<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/23691000\" target=\"_blank\">23691000<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/dx.plos.org\/10.1371\/journal.pone.0063221\" target=\"_blank\">https:\/\/dx.plos.org\/10.1371\/journal.pone.0063221<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Survey+on+Data+Reproducibility+in+Cancer+Research+Provides+Insights+into+Our+Limited+Ability+to+Translate+Findings+from+the+Laboratory+to+the+Clinic&rft.jtitle=PLoS+ONE&rft.aulast=Mobley&rft.aufirst=Aaron&rft.au=Mobley%2C%26%2332%3BAaron&rft.au=Linder%2C%26%2332%3BSuzanne+K.&rft.au=Braeuer%2C%26%2332%3BRussell&rft.au=Ellis%2C%26%2332%3BLee+M.&rft.au=Zwelling%2C%26%2332%3BLeonard&rft.date=15+May+2013&rft.volume=8&rft.issue=5&rft.pages=e63221&rft_id=info:doi\/10.1371%2Fjournal.pone.0063221&rft.issn=1932-6203&rft_id=info:pmc\/PMC3655010&rft_id=info:pmid\/23691000&rft_id=https%3A%2F%2Fdx.plos.org%2F10.1371%2Fjournal.pone.0063221&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-20\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-20\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/hudi.apache.org\/\" target=\"_blank\">\"Apache Hudi\"<\/a>. Apache Software Foundation. 2021<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/hudi.apache.org\/\" target=\"_blank\">https:\/\/hudi.apache.org\/<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 20 August 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Apache+Hudi&rft.atitle=&rft.date=2021&rft.pub=Apache+Software+Foundation&rft_id=https%3A%2F%2Fhudi.apache.org%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-21\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-21\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation book\">de Lucia, A.; Xhelo, E. (2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pure.rug.nl\/ws\/portalfiles\/portal\/124739867\/proceedings_2020.pdf\" target=\"_blank\">\"6. Data Science Pipeline Containerization\"<\/a>. In Smedinga, R.; Biehl, M.. <i>17th SC@RUG proceedings 2019-2020<\/i>. Bibliotheek der R.U.. pp. 39\u201344. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-94-034-2766-9<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pure.rug.nl\/ws\/portalfiles\/portal\/124739867\/proceedings_2020.pdf\" target=\"_blank\">https:\/\/pure.rug.nl\/ws\/portalfiles\/portal\/124739867\/proceedings_2020.pdf<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=6.+Data+Science+Pipeline+Containerization&rft.atitle=17th+SC%40RUG+proceedings+2019-2020&rft.aulast=de+Lucia%2C+A.%3B+Xhelo%2C+E.&rft.au=de+Lucia%2C+A.%3B+Xhelo%2C+E.&rft.date=2020&rft.pages=pp.%26nbsp%3B39%E2%80%9344&rft.pub=Bibliotheek+der+R.U.&rft.isbn=978-94-034-2766-9&rft_id=https%3A%2F%2Fpure.rug.nl%2Fws%2Fportalfiles%2Fportal%2F124739867%2Fproceedings_2020.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-22\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-22\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Zaharia, Matei; Xin, Reynold S.; Wendell, Patrick; Das, Tathagata; Armbrust, Michael; Dave, Ankur; Meng, Xiangrui; Rosen, Josh <i>et al.<\/i> (28 October 2016). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/dl.acm.org\/doi\/10.1145\/2934664\" target=\"_blank\">\"Apache Spark: a unified engine for big data processing\"<\/a> (in en). <i>Communications of the ACM<\/i> <b>59<\/b> (11): 56\u201365. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1145%2F2934664\" target=\"_blank\">10.1145\/2934664<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0001-0782\" target=\"_blank\">0001-0782<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/dl.acm.org\/doi\/10.1145\/2934664\" target=\"_blank\">https:\/\/dl.acm.org\/doi\/10.1145\/2934664<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Apache+Spark%3A+a+unified+engine+for+big+data+processing&rft.jtitle=Communications+of+the+ACM&rft.aulast=Zaharia&rft.aufirst=Matei&rft.au=Zaharia%2C%26%2332%3BMatei&rft.au=Xin%2C%26%2332%3BReynold+S.&rft.au=Wendell%2C%26%2332%3BPatrick&rft.au=Das%2C%26%2332%3BTathagata&rft.au=Armbrust%2C%26%2332%3BMichael&rft.au=Dave%2C%26%2332%3BAnkur&rft.au=Meng%2C%26%2332%3BXiangrui&rft.au=Rosen%2C%26%2332%3BJosh&rft.au=Venkataraman%2C%26%2332%3BShivaram&rft.date=28+October+2016&rft.volume=59&rft.issue=11&rft.pages=56%E2%80%9365&rft_id=info:doi\/10.1145%2F2934664&rft.issn=0001-0782&rft_id=https%3A%2F%2Fdl.acm.org%2Fdoi%2F10.1145%2F2934664&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:5-23\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:5_23-0\">23.0<\/a><\/sup> <sup><a href=\"#cite_ref-:5_23-1\">23.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ali, Rahman; Afzal, Muhammad; Hussain, Maqbool; Ali, Maqbool; Siddiqi, Muhammad Hameed; Lee, Sungyoung; Ho Kang, Byeong (1 February 2016). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0010482515003819\" target=\"_blank\">\"Multimodal hybrid reasoning methodology for personalized wellbeing services\"<\/a> (in en). <i>Computers in Biology and Medicine<\/i> <b>69<\/b>: 10\u201328. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.compbiomed.2015.11.013\" target=\"_blank\">10.1016\/j.compbiomed.2015.11.013<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0010482515003819\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0010482515003819<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Multimodal+hybrid+reasoning+methodology+for+personalized+wellbeing+services&rft.jtitle=Computers+in+Biology+and+Medicine&rft.aulast=Ali&rft.aufirst=Rahman&rft.au=Ali%2C%26%2332%3BRahman&rft.au=Afzal%2C%26%2332%3BMuhammad&rft.au=Hussain%2C%26%2332%3BMaqbool&rft.au=Ali%2C%26%2332%3BMaqbool&rft.au=Siddiqi%2C%26%2332%3BMuhammad+Hameed&rft.au=Lee%2C%26%2332%3BSungyoung&rft.au=Ho+Kang%2C%26%2332%3BByeong&rft.date=1+February+2016&rft.volume=69&rft.pages=10%E2%80%9328&rft_id=info:doi\/10.1016%2Fj.compbiomed.2015.11.013&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0010482515003819&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:6-24\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:6_24-0\">24.0<\/a><\/sup> <sup><a href=\"#cite_ref-:6_24-1\">24.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Lipscomb, C. E. (1 July 2000). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/10928714\" target=\"_blank\">\"Medical Subject Headings (MeSH)\"<\/a>. <i>Bulletin of the Medical Library Association<\/i> <b>88<\/b> (3): 265\u2013266. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0025-7338\" target=\"_blank\">0025-7338<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC35238\/\" target=\"_blank\">PMC35238<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/10928714\" target=\"_blank\">10928714<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/10928714\" target=\"_blank\">https:\/\/pubmed.ncbi.nlm.nih.gov\/10928714<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Medical+Subject+Headings+%28MeSH%29&rft.jtitle=Bulletin+of+the+Medical+Library+Association&rft.aulast=Lipscomb&rft.aufirst=C.+E.&rft.au=Lipscomb%2C%26%2332%3BC.+E.&rft.date=1+July+2000&rft.volume=88&rft.issue=3&rft.pages=265%E2%80%93266&rft.issn=0025-7338&rft_id=info:pmc\/PMC35238&rft_id=info:pmid\/10928714&rft_id=https%3A%2F%2Fpubmed.ncbi.nlm.nih.gov%2F10928714&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-25\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-25\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Murthy, Suntherasvaran; Abu Bakar, Asmidar; Abdul Rahim, Fiza; Ramli, Ramona (1 May 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/8819477\/\" target=\"_blank\">\"A Comparative Study of Data Anonymization Techniques\"<\/a>. <i>2019 IEEE 5th Intl Conference on Big Data Security on Cloud (BigDataSecurity), IEEE Intl Conference on High Performance and Smart Computing, (HPSC) and IEEE Intl Conference on Intelligent Data and Security (IDS)<\/i> (Washington, DC, USA: IEEE): 306\u2013309. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FBigDataSecurity-HPSC-IDS.2019.00063\" target=\"_blank\">10.1109\/BigDataSecurity-HPSC-IDS.2019.00063<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-7281-0006-7<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/8819477\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/8819477\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Comparative+Study+of+Data+Anonymization+Techniques&rft.jtitle=2019+IEEE+5th+Intl+Conference+on+Big+Data+Security+on+Cloud+%28BigDataSecurity%29%2C+IEEE+Intl+Conference+on+High+Performance+and+Smart+Computing%2C+%28HPSC%29+and+IEEE+Intl+Conference+on+Intelligent+Data+and+Security+%28IDS%29&rft.aulast=Murthy&rft.aufirst=Suntherasvaran&rft.au=Murthy%2C%26%2332%3BSuntherasvaran&rft.au=Abu+Bakar%2C%26%2332%3BAsmidar&rft.au=Abdul+Rahim%2C%26%2332%3BFiza&rft.au=Ramli%2C%26%2332%3BRamona&rft.date=1+May+2019&rft.pages=306%E2%80%93309&rft.place=Washington%2C+DC%2C+USA&rft.pub=IEEE&rft_id=info:doi\/10.1109%2FBigDataSecurity-HPSC-IDS.2019.00063&rft.isbn=978-1-7281-0006-7&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F8819477%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-26\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-26\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation book\">Kreps, J.; Narkhede, N.; Rao, J. (2011). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/cs.uwaterloo.ca\/~ssalihog\/courses\/papers\/netdb11-final12.pdf\" target=\"_blank\">\"Kafka: A Distributed Messaging System for Log Processing\"<\/a>. <i>Proceedings of the NetDB<\/i>. <b>11<\/b>. pp. 1\u20137. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-4503-0652-2\/11\/06<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/cs.uwaterloo.ca\/~ssalihog\/courses\/papers\/netdb11-final12.pdf\" target=\"_blank\">https:\/\/cs.uwaterloo.ca\/~ssalihog\/courses\/papers\/netdb11-final12.pdf<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Kafka%3A+A+Distributed+Messaging+System+for+Log+Processing&rft.atitle=Proceedings+of+the+NetDB&rft.aulast=Kreps%2C+J.%3B+Narkhede%2C+N.%3B+Rao%2C+J.&rft.au=Kreps%2C+J.%3B+Narkhede%2C+N.%3B+Rao%2C+J.&rft.date=2011&rft.volume=11&rft.pages=pp.%26nbsp%3B1%E2%80%937&rft.isbn=978-1-4503-0652-2%2F11%2F06&rft_id=https%3A%2F%2Fcs.uwaterloo.ca%2F%7Essalihog%2Fcourses%2Fpapers%2Fnetdb11-final12.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-27\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-27\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation\" id=\"rdp-ebb-CITEREFDavis2019\">Davis, Adam L. (2019), <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/978-1-4842-4176-9_6\" target=\"_blank\">\"Akka Streams\"<\/a> (in en), <i>Reactive Streams in Java<\/i> (Berkeley, CA: Apress): 57\u201370, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2F978-1-4842-4176-9_6\" target=\"_blank\">10.1007\/978-1-4842-4176-9_6<\/a>, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-4842-4175-2<span class=\"printonly\">, <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/978-1-4842-4176-9_6\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/978-1-4842-4176-9_6<\/a><\/span><\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Akka+Streams&rft.jtitle=Reactive+Streams+in+Java&rft.aulast=Davis&rft.aufirst=Adam+L.&rft.au=Davis%2C%26%2332%3BAdam+L.&rft.date=2019&rft.pages=57%E2%80%9370&rft.place=Berkeley%2C+CA&rft.pub=Apress&rft_id=info:doi\/10.1007%2F978-1-4842-4176-9_6&rft.isbn=978-1-4842-4175-2&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2F978-1-4842-4176-9_6&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:7-28\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:7_28-0\">28.0<\/a><\/sup> <sup><a href=\"#cite_ref-:7_28-1\">28.1<\/a><\/sup> <sup><a href=\"#cite_ref-:7_28-2\">28.2<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Mamounas, Eleftherios P. (1 October 2003). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.clinmedres.org\/content\/1\/4\/309\" target=\"_blank\">\"NSABP Breast Cancer Clinical Trials: Recent Results and Future Directions\"<\/a> (in en). <i>Clinical Medicine & Research<\/i> <b>1<\/b> (4): 309\u2013326. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3121%2Fcmr.1.4.309\" target=\"_blank\">10.3121\/cmr.1.4.309<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1539-4182\" target=\"_blank\">1539-4182<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC1069061\/\" target=\"_blank\">PMC1069061<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/15931325\" target=\"_blank\">15931325<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.clinmedres.org\/content\/1\/4\/309\" target=\"_blank\">http:\/\/www.clinmedres.org\/content\/1\/4\/309<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=NSABP+Breast+Cancer+Clinical+Trials%3A+Recent+Results+and+Future+Directions&rft.jtitle=Clinical+Medicine+%26+Research&rft.aulast=Mamounas&rft.aufirst=Eleftherios+P.&rft.au=Mamounas%2C%26%2332%3BEleftherios+P.&rft.date=1+October+2003&rft.volume=1&rft.issue=4&rft.pages=309%E2%80%93326&rft_id=info:doi\/10.3121%2Fcmr.1.4.309&rft.issn=1539-4182&rft_id=info:pmc\/PMC1069061&rft_id=info:pmid\/15931325&rft_id=http%3A%2F%2Fwww.clinmedres.org%2Fcontent%2F1%2F4%2F309&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-29\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-29\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.jci.org\/\" target=\"_blank\">\"The Journal of Clinical Investigation\"<\/a>. American Society for Clinical Investigation. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1558-8238\" target=\"_blank\">1558-8238<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.jci.org\/\" target=\"_blank\">https:\/\/www.jci.org\/<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 28 May 2023<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=The+Journal+of+Clinical+Investigation&rft.atitle=&rft.pub=American+Society+for+Clinical+Investigation&rft.issn=1558-8238&rft_id=https%3A%2F%2Fwww.jci.org%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-30\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-30\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/journals.sagepub.com\/home\/ctj\" target=\"_blank\">\"Clinical Trials\"<\/a>. The Society for Clinical Trials. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1740-7753\" target=\"_blank\">1740-7753<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/journals.sagepub.com\/home\/ctj\" target=\"_blank\">https:\/\/journals.sagepub.com\/home\/ctj<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 28 May 2023<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Clinical+Trials&rft.atitle=&rft.pub=The+Society+for+Clinical+Trials&rft.issn=1740-7753&rft_id=https%3A%2F%2Fjournals.sagepub.com%2Fhome%2Fctj&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-31\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-31\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Platto, Sara; Xue, Tongtong; Carafoli, Ernesto (24 September 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41419-020-02995-9\" target=\"_blank\">\"COVID19: an announced pandemic\"<\/a> (in en). <i>Cell Death & Disease<\/i> <b>11<\/b> (9): 799. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41419-020-02995-9\" target=\"_blank\">10.1038\/s41419-020-02995-9<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2041-4889\" target=\"_blank\">2041-4889<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC7513903\/\" target=\"_blank\">PMC7513903<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/32973152\" target=\"_blank\">32973152<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41419-020-02995-9\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41419-020-02995-9<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=COVID19%3A+an+announced+pandemic&rft.jtitle=Cell+Death+%26+Disease&rft.aulast=Platto&rft.aufirst=Sara&rft.au=Platto%2C%26%2332%3BSara&rft.au=Xue%2C%26%2332%3BTongtong&rft.au=Carafoli%2C%26%2332%3BErnesto&rft.date=24+September+2020&rft.volume=11&rft.issue=9&rft.pages=799&rft_id=info:doi\/10.1038%2Fs41419-020-02995-9&rft.issn=2041-4889&rft_id=info:pmc\/PMC7513903&rft_id=info:pmid\/32973152&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41419-020-02995-9&rfr_id=info:sid\/en.wikipedia.org:Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<\/ol><\/div><\/div>\n<h2><span class=\"mw-headline\" id=\"Notes\">Notes<\/span><\/h2>\n<p>This presentation is faithful to the original, with only a few minor changes to presentation, though grammar and word usage was substantially updated for improved readability. In some cases important information was missing from the references, and that information was added.\n<\/p>\n<!-- \nNewPP limit report\nCached time: 20231215024353\nCache expiry: 86400\nDynamic content: false\nComplications: []\nCPU time usage: 0.846 seconds\nReal time usage: 1.151 seconds\nPreprocessor visited node count: 30701\/1000000\nPost\u2010expand include size: 258186\/2097152 bytes\nTemplate argument size: 75674\/2097152 bytes\nHighest expansion depth: 25\/40\nExpensive parser function count: 0\/100\nUnstrip recursion depth: 0\/20\nUnstrip post\u2010expand size: 72665\/5000000 bytes\n-->\n<!--\nTransclusion expansion time report (%,ms,calls,template)\n100.00% 635.836 1 -total\n 85.54% 543.866 1 Template:Reflist\n 63.74% 405.276 31 Template:Citation\/core\n 43.06% 273.815 19 Template:Cite_journal\n 15.65% 99.478 4 Template:Cite_book\n 9.23% 58.707 1 Template:Infobox_journal_article\n 8.57% 54.483 23 Template:Date\n 8.37% 53.244 60 Template:Citation\/identifier\n 8.34% 53.042 6 Template:Cite_web\n 7.50% 47.699 1 Template:Infobox\n-->\n\n<!-- Saved in parser cache with key limswiki:pcache:idhash:14283-0!canonical and timestamp 20231215024351 and revision id 52592. Serialized with JSON.\n -->\n<\/div><\/div><div class=\"printfooter\">Source: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.limswiki.org\/index.php\/Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis\">https:\/\/www.limswiki.org\/index.php\/Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis<\/a><\/div>\n<!-- end content --><div class=\"visualClear\"><\/div><\/div><\/div><div class=\"visualClear\"><\/div><\/div><!-- end of the left (by default at least) column --><div class=\"visualClear\"><\/div><\/div>\n\n\n\n<\/body>","affac9ff82db9d386600be5eb3d77056_images":["https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/2\/27\/Fig1_Siddiqi_Healthcare23_11-12.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/2\/2b\/Fig2_Siddiqi_Healthcare23_11-12.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/9\/91\/Fig3_Siddiqi_Healthcare23_11-12.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/4\/47\/Fig4_Siddiqi_Healthcare23_11-12.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/3\/3d\/Fig5_Siddiqi_Healthcare23_11-12.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/b\/b4\/Fig6_Siddiqi_Healthcare23_11-12.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/1\/19\/Fig7_Siddiqi_Healthcare23_11-12.png"],"affac9ff82db9d386600be5eb3d77056_timestamp":1702682171,"3a2816a67d7d45f854c1e2fb9ec00f31_type":"article","3a2816a67d7d45f854c1e2fb9ec00f31_title":"Guideline for software life cycle in health informatics (Hauschild et al. 2022)","3a2816a67d7d45f854c1e2fb9ec00f31_url":"https:\/\/www.limswiki.org\/index.php\/Journal:Guideline_for_software_life_cycle_in_health_informatics","3a2816a67d7d45f854c1e2fb9ec00f31_plaintext":"\n\nJournal:Guideline for software life cycle in health informaticsFrom LIMSWikiJump to navigationJump to searchFull article title\n \nGuideline for software life cycle in health informaticsJournal\n \niScienceAuthor(s)\n \nHauschild, Anne-Christin; Martin, Roman; Holst, Sabrina C.; Wienbeck, Joachim; Heider, DominikAuthor affiliation(s)\n \nPhilipps University of Marburg, University Medical Center G\u00f6ttingenPrimary contact\n \nEmail: dominik dot heider at uni-marburg dot deYear published\n \n2022Volume and issue\n \n25(12)Article #\n \n105534DOI\n \n10.1016\/j.isci.2022.105534ISSN\n \n2589-0042Distribution license\n \nCreative Commons Attribution 4.0 InternationalWebsite\n \nhttps:\/\/www.sciencedirect.com\/science\/article\/pii\/S2589004222018065Download\n \nhttps:\/\/www.sciencedirect.com\/science\/article\/pii\/S2589004222018065\/pdfft (PDF)\n\nContents \n\n1 Abstract \n2 Introduction \n\n2.1 Challenges of scientific software development for health care \n2.2 Software life cycle \n2.3 Our goal: An academia-tailored software life cycle \n\n\n3 Software life cycle for medical software research guideline \n\n3.1 Software development: Planning \n3.2 Software development: Requirements analysis \n3.3 Software development: Architecture and design \n3.4 Software development: Implementation, testing, and verification \n3.5 Software development: Release \n3.6 Legacy software \n3.7 Configuration and change management \n\n\n4 Discussion \n5 Supplemental information \n6 Abbreviations, acronyms, and initialisms \n7 Acknowledgements \n\n7.1 Author contributions \n7.2 Conflict of interest \n\n\n8 References \n9 Notes \n\n\n\nAbstract \nThe long-lasting trend of medical informatics is to adapt novel technologies in the medical context. In particular, incorporating artificial intelligence (AI) to support clinical decision-making can significantly improve monitoring, diagnostics, and prognostics for the patient\u2019s and medic\u2019s sake. However, obstacles hinder a timely technology transfer from the medical research setting to the actual clinical setting. Due to the pressure for novelty in the research context, projects rarely implement quality standards.\nHere, we propose a guideline for academic software life cycle (SLC) processes tailored to the needs and capabilities of research organizations. While the complete implementation of an SLC according to commercial industry standards is largely not feasible in scientific research work, we propose a subset of elements that we are convinced will provide a significant benefit to research settings while keeping the development effort within a feasible range.\nUltimately, the emerging quality checks for academic research software development can pave the way for a more accelerated deployment of academic software into clinical practice.\nKeywords: health informatics, bioinformatics, software engineering\nGraphical abstract: \n\n\nIntroduction \nToday, medical informatics is an integral part of health care systems that ensure the smooth operation of processes in medical care. Moreover, standard procedures ensure the transfer of knowledge from medical research to clinical practice, for instance, via regularly updated guidelines and regulations. In contrast, newly developed software innovations\u2014such as systems based on artificial intelligence (AI) that could support clinical decisions and have already evolved to be the state-of-the-art in medical informatics research\u2014rarely transfer to application in practice.\nModern methods such as AI and machine learning (ML) increasingly unroll their potential in medical healthcare to help patients and clinicians.[1] Clinical decision support systems (CDSSs) can effectively increase diagnostics, patient safety, and cost containment.[2] Easily accessible AI-based applications can improve diagnosis and treatment of patients, e.g., by precisely detecting symptoms[3], evaluating biomarkers[4], or detecting pathogenic resistance or subtypes.[5][6] Furthermore, upcoming concepts such as federated learning[7][8] and swarm learning[9] allow the cross-clinical creation of data-driven models without disrupting patient\u2019s privacy[10][11], opening the gate for more powerful data-driven development.\nHowever, software has its risks, especially within medical devices. To protect patients from any risk of injury, disability, or other harmful interventions, medical device software (MDSW)\u2014which is intended to provide specific medical purposes, such as diagnosis, monitoring, prognosis, or treatment\u2014are subject to strict regulations. Examples include the European Medical Devices Regulation (MDR)[12], the In Vitro Diagnostic Medical Devices Regulation (IVDR)[13], and the International Medical Device Regulators Forum (IMDRF).[14]\nMDSW can be an integral part of a medical product or standalone software as an independent medical device.[15] MDSW can run in the cloud, as part of a software platform, or on a server, with both healthcare professionals and laypersons using it. Exceptions are software tools used for documentation or that solely control medical device hardware, or that serve no medical purpose.[13]\nAn integral part of all regulations for MDSW is development according to the software life cycle process, as defined by IEC 62304[16], a harmonized international standard that regulates MDSW life cycle development, requiring documentation and processes, such as software development planning, requirement analysis, architectural design, testing, verification, and maintenance.[12][17]\nThese regulations focus on minimizing patient risk, for example, harmful follow-up analysis, a wrong or missing treatment where needed, as a result of software failures, incorrect predictions, or other malfunctions. Several challenges arise in the attempt to eliminate these risks and allow for a smoother transfer of technology and greater reproducibility.\n\nChallenges of scientific software development for health care \nThe primary goal of scientists remains conducting scientific activities rather than developing software. However, many scientists aim to make their findings and methodologies available to a broader audience and to be used for the greater good. Thus, many data science and AI methodologies, as well as corresponding implementations and software packages, exist that would, in theory, allow the development of efficacious AI-driven CDSSs and MDSW. However, scientists of different backgrounds tend to have very different knowledge of software engineering practices, often acquired through self-study.[18] Moreover, academic groups often consist of small teams that undergo frequent change or researchers that work on a \"one person-one project\" basis.[19][20] Thus, a lack of attention to relevant software development processes and engineering practices defined by the software life cycle negatively affects the usefulness of developed packages, particularly for developing software as a medical device.[21]\nPinning down the most critical requirements along with an accurate description and documentation of such is a significant challenge for all software projects independent of if it is conducted in research or industry.[18][22] It necessitates a detailed analysis of the non-functional requirements as determined, for instance, by regulatory entities, addressing aspects such as security, privacy, or infrastructural limitations, as well as functional requirements like user-friendly interfaces and specific results. This is very time-consuming and relies on a close interaction of developers, stakeholders, and potential users, which is often difficult to achieve under academic circumstances.[20][22] Moreover, researchers are enticed by academic hiring procedures and driven by funders to focus on \u201cnovelty\u201d rather than software quality and practical usefulness.[19][20] Thus, implementations often fail to fulfill requirements, ensuring long-term sustainability such as documentation, usability, appropriate performance for practical application, user-friendliness optimally supporting potential users, and minimizing risks.[19][23]\nThe most critical aspects of ensuring sustainability in academic software are reproducibility, reusability, and traceability.[21][24] However, it has been shown that not only public accessibility but also documentation and portability are essential to ensure reproducibility and underpin trust in the scientific record of scientific software, enabling the re-use of research and code. Moreover, the prototype-centered development procedures often lack quality checks, such as systematic testing, that would ensure reusability.[21] Recently, scientific journals such as GigaScience or Biostatistics have promoted reproducibility and reusability by mandating the FAIR Principles (i.e., findability, accessibility, interoperability, and reusability). FAIR establishes a guideline for scientific data management and documentation.[19][25] Implementing the FAIR principles in academic software development has the potentil to lower the barriers to a successful industrial transition.\nAdditionally, for long-term software maintenance, well-structured development planning and processes can ensure the traceability of modifications via change management and version control. These aspects ultimately determine scientific rigor, transparency, and reproducibility.[19]\n\nSoftware life cycle \nProper implementation of the software life cycle (SLC) guarantees high-quality planning, development, and maintenance of MDSW. The SLC deals with the planning and specification, development, maintenance, and configuration of the software. IEC 62304 defines the SLC as a conceptual structure across its lifetime, from the requirements\u2019 definition to final release. It describes processes, tasks, and activities involved in developing a software product and their order and interdependencies. Furthermore, it defines milestones verifying the completeness of the results to be delivered.[16]\nHowever, a complete SLC described in standards like IEC 62304 is not feasible for most research projects.[26] Academic research is often subject to tight schedules and focuses on proof-of-concept development, neglecting formal documentation or procedures. Here we provide recommendations for an SLC in academia, lowering the boundary for many research organizations to implement an SLC and fostering the transfer of technology to industrial development.[13][26]\n\nOur goal: An academia-tailored software life cycle \nUntil now, there has been little guidance on supporting a structured software development culture for academic institutions according to standard SLC processes. In this article, we present a synopsis of all requirements in official standards that are relevant to academia. We adjusted these toward the specific demands of software development in research and established a limited SLC process for research organizations, which has the potential to greatly facilitate and speed up such technology transfer and reproducibility in a controlled and predictable way. Being aware that a complete SLC is not feasible for most academic settings, we proposed a subset of elements that we are convinced will provide a significant benefit without creating an excessive organizational burden for researchers and developers and keep activities in a manageable range.\nOur proposal is centered on procedures for software development planning, software requirement analysis, software architectural design, software unit implementation, integration, testing, verification, and configuration management. Depending on the specific needs, the elements of an SLC process that work best for an organization may differ from what we propose. The fact, however, that a life cycle process is set up at all and that the elements are deliberately chosen is probably a key factor for facilitating technology transfer. However, any medical software development for clinical use must strictly follow the regulations relevant to the specific country or region. Thus, our guideline can only provide a starting point intended to be adapted to institute- or project-specific requirements, considering only relevant aspects. Nevertheless, the ideas presented here are not meant to provide a shortcut for medical software. An overview of our suggested SLC activities in tandem with regulations is provided in Table S1 of the Supplemental information.\n\nSoftware life cycle for medical software research guideline \nOur guideline covers multiple processes such as development planning, requirement analysis, software architecture, software design, implementation, software, and integration testing, verification, and release. In the following, we present the most vital points for the SLC, as depicted in Figure 1.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 1. Main components of the software life cycle (SLC) for research-based medical software.\n\n\n\nThe main focus of the SLC, in compliance with the quality management system (QMS), lies in the software development arena, while also tapping into configuration and change management, as well as legacy software.\n\nSoftware development: Planning \nDefining an accurate software development plan is the first key component and must be updated regularly during the project, getting referenced and refined throughout the entire software life cycle model. It defines norms, methods, used processes, deliverable results, traceability between requirements, software testing, implemented risk control measures, configuration and change management, and verification of configuration elements, including software of unknown provenance (SOUP). (SOUP is a commonly used basic software library or set of packages that have not been developed for medical purposes.)\nTwo documents should be provided for the academic-tailored implementation: the process description and the development plan. Each document produced during software development has to contain a title, purpose, and responsible person.[16] In the case of multiple involved developers, the role and responsibility assignments should be noted down in the single process description.\nFirst, the process description is provided by the definition of standard operating procedures (SOPs) for repetitive application problems.[27] The description contains each activity within the processes, when it will be completed, by whom, how, and with which input and output. The developed process description can be applied to multiple projects and has to be implemented by the developers.[12] Since the life cycle model must be completely defined or referenced by the development plan, a software engineering model must be selected to provide a general structure for the development phase, such as the V-model. As a common approach in medical device development, the V-model is successfully used to achieve regulatory compliance.[28] Generally, the selected model should match the project\u2019s characteristics and thus can host agile practices or elements of SCRUM to support and conform to life cycle development practices.[29][30]\nSecond, the development plan describes the general documentation required for a product or project. The development plan defines concrete milestones to corresponding deadlines, assigns designated staff to the pre-defined roles, and refines measures or tools adjusted to the project. It is challenging to meet the regulatory requirement of defining tools, testing, and configuration management in advance, particularly in a volatile academic setting. Due to external factors and unforeseeable changes, the development plan must be updated over the lifetime of the project, while the pre-defined process descriptions remain. Additionally, the process should recommend defining a coding guideline or convention, including code style, nomenclature, and naming in the development plan, to increase software quality. For example, using pre-defined rules in git hooks or continuous integration (CI), combined with linting tools, can enforce coding compliance. \nThe development process gets more transparent and well-structured through the provision of these two documents.\n\nSoftware development: Requirements analysis \nA software requirement is a detailed statement about a property that a software product, system, or process should fulfill and is defined during software requirements analysis. Software requirements should cover functional requirements such as inputs, outputs, functions, processes, interface reactions, and thresholds, as well as non-functional requirements such as physical characteristics, computing environments, performance, cybersecurity, privacy, maintenance, installation, networking, and so forth. Precisely defined requirements are a vital element for the success of a project, particularly given that studies have concluded that half of all software errors are derived from mistakes in the requirement phase.[31][32] High-level requirements should be defined within the specifications, including the desired properties. These specifications describe mainly the project\u2019s total goal under defined restrictions. For practical consideration, it is beneficial to begin with general natural language requirements and refine them into graphical notations, such as Unified Model Language (UML) diagrams.[33][34]\nHowever, the original requirement within the specification must always be bidirectionally linked to allow traceability. It should be possible to describe and follow the life of requirements in all directions. This means that traceability implies the comprehension of a design, starting with the source of a requirement, its implementation, testing, and maintenance. Moreover, it facilitates a high level of software quality, a critical concern for medical devices. Therefore, the derivation and documentation of the requirements should be updated and verified during the project.[16] Finally, a critical aspect is the definition of requirements for maintenance, which is often neglected in academia since the focus is on publishing new technologies in contrast to maintaining existing software.\nNevertheless, maintenance must be considered at the beginning of the software life cycle to ensure that post-delivery support is possible.[35] Therefore, a maintenance plan in academia does not have to be complete but must define all factors influencing either the development or architecture. Table 1, which examines ISO\/IEC 25010[36], can be used to evaluate the completeness of a software requirement analysis. Further, an implementation example is provided in the Supplemental information.\n\n\n\n\n\n\n\nTable 1. The product quality properties of ISO\/IEC 25010, used to evaluate if software requirements are complete.\n\n\nAspect\n\nProperties\n\n\nUsability\n\n- Recognizable appropriateness\r\n- Learnability\r\n- Operability\r\n- User error protection\r\n- Accessibility\n\n\nFunctional suitability\n\n- Functional completeness\r\n- Functional correctness\r\n- Functional appropriateness\n\n\nSecurity\n\n- Confidentiality\r\n- Integrity\r\n- Non-repudiation\r\n- Accountability\r\n- Authenticity\n\n\nMaintainability\n\n- Modularity\r\n- Reusability\r\n- Analyzability\r\n- Modifiability\r\n- Testability\n\n\nPerformance efficiency\n\n- Time behavior\r\n- Resource utilization\r\n- Capacity\n\n\nReliability\n\n- Maturity\r\n- Availability\r\n- Fault tolerance\r\n- Recoverability\n\n\nPortability\n\n- Adaptability\r\n- Installability\r\n- Replaceability\n\n\nCompatibility\n\n- Co-existence\r\n- Interoperability\n\n\n\nSoftware development: Architecture and design \nThe regulatory authorities demand the definition of essential structural software components, identification of their primary responsibilities, visible features, and their interrelations.[16] The architecture as an overarching structure conceptually defines data storage, interfaces, and logical servers. At the same time, modularization is described within the detailed software design, specifying how the single elements of the architecture and the requirements are explicitly implemented.\nSoftware architecture consists of the system\u2019s structure in combination with architecture characteristics the system must support (e.g., availability, scalability, and security), architecture decisions (formulation of rules and constraints), and design principles. An architecture categorizes into monolithic and distributed architecture types consisting of single packages or separable sub-systems.[37] It is recommended to specify an appropriate architecture prior to implementation. However, the choice is not regulated.[16] These architectural design decisions will guide the developers throughout the development process.\nThe system has to be divided for the software design until it is represented through software units. These are sets of procedures or functions encapsulated in a package or class that cannot be further divided. Each software unit and interface needs a verified detailed design to ensure correct implementation.[16] The design principles cover every option or state of all system components in detail, such as a preferred method or protocol.[38] In order to have well testable and maintainable code, it is recommended to have software with low coupling (dependencies between the sub-systems) and high cohesion (internal dependencies).[39] These associations can be well described using widely accepted notation standards such as UML, including class and activity diagrams to document the architectural decisions, which is highly recommendable to facilitate understanding the architecture.[40]\nIn academia, two documents should be provided: the software architecture description and the detailed design. It is unlikely to narrow down the whole codebase in a detailed design. However, it must contain the utmost vital components, such as elements of design patterns, classes with crucial functionality, or specific interfaces. For example, a strict logical dissociation between the internal logic and the interface, e.g., for user interaction, is critical. Design patterns such as the model view controller (MVC) are favorable.[41]\n\n Software development: Implementation, testing, and verification \nGenerally, each software unit must be implemented, tested, and verified. The IEEE defines implementation as translating a design into hardware or software components, or both.[42] In particular, the detailed design has to be translated into source code. Following a specific coding style and documentation standards is advisable during the implementation. Subsequently, every software unit has to be tested and verified separately, ensuring it works as specified in the detailed design and complies with the coding style. After that, it has to be integrated, verified, and tested dependently and independently in the following integration tests. These evaluate the software unit\u2019s functionality combined with other components into an overall system.\nThe software is usually tested on different abstraction levels within the software\u2019s life cycle, differentiating between unit, integration, regression, and system testing. While isolated unit tests verify the functionality of a separately testable software element, integration tests verify the interaction between the software units, as described by the software architecture. Along with different test strategies, such as top-down or bottom-up, integration tests must be conducted during several stages of the development process and are tailored to each integration level.[43]\nIntegration and system tests can be combined with routine activities but must cover all software requirements. Especially, software components affecting safety require extensive tests. Appropriate evaluations of the testing procedure, verification, and integration strategy concerning the previously determined requirements are necessary. Tests and results must be recorded with acceptance criteria, providing repeatability and traceability between requirements and their verifications.\nTests can be performed either as white-box testing[44], including the knowledge of the underlying architecture, or as black-box testing[45], which does not take into account the internal structure. Besides automated tests, non-automated tests should be conducted between program coding and the beginning of computer-based testing. Moreover, the three fundamental human testing methods are inspections, walkthroughs, and usability testing.[46] As demonstrated in our example in the supplemental information, SCRUM supports software integration and system testing since, after each sprint, an increment of potentially shippable functionality consisting of tested, well-written, and executable code is required (Supplemental information, Figure S2). Consequently, verification and testing are automatically included in the process of SCRUM. Project-specific regular, complete tests which are documented and traceable to requirements, software architecture, and detailed design are critical for software development within the law. A test is successful if it passes the acceptance criteria, defined through the requirements specification, the interface design within the detailed design, and the coding guideline. Ultimately, the verification evaluates whether all specified requirements are fulfilled by validating objective proof.[16]\nTo ensure adequate software verification, it has to be well-planned and integrated into several stages of the SLC: requirements analysis, software architecture, software design, and software units, as well as their integration, changes, and problem resolutions, have to be verified. The management of verification documents can also be partially organized automatically through CI or as such with the Jira API or the Gitlab CI\/CD. Table 2 lists the suggested aspects to verify the different stages of the software development process. In particular, problem resolutions and other changes have to be re-verified and documented. Overall, verification is an activity of high importance throughout the whole development process. To verify more mature artifacts, one must verify their foundation as well. This hierarchy should always be kept in mind, as the Supplemental information example demonstrates.\n\n\n\n\n\n\n\nTable 2. Verification at all software development process stages.\n\n\nDevelopment aspect\n\nRequirement(s)\n\n\nSystem requirements\n\n- Must be derived from the stakeholder\u2019s requirements\r\n- May not contradict each other\r\n- Must be consistent, unambiguous, clearly identifiable\r\n\n\n\nSoftware requirements\n\n- Implement the system requirements\r\n- May not contradict each other\r\n- Must be consistent, unambiguous, clearly identifiable\r\n- Must be traceable to the system requirements or other sources\r\n- Testing criteria must be drivable\n\n\nSoftware architecture\n\n- All System and software requirements are implemented\r\n- Must support the interfaces as well as SOUP items\n\n\nDetailed design\n\n- Implements do not contradict the software architecture\n\n\nSoftware units\n\n- Test case for each requirement must be passed\r\n- Code may not contradict the interface design, the detailed design, or the coding guideline\r\n- Verification of all requirements, architecture, and detailed design must be documented\n\n\nSoftware integration\n\n- Software unit integration is realized according to an integration plan derived from the software architecture\r\n- Software system tests verify the software\u2019s functionality\n\n\n\nSoftware development: Release \nIn contrast to industry, academic software is released to other researchers via public repositories and journal publications. Before software release, testing and verification need to be completed and evaluated. That includes, first, all known residual anomalies that have to be documented and evaluated. Second, documentation of the release procedure and the software development environment has to be recorded with the released software version. Third, all activities and tasks of the software development plan must be completed and documented. Fourth, the medical device software, all configuration elements, and the documentation must be filled for the whole lifetime of the medical device software, defined by the development team as long as the relevant regulatory requirements demand it. Fifth, procedures to ensure a reliable delivery without damaging or unauthorized adjustments have to be defined.[16]\nAfter the software is released, all changes and updates are implemented within the software maintenance process, following the same steps as the software development process. The post-delivery maintenance decisions that must be made are included in software development planning.\nSuppose the development process is well-defined and followed, and the previous sections of this guideline are considered. In that case, the complete verification and the required documents are delivered by default. Well-implemented traceability is essential to ensure the development process can be archived transparently. Regarding SCRUM, one could include the required documentation within the definition of \"done\" to ensure everything is documented since the development takes place in a regulatory context.\n\nLegacy software \nAccording to IEC 62304, legacy software is defined as software that was not developed to be used within software as a medical device, such as general software packages and libraries. It, therefore, lacks sufficient verification that it was developed in compliance with the current version of the norm.\nThus, it is sufficient to prove it conforms to the norm, and shortcomings to the norm\u2019s requirements need assessment.[16] Hence risks of using the legacy software, as well as the risks of missing documentation, need to be identified and mitigated, if possible, as defined by the risk management process of the IEC 62366-1 standard.[47]\nIn academia, it is essential to choose legacy software and document its usage carefully. Ideally, the used legal software has to fulfill the requirements of the IEC concerning risks as well, but the scope of action only demands closing gaps if it reduces the risk of usage.\n\nConfiguration and change management \nConfiguration and change management is crucial in ensuring usability, reproducibility, reusability, and traceability of software in the industry and academia. In academic research, automated change management systems, such as GitHub or GitLab, exist for software code and data and are regularly used.[25] However, implementing adequate configuration and change management documentation for the entire SLC, as required by the IEC 62304 standard, is particularly challenging in academia, where the pressure to publish urges researchers to focus on novelty rather than maintenance.\nIn order to mitigate the ongoing replication crisis[48][49], academic research institutions and projects should establish technical and administrative procedures to identify and define configuration items and SOUP, as well as their documentation within a system. This should include the documentation of problem reports, change requests, changes, and releases necessary to restore an item, determine its components, and provide the history of its changes. In particular, configuration change requests need to be documented, approved, and verified in projects with multiple developers and stakeholders.[16] Thus, it is advised to assign a representative person, ideally permanent technical academic staff, in charge of the change management to support the correct implementation of necessary processes for groups and projects in advance.[23] In academia, ticket systems such as those provided by most repositories can be easily used as version control systems for all code and documents to facilitate tracking changes. Moreover, tools such as Jira for project management and Confluence for project documentation are advisable to ensure traceability and good configuration management.\nThese tools can be utilized to establish a change management strategy or process that includes steps like:\n\nCreate a problem report (including criticality).\nConduct problem analysis, including software risk.\nCreate a change request, if required.\nImplement and verify the change.\nDiscussion \nThe current trend toward using new technologies in the medical context[1], such as establishing AI- or ML-related software, paves the way to significantly improve monitoring, diagnostics, and prognostics for the patient\u2019s and medical team\u2019s sakes. The subjacent development, realized mainly by specialized research organizations and institutes, is time-consuming. Additionally, implementing these software achievements in the medical markets requires enormous efforts to cover all requirements of international standards.[19][20][21] This article reveals several knowledge transfer challenges from industrial standards to use in academia, focusing on the software life cycle in a biomedical context. Challenges mainly face the software development and engineering processes regarding reproducibility, reusability, and traceability.\nIn order to better establish an academia-tailored SLC, here we propose a comprehensive guideline for research facilities derived from the requirements of IEC 62304.[16] Complementary to our quality management guideline[50], we propose to address these challenges by following our guideline, which lowers the barriers to a potential technology transfer toward the medical industry. Furthermore, in the Supplemental information, we provide a comprehensive checklist for a successful SLC and demonstrate the feasibility of our guideline with our implementation example.\nSince realizing industry-based regulatory requirements is mostly not feasible in an academic context, we focus on the most vital aspects of the SLC, covering software planning, development, architecture, maintenance, and legacy software. The implementation of our guidelines will not only improve the quality of medical software and avoid engineering errors but also mitigate potential risks that might arise from the introduction of AI in healthcare. An integration of SLC as standard procedure in academic programming could increase the overall quality of processing pipelines and thus the quality of data. Since evaluation strategies are well planned, this lends to a closer examination of the potential risks of false positives and false negatives. Ultimately, this supports a smooth transfer with potential manufacturers by delivering all demanded documents of a certain quality related to the software. Although some research organizations indeed have the resources to realize industrial standards such as quality management[51], we encourage scientists to further introduce SLC as a usual practice for software in research institutes. We envision that such a focus on SLC, in addition to focus on the FAIR principles for scientific data management and documentation, could become the standard for scientific health software publishing. Finally, this may pave the way for a smoother transition from research toward clinical practice.\n\nSupplemental information \nDocument S1 (PDF): Figures S1\u2013S8 and Tables S1\u2013S7.\n Abbreviations, acronyms, and initialisms \nAI: artificial intelligence\nCDSS: clinical decision support system\nCI: continuous integration\nFAIR: findable, accessible, interoperable, and reusable\nIMDRF: International Medical Device Regulators Forum\nIVDR: In Vitro Diagnostic Medical Devices Regulation\nMDR: European Medical Devices Regulation\nMDSW: medical device software\nML: machine learning\nMVC: model view controller\nQMS: quality management system\nSLC: software life cycle\nSOP: standard operating procedure\nSOUP: software of unknown provenance\nAcknowledgements \nThis project has received funding from the European Union\u2019s Horizon2020 research and innovation program under grant agreement No 826078. This publication reflects only the authors\u2019 view and the European Commission is not responsible for any use that may be made of the information it contains.\n\nAuthor contributions \nConceptualization, A.-C.H., R.M., S.C.H., J.W. and D.H.; Methodology, A.-C.H., R.M., S.C.H. and J. W.; Software, A.-C.H., S.C.H. and J.W.; Investigation, A.-C.H., R.M., S.C.H. and J.W.; Writing - Original Draft, A.-C.H., R.M., S.C.H. and J.W.; Writing - Review & Editing: A.-C.H., R.M., S.C.H., J.W. and D.H.; Supervision, D.H.\n\nConflict of interest \nThe authors declare no competing interests.\n\nReferences \n\n\n\u2191 1.0 1.1 Muehlematter, Urs J; Daniore, Paola; Vokinger, Kerstin N (1 March 2021). \"Approval of artificial intelligence and machine learning-based medical devices in the USA and Europe (2015\u201320): a comparative analysis\" (in en). The Lancet Digital Health 3 (3): e195\u2013e203. doi:10.1016\/S2589-7500(20)30292-2. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2589750020302922 .   \n \n\n\u2191 Sutton, Reed T.; Pincock, David; Baumgart, Daniel C.; Sadowski, Daniel C.; Fedorak, Richard N.; Kroeker, Karen I. (6 February 2020). \"An overview of clinical decision support systems: benefits, risks, and strategies for success\" (in en). npj Digital Medicine 3 (1): 17. doi:10.1038\/s41746-020-0221-y. ISSN 2398-6352. PMC PMC7005290. PMID 32047862. https:\/\/www.nature.com\/articles\/s41746-020-0221-y .   \n \n\n\u2191 Ceney, Adam; Tolond, Stephanie; Glowinski, Andrzej; Marks, Ben; Swift, Simon; Palser, Tom (15 July 2021). Wilson, Fernando A.. ed. \"Accuracy of online symptom checkers and the potential impact on service utilisation\" (in en). PLOS ONE 16 (7): e0254088. doi:10.1371\/journal.pone.0254088. ISSN 1932-6203. PMC PMC8282353. PMID 34265845. https:\/\/dx.plos.org\/10.1371\/journal.pone.0254088 .   \n \n\n\u2191 Anastasiou, Olympia E.; K\u00e4lsch, Julia; Hakmouni, Mahdi; Kucukoglu, Ozlem; Heider, Dominik; Korth, Johannes; Manka, Paul; Sowa, Jan-Peter et al. (1 July 2017). \"Low transferrin and high ferritin concentrations are associated with worse outcome in acute liver failure\" (in en). Liver International 37 (7): 1032\u20131041. doi:10.1111\/liv.13369. https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/liv.13369 .   \n \n\n\u2191 Riemenschneider, Mona; Hummel, Thomas; Heider, Dominik (1 December 2016). \"SHIVA - a web application for drug resistance and tropism testing in HIV\" (in en). BMC Bioinformatics 17 (1): 314. doi:10.1186\/s12859-016-1179-2. ISSN 1471-2105. PMC PMC4994198. PMID 27549230. http:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/s12859-016-1179-2 .   \n \n\n\u2191 Riemenschneider, Mona; Cashin, Kieran Y.; Budeus, Bettina; Sierra, Saleta; Shirvani-Dastgerdi, Elham; Bayanolhagh, Saeed; Kaiser, Rolf; Gorry, Paul R. et al. (29 April 2016). \"Genotypic Prediction of Co-receptor Tropism of HIV-1 Subtypes A and C\" (in en). Scientific Reports 6 (1): 24883. doi:10.1038\/srep24883. ISSN 2045-2322. PMC PMC4850382. PMID 27126912. https:\/\/www.nature.com\/articles\/srep24883 .   \n \n\n\u2191 Rieke, Nicola; Hancox, Jonny; Li, Wenqi; Milletar\u00ec, Fausto; Roth, Holger R.; Albarqouni, Shadi; Bakas, Spyridon; Galtier, Mathieu N. et al. (14 September 2020). \"The future of digital health with federated learning\" (in en). npj Digital Medicine 3 (1): 119. doi:10.1038\/s41746-020-00323-1. ISSN 2398-6352. PMC PMC7490367. PMID 33015372. https:\/\/www.nature.com\/articles\/s41746-020-00323-1 .   \n \n\n\u2191 Hauschild, Anne-Christin; Lemanczyk, Marta; Matschinske, Julian; Frisch, Tobias; Zolotareva, Olga; Holzinger, Andreas; Baumbach, Jan; Heider, Dominik (12 April 2022). Wren, Jonathan. ed. \"Federated Random Forests can improve local performance of predictive models for various healthcare applications\" (in en). Bioinformatics 38 (8): 2278\u20132286. doi:10.1093\/bioinformatics\/btac065. ISSN 1367-4803. https:\/\/academic.oup.com\/bioinformatics\/article\/38\/8\/2278\/6525214 .   \n \n\n\u2191 Warnat-Herresthal, Stefanie; Schultze, Hartmut; Shastry, Krishnaprasad Lingadahalli; Manamohan, Sathyanarayanan; Mukherjee, Saikat; Garg, Vishesh; Sarveswara, Ravi; H\u00e4ndler, Kristian et al. (10 June 2021). \"Swarm Learning for decentralized and confidential clinical machine learning\" (in en). Nature 594 (7862): 265\u2013270. doi:10.1038\/s41586-021-03583-3. ISSN 0028-0836. PMC PMC8189907. PMID 34040261. https:\/\/www.nature.com\/articles\/s41586-021-03583-3 .   \n \n\n\u2191 Torkzadehmahani, Reihaneh; Nasirigerdeh, Reza; Blumenthal, David B.; Kacprowski, Tim; List, Markus; Matschinske, Julian; Spaeth, Julian; Wenke, Nina Kerstin et al. (1 June 2022). \"Privacy-Preserving Artificial Intelligence Techniques in Biomedicine\" (in en). Methods of Information in Medicine 61 (S 01): e12\u2013e27. doi:10.1055\/s-0041-1740630. ISSN 0026-1270. PMC PMC9246509. PMID 35062032. http:\/\/www.thieme-connect.de\/DOI\/DOI?10.1055\/s-0041-1740630 .   \n \n\n\u2191 Nasirigerdeh, Reza; Torkzadehmahani, Reihaneh; Matschinske, Julian; Frisch, Tobias; List, Markus; Sp\u00e4th, Julian; Weiss, Stefan; V\u00f6lker, Uwe et al. (24 January 2022). \"sPLINK: a hybrid federated tool as a robust alternative to meta-analysis in genome-wide association studies\" (in en). Genome Biology 23 (1): 32. doi:10.1186\/s13059-021-02562-1. ISSN 1474-760X. PMC PMC8785575. PMID 35073941. https:\/\/genomebiology.biomedcentral.com\/articles\/10.1186\/s13059-021-02562-1 .   \n \n\n\u2191 12.0 12.1 12.2 \"Regulation (EU) 2017\/745 of the European Parliament and of the Council of 5 April 2017 on medical devices\". EUR-Lex. Publications Office of the European Union. 5 April 2017. http:\/\/data.europa.eu\/eli\/reg\/2017\/745\/oj .   \n \n\n\u2191 13.0 13.1 13.2 Medical Device Coordination Group (October 2019). \"MDCG 2019-11 Guidance on Qualification and Classification of Software in Regulation (EU) 2017\/745 \u2013 MDR and Regulation (EU) 2017\/746 \u2013 IVDR\" (PDF). https:\/\/health.ec.europa.eu\/system\/files\/2020-09\/md_mdcg_2019_11_guidance_qualification_classification_software_en_0.pdf .   \n \n\n\u2191 International Medical Device Regulators Forum SaMD Working Group (9 December 2013). \"Software as a Medical Device (SaMD): Key Definitions\" (PDF). International Medical Device Regulators Forum. https:\/\/www.imdrf.org\/sites\/default\/files\/docs\/imdrf\/final\/technical\/imdrf-tech-131209-samd-key-definitions-140901.pdf .   \n \n\n\u2191 Oen, R.D.R. (2009). \"Software als medizinprodukt\". Medizin Produkte Recht 2: 55\u20137.   \n \n\n\u2191 16.00 16.01 16.02 16.03 16.04 16.05 16.06 16.07 16.08 16.09 16.10 16.11 \"IEC 62304:2006\/Amd 1:2015 Medical device software \u2014 Software life cycle processes \u2014 Amendment 1\". International Organization for Standardization. June 2015. https:\/\/www.iso.org\/standard\/64686.html .   \n \n\n\u2191 \"IEC 62304 Edition 1.1 2015-06 CONSOLIDATED VERSION Medical device software - Software life cycle processes\". Recognized Consensus Standards. U.S. Food and Drug Administration. 14 January 2019. https:\/\/www.accessdata.fda.gov\/scripts\/cdrh\/cfdocs\/cfstandards\/detail.cfm?standard__identification_no=38829 .   \n \n\n\u2191 18.0 18.1 Pinto, Gustavo; Wiese, Igor; Dias, Luiz Felipe (1 March 2018). \"How do scientists develop scientific software? An external replication\". 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER) (Campobasso: IEEE): 582\u2013591. doi:10.1109\/SANER.2018.8330263. ISBN 978-1-5386-4969-5. http:\/\/ieeexplore.ieee.org\/document\/8330263\/ .   \n \n\n\u2191 19.0 19.1 19.2 19.3 19.4 19.5 Brito, Jaqueline J; Li, Jun; Moore, Jason H; Greene, Casey S; Nogoy, Nicole A; Garmire, Lana X; Mangul, Serghei (1 June 2020). \"Recommendations to enhance rigor and reproducibility in biomedical research\" (in en). GigaScience 9 (6): giaa056. doi:10.1093\/gigascience\/giaa056. ISSN 2047-217X. PMC PMC7263079. PMID 32479592. https:\/\/academic.oup.com\/gigascience\/article\/doi\/10.1093\/gigascience\/giaa056\/5849489 .   \n \n\n\u2191 20.0 20.1 20.2 20.3 Mangul, Serghei; Mosqueiro, Thiago; Abdill, Richard J.; Duong, Dat; Mitchell, Keith; Sarwal, Varuni; Hill, Brian; Brito, Jaqueline et al. (20 June 2019). \"Challenges and recommendations to improve the installability and archival stability of omics computational tools\" (in en). PLOS Biology 17 (6): e3000333. doi:10.1371\/journal.pbio.3000333. ISSN 1545-7885. PMC PMC6605654. PMID 31220077. https:\/\/dx.plos.org\/10.1371\/journal.pbio.3000333 .   \n \n\n\u2191 21.0 21.1 21.2 21.3 Lee, Graham; Bacon, Sebastian; Bush, Ian; Fortunato, Laura; Gavaghan, David; Lestang, Thibault; Morton, Caroline; Robinson, Martin et al. (1 February 2021). \"Barely sufficient practices in scientific computing\" (in en). Patterns 2 (2): 100206. doi:10.1016\/j.patter.2021.100206. PMC PMC7892476. PMID 33659915. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2666389921000167 .   \n \n\n\u2191 22.0 22.1 Wiese, Igor; Polato, Ivanilton; Pinto, Gustavo (1 July 2020). \"Naming the Pain in Developing Scientific Software\". IEEE Software 37 (4): 75\u201382. doi:10.1109\/MS.2019.2899838. ISSN 0740-7459. https:\/\/ieeexplore.ieee.org\/document\/8664473\/ .   \n \n\n\u2191 23.0 23.1 Riemenschneider, Mona; Wienbeck, Joachim; Scherag, Andr\u00e9; Heider, Dominik (1 June 2018). \"Data Science for Molecular Diagnostics Applications: From Academia to Clinic to Industry\" (in en). Systems Medicine 1 (1): 13\u201317. doi:10.1089\/sysm.2018.0002. ISSN 2573-3370. http:\/\/www.liebertpub.com\/doi\/10.1089\/sysm.2018.0002 .   \n \n\n\u2191 Coiera, Enrico; Ammenwerth, Elske; Georgiou, Andrew; Magrabi, Farah (1 August 2018). \"Does health informatics have a replication crisis?\" (in en). Journal of the American Medical Informatics Association 25 (8): 963\u2013968. doi:10.1093\/jamia\/ocy028. ISSN 1067-5027. PMC PMC6077781. PMID 29669066. https:\/\/academic.oup.com\/jamia\/article\/25\/8\/963\/4970161 .   \n \n\n\u2191 25.0 25.1 Wilkinson, Mark D.; Dumontier, Michel; Aalbersberg, IJsbrand Jan; Appleton, Gabrielle; Axton, Myles; Baak, Arie; Blomberg, Niklas; Boiten, Jan-Willem et al. (15 March 2016). \"The FAIR Guiding Principles for scientific data management and stewardship\" (in en). Scientific Data 3 (1): 160018. doi:10.1038\/sdata.2016.18. ISSN 2052-4463. PMC PMC4792175. PMID 26978244. https:\/\/www.nature.com\/articles\/sdata201618 .   \n \n\n\u2191 26.0 26.1 Sharma, Arjun; Blank, Anthony; Patel, Parashar; Stein, Kenneth (1 March 2013). \"Health care policy and regulatory implications on medical device innovations: a cardiac rhythm medical device industry perspective\" (in en). Journal of Interventional Cardiac Electrophysiology 36 (2): 107\u2013117. doi:10.1007\/s10840-013-9781-y. ISSN 1383-875X. PMC PMC3606523. PMID 23474980. http:\/\/link.springer.com\/10.1007\/s10840-013-9781-y .   \n \n\n\u2191 Manghani, Kishu (2011). \"Quality assurance: Importance of systems and standard operating procedures\" (in en). Perspectives in Clinical Research 2 (1): 34. doi:10.4103\/2229-3485.76288. ISSN 2229-3485. PMC PMC3088954. PMID 21584180. http:\/\/www.picronline.org\/text.asp?2011\/2\/1\/34\/76288 .   \n \n\n\u2191 McHugh, Martin; Ali, Abder-Rahman; McCaffery, Fergal (2013). The Significance of Requirements in Medical Device Software Development. doi:10.21427\/7ADY-5269. https:\/\/arrow.tudublin.ie\/scschcomcon\/126\/ .   \n \n\n\u2191 Memon, M.; Jalbani, A.A.; Menghwar, G.D. et al. (2016). \"I2A: An Interoperability & Integration Architecture for Medical Device Software and eHealth Systems\" (PDF). Science International 28 (4): 3783\u201387. ISSN 1013-5316. http:\/\/www.sci-int.com\/pdf\/637305175103306413.pdf .   \n \n\n\u2191 McHugh, Martin; Cawley, Oisin; McCaffcry, Fergal; Richardson, Ita; Wang, Xiaofeng (1 May 2013). \"An agile V-model for medical device software development to overcome the challenges with plan-driven software development lifecycles\". 2013 5th International Workshop on Software Engineering in Health Care (SEHC) (San Francisco, CA, USA: IEEE): 12\u201319. doi:10.1109\/SEHC.2013.6602471. ISBN 978-1-4673-6282-5. https:\/\/ieeexplore.ieee.org\/document\/6602471\/ .   \n \n\n\u2191 Davis, Alan Mark (1993). Software requirements: objects, functions, and states (Rev ed.). Englewood Cliffs, N.J: PTR Prentice Hall. ISBN 978-0-13-805763-3.   \n \n\n\u2191 Leffingwell, D. (1997). \"Calculating your return on investment from more effective requirements management\". American Programmer 10 (4): 13\u201316.   \n \n\n\u2191 Booch, Grady; Rumbaugh, James; Jacobson, Ivar (1999). The unified modeling language user guide. The Addison-Wesley object technology series. Reading Mass: Addison-Wesley. ISBN 978-0-201-57168-4.   \n \n\n\u2191 Booch, G.; Jacobson, I.; Rumbaugh, J. et al. (4 January 2005). \"Unified Modeling Language Specification Version 1.4.2\" (PDF). Object Management Group. https:\/\/www.omg.org\/spec\/UML\/ISO\/19501\/PDF .   \n \n\n\u2191 Fairley, Richard E. Dick; Bourque, Pierre; Keppler, John (1 April 2014). \"The impact of SWEBOK Version 3 on software engineering education and training\". 2014 IEEE 27th Conference on Software Engineering Education and Training (CSEE&T): 192\u2013200. doi:10.1109\/CSEET.2014.6816804. https:\/\/ieeexplore.ieee.org\/document\/6816804\/ .   \n \n\n\u2191 \"ISO\/IEC 25010:2011 Systems and software engineering \u2014 Systems and software Quality Requirements and Evaluation (SQuaRE) \u2014 System and software quality models\". International Organization for Standardization. March 2011. https:\/\/www.iso.org\/standard\/35733.html .   \n \n\n\u2191 Mosleh, Mohsen; Dalili, Kia; Heydari, Babak (1 March 2018). \"Distributed or Monolithic? A Computational Architecture Decision Framework\". IEEE Systems Journal 12 (1): 125\u2013136. doi:10.1109\/JSYST.2016.2594290. ISSN 1932-8184. http:\/\/ieeexplore.ieee.org\/document\/7539535\/ .   \n \n\n\u2191 Richards, Mark; Ford, Neal (2020). Fundamentals of software architecture: an engineering approach (First edition ed.). Sebastopol, CA: O'Reilly Media, Inc. ISBN 978-1-4920-4345-4. OCLC on1089438191. https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/on1089438191 .   \n \n\n\u2191 Jiang, Zhen Ming; Hassan, Ahmed E.; Holt, Richard C. (1 December 2006). \"Visualizing Clone Cohesion and Coupling\". 2006 13th Asia Pacific Software Engineering Conference (APSEC'06): 467\u2013476. doi:10.1109\/APSEC.2006.63. https:\/\/ieeexplore.ieee.org\/document\/4137451\/ .   \n \n\n\u2191 Hofmeister, C.; Nord, R. L.; Soni, D. (1999), Donohoe, Patrick, ed., \"Describing Software Architecture with UML\", Software Architecture (Boston, MA: Springer US) 12: 145\u2013159, doi:10.1007\/978-0-387-35563-4_9, ISBN 978-1-4757-6536-6, http:\/\/link.springer.com\/10.1007\/978-0-387-35563-4_9 . Retrieved 2023-07-04   \n \n\n\u2191 Sp\u00e4th, Peter (2021), \"About MVC: Model, View, Controller\" (in en), Beginning Java MVC 1.0 (Berkeley, CA: Apress): 1\u201318, doi:10.1007\/978-1-4842-6280-1_1, ISBN 978-1-4842-6279-5, http:\/\/link.springer.com\/10.1007\/978-1-4842-6280-1_1 . Retrieved 2023-07-04   \n \n\n\u2191  IEEE Standard Glossary of Software Engineering Terminology. IEEE. 31 December 1990. doi:10.1109\/ieeestd.1990.101064. ISBN 978-0-7381-0391-4. https:\/\/ieeexplore.ieee.org\/servlet\/opac?punumber=2238 .   \n \n\n\u2191 Abran, Alain; Moore, James W., eds. (2001). Guide to the software engineering body of knowledge: trial version [version 0.95]; SWEBOK*; a project of the Software Engineering Coordinating Commettee. Los Alamitos, Calif.: IEEE Computer Society. ISBN 978-0-7695-1000-2.   \n \n\n\u2191 Nidhra, Srinivas (30 June 2012). \"Black Box and White Box Testing Techniques - A Literature Review\". International Journal of Embedded Systems and Applications 2 (2): 29\u201350. doi:10.5121\/ijesa.2012.2204. http:\/\/www.airccse.org\/journal\/ijesa\/papers\/2212ijesa04.pdf .   \n \n\n\u2191 Murnane, T.; Reed, K. (2001). \"On the effectiveness of mutation analysis as a black box testing technique\". Proceedings 2001 Australian Software Engineering Conference (Canberra, ACT, Australia: IEEE Comput. Soc): 12\u201320. doi:10.1109\/ASWEC.2001.948492. ISBN 978-0-7695-1254-9. http:\/\/ieeexplore.ieee.org\/document\/948492\/ .   \n \n\n\u2191 Myers, Glenford J.; Sandler, Corey; Badgett, Tom (2012). The art of software testing (3rd ed ed.). Hoboken, N.J: John Wiley & Sons. ISBN 978-1-118-03196-4. OCLC 728656684. https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/728656684 .   \n \n\n\u2191 \"IEC 62366-1:2015 Medical devices \u2014 Part 1: Application of usability engineering to medical devices\". International Organization for Standardization. February 2015. https:\/\/www.iso.org\/standard\/63179.html .   \n \n\n\u2191 Hunter, Philip (1 September 2017). \"The reproducibility \u201ccrisis\u201d: Reaction to replication crisis should not stifle innovation\" (in en). EMBO reports 18 (9): 1493\u20131496. doi:10.15252\/embr.201744876. ISSN 1469-221X. PMC PMC5579390. PMID 28794201. https:\/\/www.embopress.org\/doi\/10.15252\/embr.201744876 .   \n \n\n\u2191 Boulesteix, Anne-Laure; Hoffmann, Sabine; Charlton, Alethea; Seibold, Heidi (1 October 2020). \"A Replication Crisis in Methodological Research?\" (in en). Significance 17 (5): 18\u201321. doi:10.1111\/1740-9713.01444. ISSN 1740-9705. https:\/\/academic.oup.com\/jrssig\/article\/17\/5\/18\/7038554 .   \n \n\n\u2191 Hauschild, Anne-Christin; Eick, Lisa; Wienbeck, Joachim; Heider, Dominik (1 July 2021). \"Fostering reproducibility, reusability, and technology transfer in health informatics\" (in en). iScience 24 (7): 102803. doi:10.1016\/j.isci.2021.102803. PMC PMC8282945. PMID 34296072. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2589004221007719 .   \n \n\n\u2191 Sapunar, Damir (28 May 2016). \"The business process management software for successful quality management and organization: case study from the University of Split School of medicine\". Acta Medica Academica 45 (1): 26\u201333. doi:10.5644\/ama2006-124.153. http:\/\/ama.ba\/index.php\/ama\/article\/view\/268\/pdf .   \n \n\n\nNotes \nThis presentation is faithful to the original, with only a few minor changes to presentation, though grammar and word usage was substantially updated for improved readability. In some cases important information was missing from the references, and that information was added.\n\n\n\n\n\nSource: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.limswiki.org\/index.php\/Journal:Guideline_for_software_life_cycle_in_health_informatics\">https:\/\/www.limswiki.org\/index.php\/Journal:Guideline_for_software_life_cycle_in_health_informatics<\/a>\nCategories: LIMSwiki journal articles (added in 2023)LIMSwiki journal articles (all)LIMSwiki journal articles on health informaticsLIMSwiki journal articles on softwareNavigation menuPage actionsJournalDiscussionView sourceHistoryPage actionsJournalDiscussionMoreToolsIn other languagesPersonal toolsLog inNavigationMain pageEncyclopedic articlesRecent changesRandom pageHelp about MediaWikiSearch\u00a0 ToolsWhat links hereRelated changesSpecial pagesPermanent linkPage informationPopular publications\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\nPrint\/exportCreate a bookDownload as PDFDownload as PDFDownload as Plain textPrintable version This page was last edited on 4 July 2023, at 22:20.Content is available under a Creative Commons Attribution-ShareAlike 4.0 International License unless otherwise noted.This page has been accessed 633 times.Privacy policyAbout LIMSWikiDisclaimers\n\n\n\n","3a2816a67d7d45f854c1e2fb9ec00f31_html":"<body class=\"mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-206 ns-subject page-Journal_Guideline_for_software_life_cycle_in_health_informatics rootpage-Journal_Guideline_for_software_life_cycle_in_health_informatics skin-monobook action-view skin--responsive\"><div id=\"rdp-ebb-globalWrapper\"><div id=\"rdp-ebb-column-content\"><div id=\"rdp-ebb-content\" class=\"mw-body\" role=\"main\"><a id=\"rdp-ebb-top\"><\/a>\n<h1 id=\"rdp-ebb-firstHeading\" class=\"firstHeading\" lang=\"en\">Journal:Guideline for software life cycle in health informatics<\/h1><div id=\"rdp-ebb-bodyContent\" class=\"mw-body-content\"><!-- start content --><div id=\"rdp-ebb-mw-content-text\" lang=\"en\" dir=\"ltr\" class=\"mw-content-ltr\"><div class=\"mw-parser-output\">\n\n\n<h2><span class=\"mw-headline\" id=\"Abstract\">Abstract<\/span><\/h2>\n<p>The long-lasting trend of <a href=\"https:\/\/www.limswiki.org\/index.php\/Medical_informatics\" class=\"mw-redirect wiki-link\" title=\"Medical informatics\" data-key=\"f89ecb3b26617b8c6e09bc5e050cfd5d\">medical informatics<\/a> is to adapt novel technologies in the medical context. In particular, incorporating <a href=\"https:\/\/www.limswiki.org\/index.php\/Artificial_intelligence\" title=\"Artificial intelligence\" class=\"wiki-link\" data-key=\"0c45a597361ca47e1cd8112af676276e\">artificial intelligence<\/a> (AI) to support clinical decision-making can significantly improve monitoring, diagnostics, and prognostics for the patient\u2019s and medic\u2019s sake. However, obstacles hinder a timely technology transfer from the medical research setting to the actual clinical setting. Due to the pressure for novelty in the <a href=\"https:\/\/www.limswiki.org\/index.php\/Research\" title=\"Research\" class=\"wiki-link\" data-key=\"409634fd90113f119362927fe222f549\">research<\/a> context, projects rarely implement <a href=\"https:\/\/www.limswiki.org\/index.php\/Quality_(business)\" title=\"Quality (business)\" class=\"wiki-link\" data-key=\"c4ac43430d1c3a3a15d1255257aaea37\">quality<\/a> standards.\n<\/p><p>Here, we propose a guideline for academic software life cycle (SLC) processes tailored to the needs and capabilities of research organizations. While the complete implementation of an SLC according to commercial industry standards is largely not feasible in scientific research work, we propose a subset of elements that we are convinced will provide a significant benefit to research settings while keeping the development effort within a feasible range.\n<\/p><p>Ultimately, the emerging quality checks for academic research software development can pave the way for a more accelerated deployment of academic software into clinical practice.\n<\/p><p><b>Keywords<\/b>: health informatics, bioinformatics, software engineering\n<\/p><p><b>Graphical abstract<\/b>: <a href=\"https:\/\/www.limswiki.org\/index.php\/File:GA_Hauschild_iScience2022_25-12.jpg\" class=\"image wiki-link\" data-key=\"41e6d3a5bf1ef25bc443ffce1799c2d9\"><img alt=\"GA Hauschild iScience2022 25-12.jpg\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/1\/1a\/GA_Hauschild_iScience2022_25-12.jpg\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<h2><span class=\"mw-headline\" id=\"Introduction\">Introduction<\/span><\/h2>\n<p>Today, <a href=\"https:\/\/www.limswiki.org\/index.php\/Medical_informatics\" class=\"mw-redirect wiki-link\" title=\"Medical informatics\" data-key=\"f89ecb3b26617b8c6e09bc5e050cfd5d\">medical informatics<\/a> is an integral part of health care systems that ensure the smooth operation of processes in medical care. Moreover, standard procedures ensure the transfer of knowledge from medical <a href=\"https:\/\/www.limswiki.org\/index.php\/Research\" title=\"Research\" class=\"wiki-link\" data-key=\"409634fd90113f119362927fe222f549\">research<\/a> to clinical practice, for instance, via regularly updated guidelines and <a href=\"https:\/\/www.limswiki.org\/index.php\/Regulatory_compliance\" title=\"Regulatory compliance\" class=\"wiki-link\" data-key=\"7dbc9be278a8efda25a4b592ee6ef0ca\">regulations<\/a>. In contrast, newly developed software innovations\u2014such as systems based on <a href=\"https:\/\/www.limswiki.org\/index.php\/Artificial_intelligence\" title=\"Artificial intelligence\" class=\"wiki-link\" data-key=\"0c45a597361ca47e1cd8112af676276e\">artificial intelligence<\/a> (AI) that could support clinical decisions and have already evolved to be the state-of-the-art in medical informatics research\u2014rarely transfer to application in practice.\n<\/p><p>Modern methods such as AI and <a href=\"https:\/\/www.limswiki.org\/index.php\/Machine_learning\" title=\"Machine learning\" class=\"wiki-link\" data-key=\"79aab39cfa124c958cd1dbcab3dde122\">machine learning<\/a> (ML) increasingly unroll their potential in medical healthcare to help patients and clinicians.<sup id=\"rdp-ebb-cite_ref-:0_1-0\" class=\"reference\"><a href=\"#cite_note-:0-1\">[1]<\/a><\/sup> <a href=\"https:\/\/www.limswiki.org\/index.php\/Clinical_decision_support_system\" title=\"Clinical decision support system\" class=\"wiki-link\" data-key=\"095141425468d057aa977016869ca37d\">Clinical decision support systems<\/a> (CDSSs) can effectively increase diagnostics, patient safety, and cost containment.<sup id=\"rdp-ebb-cite_ref-2\" class=\"reference\"><a href=\"#cite_note-2\">[2]<\/a><\/sup> Easily accessible AI-based applications can improve diagnosis and treatment of patients, e.g., by precisely detecting symptoms<sup id=\"rdp-ebb-cite_ref-3\" class=\"reference\"><a href=\"#cite_note-3\">[3]<\/a><\/sup>, evaluating biomarkers<sup id=\"rdp-ebb-cite_ref-4\" class=\"reference\"><a href=\"#cite_note-4\">[4]<\/a><\/sup>, or detecting pathogenic resistance or subtypes.<sup id=\"rdp-ebb-cite_ref-5\" class=\"reference\"><a href=\"#cite_note-5\">[5]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-6\" class=\"reference\"><a href=\"#cite_note-6\">[6]<\/a><\/sup> Furthermore, upcoming concepts such as federated learning<sup id=\"rdp-ebb-cite_ref-7\" class=\"reference\"><a href=\"#cite_note-7\">[7]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-8\" class=\"reference\"><a href=\"#cite_note-8\">[8]<\/a><\/sup> and swarm learning<sup id=\"rdp-ebb-cite_ref-9\" class=\"reference\"><a href=\"#cite_note-9\">[9]<\/a><\/sup> allow the cross-clinical creation of data-driven models without disrupting patient\u2019s privacy<sup id=\"rdp-ebb-cite_ref-10\" class=\"reference\"><a href=\"#cite_note-10\">[10]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-11\" class=\"reference\"><a href=\"#cite_note-11\">[11]<\/a><\/sup>, opening the gate for more powerful data-driven development.\n<\/p><p>However, software has its risks, especially within <a href=\"https:\/\/www.limswiki.org\/index.php\/Medical_device\" title=\"Medical device\" class=\"wiki-link\" data-key=\"8e821122daa731f0fa8782fae57831fa\">medical devices<\/a>. To protect patients from any risk of injury, disability, or other harmful interventions, medical device software (MDSW)\u2014which is intended to provide specific medical purposes, such as diagnosis, monitoring, prognosis, or treatment\u2014are subject to strict regulations. Examples include the European Medical Devices Regulation (MDR)<sup id=\"rdp-ebb-cite_ref-:1_12-0\" class=\"reference\"><a href=\"#cite_note-:1-12\">[12]<\/a><\/sup>, the In Vitro Diagnostic Medical Devices Regulation (IVDR)<sup id=\"rdp-ebb-cite_ref-:2_13-0\" class=\"reference\"><a href=\"#cite_note-:2-13\">[13]<\/a><\/sup>, and the International Medical Device Regulators Forum (IMDRF).<sup id=\"rdp-ebb-cite_ref-14\" class=\"reference\"><a href=\"#cite_note-14\">[14]<\/a><\/sup>\n<\/p><p>MDSW can be an integral part of a medical product or standalone software as an independent medical device.<sup id=\"rdp-ebb-cite_ref-15\" class=\"reference\"><a href=\"#cite_note-15\">[15]<\/a><\/sup> MDSW can run in the <a href=\"https:\/\/www.limswiki.org\/index.php\/Cloud_computing\" title=\"Cloud computing\" class=\"wiki-link\" data-key=\"fcfe5882eaa018d920cedb88398b604f\">cloud<\/a>, as part of a software platform, or on a server, with both healthcare professionals and laypersons using it. Exceptions are software tools used for documentation or that solely control medical device hardware, or that serve no medical purpose.<sup id=\"rdp-ebb-cite_ref-:2_13-1\" class=\"reference\"><a href=\"#cite_note-:2-13\">[13]<\/a><\/sup>\n<\/p><p>An integral part of all regulations for MDSW is development according to the software life cycle process, as defined by <a href=\"https:\/\/www.limswiki.org\/index.php\/IEC_62304\" title=\"IEC 62304\" class=\"wiki-link\" data-key=\"5a40869706914f318910e6ff550eb941\">IEC 62304<\/a><sup id=\"rdp-ebb-cite_ref-:3_16-0\" class=\"reference\"><a href=\"#cite_note-:3-16\">[16]<\/a><\/sup>, a harmonized international standard that regulates MDSW life cycle development, requiring documentation and processes, such as software development planning, requirement analysis, architectural design, testing, verification, and maintenance.<sup id=\"rdp-ebb-cite_ref-:1_12-1\" class=\"reference\"><a href=\"#cite_note-:1-12\">[12]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-17\" class=\"reference\"><a href=\"#cite_note-17\">[17]<\/a><\/sup>\n<\/p><p>These regulations focus on minimizing patient risk, for example, harmful follow-up analysis, a wrong or missing treatment where needed, as a result of software failures, incorrect predictions, or other malfunctions. Several challenges arise in the attempt to eliminate these risks and allow for a smoother transfer of technology and greater reproducibility.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Challenges_of_scientific_software_development_for_health_care\">Challenges of scientific software development for health care<\/span><\/h3>\n<p>The primary goal of scientists remains conducting scientific activities rather than developing software. However, many scientists aim to make their findings and methodologies available to a broader audience and to be used for the greater good. Thus, many data science and AI methodologies, as well as corresponding implementations and software packages, exist that would, in theory, allow the development of efficacious AI-driven CDSSs and MDSW. However, scientists of different backgrounds tend to have very different knowledge of software engineering practices, often acquired through self-study.<sup id=\"rdp-ebb-cite_ref-:4_18-0\" class=\"reference\"><a href=\"#cite_note-:4-18\">[18]<\/a><\/sup> Moreover, academic groups often consist of small teams that undergo frequent change or researchers that work on a \"one person-one project\" basis.<sup id=\"rdp-ebb-cite_ref-:5_19-0\" class=\"reference\"><a href=\"#cite_note-:5-19\">[19]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:6_20-0\" class=\"reference\"><a href=\"#cite_note-:6-20\">[20]<\/a><\/sup> Thus, a lack of attention to relevant software development processes and engineering practices defined by the software life cycle negatively affects the usefulness of developed packages, particularly for developing software as a medical device.<sup id=\"rdp-ebb-cite_ref-:7_21-0\" class=\"reference\"><a href=\"#cite_note-:7-21\">[21]<\/a><\/sup>\n<\/p><p>Pinning down the most critical requirements along with an accurate description and documentation of such is a significant challenge for all software projects independent of if it is conducted in research or industry.<sup id=\"rdp-ebb-cite_ref-:4_18-1\" class=\"reference\"><a href=\"#cite_note-:4-18\">[18]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:8_22-0\" class=\"reference\"><a href=\"#cite_note-:8-22\">[22]<\/a><\/sup> It necessitates a detailed analysis of the non-functional requirements as determined, for instance, by regulatory entities, addressing aspects such as <a href=\"https:\/\/www.limswiki.org\/index.php\/Software_development_security\" title=\"Software development security\" class=\"wiki-link\" data-key=\"20e75bd7ff6754e63e7ae5bb9e9ce4fb\">security<\/a>, <a href=\"https:\/\/www.limswiki.org\/index.php\/Information_privacy\" title=\"Information privacy\" class=\"wiki-link\" data-key=\"185f6d9f874e48914b5789317408f782\">privacy<\/a>, or infrastructural limitations, as well as functional requirements like user-friendly interfaces and specific results. This is very time-consuming and relies on a close interaction of developers, stakeholders, and potential users, which is often difficult to achieve under academic circumstances.<sup id=\"rdp-ebb-cite_ref-:6_20-1\" class=\"reference\"><a href=\"#cite_note-:6-20\">[20]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:8_22-1\" class=\"reference\"><a href=\"#cite_note-:8-22\">[22]<\/a><\/sup> Moreover, researchers are enticed by academic hiring procedures and driven by funders to focus on \u201cnovelty\u201d rather than <a href=\"https:\/\/www.limswiki.org\/index.php\/Software_quality\" title=\"Software quality\" class=\"wiki-link\" data-key=\"dc7ddaaea83c58d8aa99e6ea33d93486\">software quality<\/a> and practical usefulness.<sup id=\"rdp-ebb-cite_ref-:5_19-1\" class=\"reference\"><a href=\"#cite_note-:5-19\">[19]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:6_20-2\" class=\"reference\"><a href=\"#cite_note-:6-20\">[20]<\/a><\/sup> Thus, implementations often fail to fulfill requirements, ensuring long-term sustainability such as documentation, usability, appropriate performance for practical application, user-friendliness optimally supporting potential users, and minimizing risks.<sup id=\"rdp-ebb-cite_ref-:5_19-2\" class=\"reference\"><a href=\"#cite_note-:5-19\">[19]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:9_23-0\" class=\"reference\"><a href=\"#cite_note-:9-23\">[23]<\/a><\/sup>\n<\/p><p>The most critical aspects of ensuring sustainability in academic software are reproducibility, reusability, and traceability.<sup id=\"rdp-ebb-cite_ref-:7_21-1\" class=\"reference\"><a href=\"#cite_note-:7-21\">[21]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-24\" class=\"reference\"><a href=\"#cite_note-24\">[24]<\/a><\/sup> However, it has been shown that not only public accessibility but also documentation and portability are essential to ensure reproducibility and underpin trust in the scientific record of scientific software, enabling the re-use of research and code. Moreover, the prototype-centered development procedures often lack quality checks, such as systematic testing, that would ensure reusability.<sup id=\"rdp-ebb-cite_ref-:7_21-2\" class=\"reference\"><a href=\"#cite_note-:7-21\">[21]<\/a><\/sup> Recently, scientific journals such as <i>GigaScience<\/i> or <i>Biostatistics<\/i> have promoted reproducibility and reusability by mandating the FAIR Principles (i.e., findability, accessibility, interoperability, and reusability). FAIR establishes a guideline for scientific <a href=\"https:\/\/www.limswiki.org\/index.php\/Information_management\" title=\"Information management\" class=\"wiki-link\" data-key=\"f8672d270c0750a858ed940158ca0a73\">data management<\/a> and documentation.<sup id=\"rdp-ebb-cite_ref-:5_19-3\" class=\"reference\"><a href=\"#cite_note-:5-19\">[19]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:10_25-0\" class=\"reference\"><a href=\"#cite_note-:10-25\">[25]<\/a><\/sup> Implementing the FAIR principles in academic software development has the potentil to lower the barriers to a successful industrial transition.\n<\/p><p>Additionally, for long-term software maintenance, well-structured development planning and processes can ensure the traceability of modifications via <a href=\"https:\/\/www.limswiki.org\/index.php\/Change_management\" title=\"Change management\" class=\"wiki-link\" data-key=\"11dfc2500cc1d506d28f1ac94b051415\">change management<\/a> and <a href=\"https:\/\/www.limswiki.org\/index.php\/Version_control\" title=\"Version control\" class=\"wiki-link\" data-key=\"81823f6b21d385f8db9ac0a17b571cc1\">version control<\/a>. These aspects ultimately determine scientific rigor, transparency, and reproducibility.<sup id=\"rdp-ebb-cite_ref-:5_19-4\" class=\"reference\"><a href=\"#cite_note-:5-19\">[19]<\/a><\/sup>\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Software_life_cycle\">Software life cycle<\/span><\/h3>\n<p>Proper implementation of the software life cycle (SLC) guarantees high-quality planning, development, and maintenance of MDSW. The SLC deals with the planning and specification, development, maintenance, and configuration of the software. IEC 62304 defines the SLC as a conceptual structure across its lifetime, from the requirements\u2019 definition to final release. It describes processes, tasks, and activities involved in developing a software product and their order and interdependencies. Furthermore, it defines milestones verifying the completeness of the results to be delivered.<sup id=\"rdp-ebb-cite_ref-:3_16-1\" class=\"reference\"><a href=\"#cite_note-:3-16\">[16]<\/a><\/sup>\n<\/p><p>However, a complete SLC described in standards like IEC 62304 is not feasible for most research projects.<sup id=\"rdp-ebb-cite_ref-:11_26-0\" class=\"reference\"><a href=\"#cite_note-:11-26\">[26]<\/a><\/sup> Academic research is often subject to tight schedules and focuses on proof-of-concept development, neglecting formal documentation or procedures. Here we provide recommendations for an SLC in academia, lowering the boundary for many research organizations to implement an SLC and fostering the transfer of technology to industrial development.<sup id=\"rdp-ebb-cite_ref-:2_13-2\" class=\"reference\"><a href=\"#cite_note-:2-13\">[13]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:11_26-1\" class=\"reference\"><a href=\"#cite_note-:11-26\">[26]<\/a><\/sup>\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Our_goal:_An_academia-tailored_software_life_cycle\">Our goal: An academia-tailored software life cycle<\/span><\/h3>\n<p>Until now, there has been little guidance on supporting a structured software development culture for academic institutions according to standard SLC processes. In this article, we present a synopsis of all requirements in official standards that are relevant to academia. We adjusted these toward the specific demands of software development in research and established a limited SLC process for research organizations, which has the potential to greatly facilitate and speed up such technology transfer and reproducibility in a controlled and predictable way. Being aware that a complete SLC is not feasible for most academic settings, we proposed a subset of elements that we are convinced will provide a significant benefit without creating an excessive organizational burden for researchers and developers and keep activities in a manageable range.\n<\/p><p>Our proposal is centered on procedures for software development planning, software requirement analysis, software architectural design, software unit implementation, integration, <a href=\"https:\/\/www.limswiki.org\/index.php\/Software_reliability_testing\" title=\"Software reliability testing\" class=\"wiki-link\" data-key=\"5926c9aa3c09fd81e233cdbc8d644a75\">testing<\/a>, <a href=\"https:\/\/www.limswiki.org\/index.php\/Software_verification_and_validation\" title=\"Software verification and validation\" class=\"wiki-link\" data-key=\"14bdd953b2bde31c520ba57b1002195e\">verification<\/a>, and configuration management. Depending on the specific needs, the elements of an SLC process that work best for an organization may differ from what we propose. The fact, however, that a life cycle process is set up at all and that the elements are deliberately chosen is probably a key factor for facilitating technology transfer. However, any medical software development for clinical use must strictly follow the regulations relevant to the specific country or region. Thus, our guideline can only provide a starting point intended to be adapted to institute- or project-specific requirements, considering only relevant aspects. Nevertheless, the ideas presented here are not meant to provide a shortcut for medical software. An overview of our suggested SLC activities in tandem with regulations is provided in Table S1 of the Supplemental information.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Software_life_cycle_for_medical_software_research_guideline\">Software life cycle for medical software research guideline<\/span><\/h2>\n<p>Our guideline covers multiple processes such as development planning, requirement analysis, software architecture, software design, implementation, software, and integration testing, verification, and release. In the following, we present the most vital points for the SLC, as depicted in Figure 1.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig1_Hauschild_iScience2022_25-12.jpg\" class=\"image wiki-link\" data-key=\"86e77cee65277308d686180677b4cc5d\"><img alt=\"Fig1 Hauschild iScience2022 25-12.jpg\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/5\/5b\/Fig1_Hauschild_iScience2022_25-12.jpg\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 1.<\/b> Main components of the software life cycle (SLC) for research-based medical software.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>The main focus of the SLC, in compliance with the <a href=\"https:\/\/www.limswiki.org\/index.php\/Quality_management_system\" title=\"Quality management system\" class=\"wiki-link\" data-key=\"dfecf3cd6f18d4a5e9ac49ca360b447d\">quality management system<\/a> (QMS), lies in the software development arena, while also tapping into configuration and change management, as well as legacy software.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Software_development:_Planning\">Software development: Planning<\/span><\/h3>\n<p>Defining an accurate software development plan is the first key component and must be updated regularly during the project, getting referenced and refined throughout the entire software life cycle model. It defines norms, methods, used processes, deliverable results, traceability between requirements, software testing, implemented risk control measures, configuration and change management, and verification of configuration elements, including software of unknown provenance (SOUP). (SOUP is a commonly used basic software library or set of packages that have not been developed for medical purposes.)\n<\/p><p>Two documents should be provided for the academic-tailored implementation: the process description and the development plan. Each document produced during software development has to contain a title, purpose, and responsible person.<sup id=\"rdp-ebb-cite_ref-:3_16-2\" class=\"reference\"><a href=\"#cite_note-:3-16\">[16]<\/a><\/sup> In the case of multiple involved developers, the role and responsibility assignments should be noted down in the single process description.\n<\/p><p>First, the process description is provided by the definition of standard operating procedures (SOPs) for repetitive application problems.<sup id=\"rdp-ebb-cite_ref-27\" class=\"reference\"><a href=\"#cite_note-27\">[27]<\/a><\/sup> The description contains each activity within the processes, when it will be completed, by whom, how, and with which input and output. The developed process description can be applied to multiple projects and has to be implemented by the developers.<sup id=\"rdp-ebb-cite_ref-:1_12-2\" class=\"reference\"><a href=\"#cite_note-:1-12\">[12]<\/a><\/sup> Since the life cycle model must be completely defined or referenced by the development plan, a software engineering model must be selected to provide a general structure for the development phase, such as the V-model. As a common approach in medical device development, the V-model is successfully used to achieve <a href=\"https:\/\/www.limswiki.org\/index.php\/Regulatory_compliance\" title=\"Regulatory compliance\" class=\"wiki-link\" data-key=\"7dbc9be278a8efda25a4b592ee6ef0ca\">regulatory compliance<\/a>.<sup id=\"rdp-ebb-cite_ref-28\" class=\"reference\"><a href=\"#cite_note-28\">[28]<\/a><\/sup> Generally, the selected model should match the project\u2019s characteristics and thus can host agile practices or elements of SCRUM to support and conform to life cycle development practices.<sup id=\"rdp-ebb-cite_ref-29\" class=\"reference\"><a href=\"#cite_note-29\">[29]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-30\" class=\"reference\"><a href=\"#cite_note-30\">[30]<\/a><\/sup>\n<\/p><p>Second, the development plan describes the general documentation required for a product or project. The development plan defines concrete milestones to corresponding deadlines, assigns designated staff to the pre-defined roles, and refines measures or tools adjusted to the project. It is challenging to meet the regulatory requirement of defining tools, testing, and configuration management in advance, particularly in a volatile academic setting. Due to external factors and unforeseeable changes, the development plan must be updated over the lifetime of the project, while the pre-defined process descriptions remain. Additionally, the process should recommend defining a coding guideline or convention, including code style, nomenclature, and naming in the development plan, to increase software quality. For example, using pre-defined rules in git hooks or continuous integration (CI), combined with linting tools, can enforce coding compliance. \n<\/p><p>The development process gets more transparent and well-structured through the provision of these two documents.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Software_development:_Requirements_analysis\">Software development: Requirements analysis<\/span><\/h3>\n<p>A software requirement is a detailed statement about a property that a software product, system, or process should fulfill and is defined during software requirements analysis. Software requirements should cover functional requirements such as inputs, outputs, functions, processes, interface reactions, and thresholds, as well as non-functional requirements such as physical characteristics, computing environments, performance, <a href=\"https:\/\/www.limswiki.org\/index.php\/Cybersecurity\" class=\"mw-redirect wiki-link\" title=\"Cybersecurity\" data-key=\"ba653dc2a1384e5f9f6ac9dc1a740109\">cybersecurity<\/a>, privacy, maintenance, installation, networking, and so forth. Precisely defined requirements are a vital element for the success of a project, particularly given that studies have concluded that half of all software errors are derived from mistakes in the requirement phase.<sup id=\"rdp-ebb-cite_ref-31\" class=\"reference\"><a href=\"#cite_note-31\">[31]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-32\" class=\"reference\"><a href=\"#cite_note-32\">[32]<\/a><\/sup> High-level requirements should be defined within the specifications, including the desired properties. These specifications describe mainly the project\u2019s total goal under defined restrictions. For practical consideration, it is beneficial to begin with general natural language requirements and refine them into graphical notations, such as Unified Model Language (UML) diagrams.<sup id=\"rdp-ebb-cite_ref-33\" class=\"reference\"><a href=\"#cite_note-33\">[33]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-34\" class=\"reference\"><a href=\"#cite_note-34\">[34]<\/a><\/sup>\n<\/p><p>However, the original requirement within the specification must always be bidirectionally linked to allow traceability. It should be possible to describe and follow the life of requirements in all directions. This means that traceability implies the comprehension of a design, starting with the source of a requirement, its implementation, testing, and maintenance. Moreover, it facilitates a high level of software quality, a critical concern for medical devices. Therefore, the derivation and documentation of the requirements should be updated and verified during the project.<sup id=\"rdp-ebb-cite_ref-:3_16-3\" class=\"reference\"><a href=\"#cite_note-:3-16\">[16]<\/a><\/sup> Finally, a critical aspect is the definition of requirements for maintenance, which is often neglected in academia since the focus is on publishing new technologies in contrast to maintaining existing software.\n<\/p><p>Nevertheless, maintenance must be considered at the beginning of the software life cycle to ensure that post-delivery support is possible.<sup id=\"rdp-ebb-cite_ref-35\" class=\"reference\"><a href=\"#cite_note-35\">[35]<\/a><\/sup> Therefore, a maintenance plan in academia does not have to be complete but must define all factors influencing either the development or architecture. Table 1, which examines ISO\/IEC 25010<sup id=\"rdp-ebb-cite_ref-36\" class=\"reference\"><a href=\"#cite_note-36\">[36]<\/a><\/sup>, can be used to evaluate the completeness of a software requirement analysis. Further, an implementation example is provided in the Supplemental information.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"2\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 1.<\/b> The product quality properties of ISO\/IEC 25010, used to evaluate if software requirements are complete.\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Aspect\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Properties\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Usability\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">- Recognizable appropriateness<br \/>- Learnability<br \/>- Operability<br \/>- User error protection<br \/>- Accessibility\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Functional suitability\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">- Functional completeness<br \/>- Functional correctness<br \/>- Functional appropriateness\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Security\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">- Confidentiality<br \/>- Integrity<br \/>- Non-repudiation<br \/>- Accountability<br \/>- Authenticity\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Maintainability\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">- Modularity<br \/>- Reusability<br \/>- Analyzability<br \/>- Modifiability<br \/>- Testability\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Performance efficiency\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">- Time behavior<br \/>- Resource utilization<br \/>- Capacity\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Reliability\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">- Maturity<br \/>- Availability<br \/>- Fault tolerance<br \/>- Recoverability\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Portability\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">- Adaptability<br \/>- Installability<br \/>- Replaceability\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Compatibility\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">- Co-existence<br \/>- Interoperability\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h3><span class=\"mw-headline\" id=\"Software_development:_Architecture_and_design\">Software development: Architecture and design<\/span><\/h3>\n<p>The regulatory authorities demand the definition of essential structural software components, identification of their primary responsibilities, visible features, and their interrelations.<sup id=\"rdp-ebb-cite_ref-:3_16-4\" class=\"reference\"><a href=\"#cite_note-:3-16\">[16]<\/a><\/sup> The architecture as an overarching structure conceptually defines data storage, interfaces, and logical servers. At the same time, modularization is described within the detailed software design, specifying how the single elements of the architecture and the requirements are explicitly implemented.\n<\/p><p>Software architecture consists of the system\u2019s structure in combination with architecture characteristics the system must support (e.g., availability, scalability, and security), architecture decisions (formulation of rules and constraints), and design principles. An architecture categorizes into monolithic and distributed architecture types consisting of single packages or separable sub-systems.<sup id=\"rdp-ebb-cite_ref-37\" class=\"reference\"><a href=\"#cite_note-37\">[37]<\/a><\/sup> It is recommended to specify an appropriate architecture prior to implementation. However, the choice is not regulated.<sup id=\"rdp-ebb-cite_ref-:3_16-5\" class=\"reference\"><a href=\"#cite_note-:3-16\">[16]<\/a><\/sup> These architectural design decisions will guide the developers throughout the development process.\n<\/p><p>The system has to be divided for the software design until it is represented through software units. These are sets of procedures or functions encapsulated in a package or class that cannot be further divided. Each software unit and interface needs a verified detailed design to ensure correct implementation.<sup id=\"rdp-ebb-cite_ref-:3_16-6\" class=\"reference\"><a href=\"#cite_note-:3-16\">[16]<\/a><\/sup> The design principles cover every option or state of all system components in detail, such as a preferred method or protocol.<sup id=\"rdp-ebb-cite_ref-38\" class=\"reference\"><a href=\"#cite_note-38\">[38]<\/a><\/sup> In order to have well testable and maintainable code, it is recommended to have software with low coupling (dependencies between the sub-systems) and high cohesion (internal dependencies).<sup id=\"rdp-ebb-cite_ref-39\" class=\"reference\"><a href=\"#cite_note-39\">[39]<\/a><\/sup> These associations can be well described using widely accepted notation standards such as UML, including class and activity diagrams to document the architectural decisions, which is highly recommendable to facilitate understanding the architecture.<sup id=\"rdp-ebb-cite_ref-40\" class=\"reference\"><a href=\"#cite_note-40\">[40]<\/a><\/sup>\n<\/p><p>In academia, two documents should be provided: the software architecture description and the detailed design. It is unlikely to narrow down the whole codebase in a detailed design. However, it must contain the utmost vital components, such as elements of design patterns, classes with crucial functionality, or specific interfaces. For example, a strict logical dissociation between the internal logic and the interface, e.g., for user interaction, is critical. Design patterns such as the model view controller (MVC) are favorable.<sup id=\"rdp-ebb-cite_ref-41\" class=\"reference\"><a href=\"#cite_note-41\">[41]<\/a><\/sup>\n<\/p>\n<h3><span id=\"rdp-ebb-Software_development:_Implementation,_testing,_and_verification\"><\/span><span class=\"mw-headline\" id=\"Software_development:_Implementation.2C_testing.2C_and_verification\">Software development: Implementation, testing, and verification<\/span><\/h3>\n<p>Generally, each software unit must be implemented, tested, and verified. The IEEE defines implementation as translating a design into hardware or software components, or both.<sup id=\"rdp-ebb-cite_ref-42\" class=\"reference\"><a href=\"#cite_note-42\">[42]<\/a><\/sup> In particular, the detailed design has to be translated into source code. Following a specific coding style and documentation standards is advisable during the implementation. Subsequently, every software unit has to be tested and verified separately, ensuring it works as specified in the detailed design and complies with the coding style. After that, it has to be integrated, verified, and tested dependently and independently in the following integration tests. These evaluate the software unit\u2019s functionality combined with other components into an overall system.\n<\/p><p>The software is usually tested on different abstraction levels within the software\u2019s life cycle, differentiating between unit, integration, regression, and system testing. While isolated unit tests verify the functionality of a separately testable software element, integration tests verify the interaction between the software units, as described by the software architecture. Along with different test strategies, such as top-down or bottom-up, integration tests must be conducted during several stages of the development process and are tailored to each integration level.<sup id=\"rdp-ebb-cite_ref-43\" class=\"reference\"><a href=\"#cite_note-43\">[43]<\/a><\/sup>\n<\/p><p>Integration and system tests can be combined with routine activities but must cover all software requirements. Especially, software components affecting safety require extensive tests. Appropriate evaluations of the testing procedure, verification, and integration strategy concerning the previously determined requirements are necessary. Tests and results must be recorded with acceptance criteria, providing repeatability and traceability between requirements and their verifications.\n<\/p><p>Tests can be performed either as white-box testing<sup id=\"rdp-ebb-cite_ref-44\" class=\"reference\"><a href=\"#cite_note-44\">[44]<\/a><\/sup>, including the knowledge of the underlying architecture, or as black-box testing<sup id=\"rdp-ebb-cite_ref-45\" class=\"reference\"><a href=\"#cite_note-45\">[45]<\/a><\/sup>, which does not take into account the internal structure. Besides automated tests, non-automated tests should be conducted between program coding and the beginning of computer-based testing. Moreover, the three fundamental human testing methods are inspections, walkthroughs, and usability testing.<sup id=\"rdp-ebb-cite_ref-46\" class=\"reference\"><a href=\"#cite_note-46\">[46]<\/a><\/sup> As demonstrated in our example in the supplemental information, SCRUM supports software integration and system testing since, after each sprint, an increment of potentially shippable functionality consisting of tested, well-written, and executable code is required (Supplemental information, Figure S2). Consequently, verification and testing are automatically included in the process of SCRUM. Project-specific regular, complete tests which are documented and traceable to requirements, software architecture, and detailed design are critical for software development within the law. A test is successful if it passes the acceptance criteria, defined through the requirements specification, the interface design within the detailed design, and the coding guideline. Ultimately, the verification evaluates whether all specified requirements are fulfilled by validating objective proof.<sup id=\"rdp-ebb-cite_ref-:3_16-7\" class=\"reference\"><a href=\"#cite_note-:3-16\">[16]<\/a><\/sup>\n<\/p><p>To ensure adequate software verification, it has to be well-planned and integrated into several stages of the SLC: requirements analysis, software architecture, software design, and software units, as well as their integration, changes, and problem resolutions, have to be verified. The management of verification documents can also be partially organized automatically through CI or as such with the Jira API or the Gitlab CI\/CD. Table 2 lists the suggested aspects to verify the different stages of the software development process. In particular, problem resolutions and other changes have to be re-verified and documented. Overall, verification is an activity of high importance throughout the whole development process. To verify more mature artifacts, one must verify their foundation as well. This hierarchy should always be kept in mind, as the Supplemental information example demonstrates.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"2\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 2.<\/b> Verification at all software development process stages.\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Development aspect\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Requirement(s)\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">System requirements\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">- Must be derived from the stakeholder\u2019s requirements<br \/>- May not contradict each other<br \/>- Must be consistent, unambiguous, clearly identifiable<br \/>\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Software requirements\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">- Implement the system requirements<br \/>- May not contradict each other<br \/>- Must be consistent, unambiguous, clearly identifiable<br \/>- Must be traceable to the system requirements or other sources<br \/>- Testing criteria must be drivable\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Software architecture\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">- All System and software requirements are implemented<br \/>- Must support the interfaces as well as SOUP items\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Detailed design\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">- Implements do not contradict the software architecture\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Software units\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">- Test case for each requirement must be passed<br \/>- Code may not contradict the interface design, the detailed design, or the coding guideline<br \/>- Verification of all requirements, architecture, and detailed design must be documented\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Software integration\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">- Software unit integration is realized according to an integration plan derived from the software architecture<br \/>- Software system tests verify the software\u2019s functionality\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h3><span class=\"mw-headline\" id=\"Software_development:_Release\">Software development: Release<\/span><\/h3>\n<p>In contrast to industry, academic software is released to other researchers via public repositories and journal publications. Before software release, testing and verification need to be completed and evaluated. That includes, first, all known residual anomalies that have to be documented and evaluated. Second, documentation of the release procedure and the software development environment has to be recorded with the released software version. Third, all activities and tasks of the software development plan must be completed and documented. Fourth, the medical device software, all configuration elements, and the documentation must be filled for the whole lifetime of the medical device software, defined by the development team as long as the relevant regulatory requirements demand it. Fifth, procedures to ensure a reliable delivery without damaging or unauthorized adjustments have to be defined.<sup id=\"rdp-ebb-cite_ref-:3_16-8\" class=\"reference\"><a href=\"#cite_note-:3-16\">[16]<\/a><\/sup>\n<\/p><p>After the software is released, all changes and updates are implemented within the software maintenance process, following the same steps as the software development process. The post-delivery maintenance decisions that must be made are included in software development planning.\n<\/p><p>Suppose the development process is well-defined and followed, and the previous sections of this guideline are considered. In that case, the complete verification and the required documents are delivered by default. Well-implemented traceability is essential to ensure the development process can be archived transparently. Regarding SCRUM, one could include the required documentation within the definition of \"done\" to ensure everything is documented since the development takes place in a regulatory context.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Legacy_software\">Legacy software<\/span><\/h3>\n<p>According to IEC 62304, legacy software is defined as software that was not developed to be used within software as a medical device, such as general software packages and libraries. It, therefore, lacks sufficient verification that it was developed in compliance with the current version of the norm.\n<\/p><p>Thus, it is sufficient to prove it conforms to the norm, and shortcomings to the norm\u2019s requirements need assessment.<sup id=\"rdp-ebb-cite_ref-:3_16-9\" class=\"reference\"><a href=\"#cite_note-:3-16\">[16]<\/a><\/sup> Hence risks of using the legacy software, as well as the risks of missing documentation, need to be identified and mitigated, if possible, as defined by the risk management process of the IEC 62366-1 standard.<sup id=\"rdp-ebb-cite_ref-47\" class=\"reference\"><a href=\"#cite_note-47\">[47]<\/a><\/sup>\n<\/p><p>In academia, it is essential to choose legacy software and document its usage carefully. Ideally, the used legal software has to fulfill the requirements of the IEC concerning risks as well, but the scope of action only demands closing gaps if it reduces the risk of usage.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Configuration_and_change_management\">Configuration and change management<\/span><\/h3>\n<p>Configuration and change management is crucial in ensuring usability, reproducibility, reusability, and traceability of software in the industry and academia. In academic research, automated change management systems, such as GitHub or GitLab, exist for software code and data and are regularly used.<sup id=\"rdp-ebb-cite_ref-:10_25-1\" class=\"reference\"><a href=\"#cite_note-:10-25\">[25]<\/a><\/sup> However, implementing adequate configuration and change management documentation for the entire SLC, as required by the IEC 62304 standard, is particularly challenging in academia, where the pressure to publish urges researchers to focus on novelty rather than maintenance.\n<\/p><p>In order to mitigate the ongoing replication crisis<sup id=\"rdp-ebb-cite_ref-48\" class=\"reference\"><a href=\"#cite_note-48\">[48]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-49\" class=\"reference\"><a href=\"#cite_note-49\">[49]<\/a><\/sup>, academic research institutions and projects should establish technical and administrative procedures to identify and define configuration items and SOUP, as well as their documentation within a system. This should include the documentation of problem reports, change requests, changes, and releases necessary to restore an item, determine its components, and provide the history of its changes. In particular, configuration change requests need to be documented, approved, and verified in projects with multiple developers and stakeholders.<sup id=\"rdp-ebb-cite_ref-:3_16-10\" class=\"reference\"><a href=\"#cite_note-:3-16\">[16]<\/a><\/sup> Thus, it is advised to assign a representative person, ideally permanent technical academic staff, in charge of the change management to support the correct implementation of necessary processes for groups and projects in advance.<sup id=\"rdp-ebb-cite_ref-:9_23-1\" class=\"reference\"><a href=\"#cite_note-:9-23\">[23]<\/a><\/sup> In academia, ticket systems such as those provided by most repositories can be easily used as version control systems for all code and documents to facilitate tracking changes. Moreover, tools such as Jira for project management and Confluence for project documentation are advisable to ensure traceability and good configuration management.\n<\/p><p>These tools can be utilized to establish a change management strategy or process that includes steps like:\n<\/p>\n<ol><li>Create a problem report (including criticality).<\/li>\n<li>Conduct problem analysis, including software risk.<\/li>\n<li>Create a change request, if required.<\/li>\n<li>Implement and verify the change.<\/li><\/ol>\n<h2><span class=\"mw-headline\" id=\"Discussion\">Discussion<\/span><\/h2>\n<p>The current trend toward using new technologies in the medical context<sup id=\"rdp-ebb-cite_ref-:0_1-1\" class=\"reference\"><a href=\"#cite_note-:0-1\">[1]<\/a><\/sup>, such as establishing AI- or ML-related software, paves the way to significantly improve monitoring, diagnostics, and prognostics for the patient\u2019s and medical team\u2019s sakes. The subjacent development, realized mainly by specialized research organizations and institutes, is time-consuming. Additionally, implementing these software achievements in the medical markets requires enormous efforts to cover all requirements of international standards.<sup id=\"rdp-ebb-cite_ref-:5_19-5\" class=\"reference\"><a href=\"#cite_note-:5-19\">[19]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:6_20-3\" class=\"reference\"><a href=\"#cite_note-:6-20\">[20]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:7_21-3\" class=\"reference\"><a href=\"#cite_note-:7-21\">[21]<\/a><\/sup> This article reveals several knowledge transfer challenges from industrial standards to use in academia, focusing on the software life cycle in a biomedical context. Challenges mainly face the software development and engineering processes regarding reproducibility, reusability, and traceability.\n<\/p><p>In order to better establish an academia-tailored SLC, here we propose a comprehensive guideline for research facilities derived from the requirements of IEC 62304.<sup id=\"rdp-ebb-cite_ref-:3_16-11\" class=\"reference\"><a href=\"#cite_note-:3-16\">[16]<\/a><\/sup> Complementary to our quality management guideline<sup id=\"rdp-ebb-cite_ref-50\" class=\"reference\"><a href=\"#cite_note-50\">[50]<\/a><\/sup>, we propose to address these challenges by following our guideline, which lowers the barriers to a potential technology transfer toward the medical industry. Furthermore, in the Supplemental information, we provide a comprehensive checklist for a successful SLC and demonstrate the feasibility of our guideline with our implementation example.\n<\/p><p>Since realizing industry-based regulatory requirements is mostly not feasible in an academic context, we focus on the most vital aspects of the SLC, covering software planning, development, architecture, maintenance, and legacy software. The implementation of our guidelines will not only improve the quality of medical software and avoid engineering errors but also mitigate potential risks that might arise from the introduction of AI in healthcare. An integration of SLC as standard procedure in academic programming could increase the overall quality of processing pipelines and thus the quality of data. Since evaluation strategies are well planned, this lends to a closer examination of the potential risks of false positives and false negatives. Ultimately, this supports a smooth transfer with potential manufacturers by delivering all demanded documents of a certain quality related to the software. Although some research organizations indeed have the resources to realize industrial standards such as quality management<sup id=\"rdp-ebb-cite_ref-51\" class=\"reference\"><a href=\"#cite_note-51\">[51]<\/a><\/sup>, we encourage scientists to further introduce SLC as a usual practice for software in research institutes. We envision that such a focus on SLC, in addition to focus on the FAIR principles for scientific data management and documentation, could become the standard for scientific health software publishing. Finally, this may pave the way for a smoother transition from research toward clinical practice.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Supplemental_information\">Supplemental information<\/span><\/h2>\n<ul><li><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ars.els-cdn.com\/content\/image\/1-s2.0-S2589004222018065-mmc1.pdf\" target=\"_blank\">Document S1<\/a> (PDF): Figures S1\u2013S8 and Tables S1\u2013S7.<\/li><\/ul>\n<h2><span id=\"rdp-ebb-Abbreviations,_acronyms,_and_initialisms\"><\/span><span class=\"mw-headline\" id=\"Abbreviations.2C_acronyms.2C_and_initialisms\">Abbreviations, acronyms, and initialisms<\/span><\/h2>\n<ul><li><b>AI<\/b>: artificial intelligence<\/li>\n<li><b>CDSS<\/b>: clinical decision support system<\/li>\n<li><b>CI<\/b>: continuous integration<\/li>\n<li><b>FAIR<\/b>: findable, accessible, interoperable, and reusable<\/li>\n<li><b>IMDRF<\/b>: International Medical Device Regulators Forum<\/li>\n<li><b>IVDR<\/b>: In Vitro Diagnostic Medical Devices Regulation<\/li>\n<li><b>MDR<\/b>: European Medical Devices Regulation<\/li>\n<li><b>MDSW<\/b>: medical device software<\/li>\n<li><b>ML<\/b>: machine learning<\/li>\n<li><b>MVC<\/b>: model view controller<\/li>\n<li><b>QMS<\/b>: quality management system<\/li>\n<li><b>SLC<\/b>: software life cycle<\/li>\n<li><b>SOP<\/b>: standard operating procedure<\/li>\n<li><b>SOUP<\/b>: software of unknown provenance<\/li><\/ul>\n<h2><span class=\"mw-headline\" id=\"Acknowledgements\">Acknowledgements<\/span><\/h2>\n<p>This project has received funding from the European Union\u2019s Horizon2020 research and innovation program under grant agreement No 826078. This publication reflects only the authors\u2019 view and the European Commission is not responsible for any use that may be made of the information it contains.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Author_contributions\">Author contributions<\/span><\/h3>\n<p>Conceptualization, A.-C.H., R.M., S.C.H., J.W. and D.H.; Methodology, A.-C.H., R.M., S.C.H. and J. W.; Software, A.-C.H., S.C.H. and J.W.; Investigation, A.-C.H., R.M., S.C.H. and J.W.; Writing - Original Draft, A.-C.H., R.M., S.C.H. and J.W.; Writing - Review & Editing: A.-C.H., R.M., S.C.H., J.W. and D.H.; Supervision, D.H.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Conflict_of_interest\">Conflict of interest<\/span><\/h3>\n<p>The authors declare no competing interests.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"References\">References<\/span><\/h2>\n<div class=\"reflist references-column-width\" style=\"-moz-column-width: 30em; -webkit-column-width: 30em; column-width: 30em; list-style-type: decimal;\">\n<div class=\"mw-references-wrap mw-references-columns\"><ol class=\"references\">\n<li id=\"cite_note-:0-1\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:0_1-0\">1.0<\/a><\/sup> <sup><a href=\"#cite_ref-:0_1-1\">1.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Muehlematter, Urs J; Daniore, Paola; Vokinger, Kerstin N (1 March 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2589750020302922\" target=\"_blank\">\"Approval of artificial intelligence and machine learning-based medical devices in the USA and Europe (2015\u201320): a comparative analysis\"<\/a> (in en). <i>The Lancet Digital Health<\/i> <b>3<\/b> (3): e195\u2013e203. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2FS2589-7500%2820%2930292-2\" target=\"_blank\">10.1016\/S2589-7500(20)30292-2<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2589750020302922\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2589750020302922<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Approval+of+artificial+intelligence+and+machine+learning-based+medical+devices+in+the+USA+and+Europe+%282015%E2%80%9320%29%3A+a+comparative+analysis&rft.jtitle=The+Lancet+Digital+Health&rft.aulast=Muehlematter&rft.aufirst=Urs+J&rft.au=Muehlematter%2C%26%2332%3BUrs+J&rft.au=Daniore%2C%26%2332%3BPaola&rft.au=Vokinger%2C%26%2332%3BKerstin+N&rft.date=1+March+2021&rft.volume=3&rft.issue=3&rft.pages=e195%E2%80%93e203&rft_id=info:doi\/10.1016%2FS2589-7500%2820%2930292-2&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2589750020302922&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-2\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-2\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Sutton, Reed T.; Pincock, David; Baumgart, Daniel C.; Sadowski, Daniel C.; Fedorak, Richard N.; Kroeker, Karen I. (6 February 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41746-020-0221-y\" target=\"_blank\">\"An overview of clinical decision support systems: benefits, risks, and strategies for success\"<\/a> (in en). <i>npj Digital Medicine<\/i> <b>3<\/b> (1): 17. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41746-020-0221-y\" target=\"_blank\">10.1038\/s41746-020-0221-y<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2398-6352\" target=\"_blank\">2398-6352<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC7005290\/\" target=\"_blank\">PMC7005290<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/32047862\" target=\"_blank\">32047862<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41746-020-0221-y\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41746-020-0221-y<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=An+overview+of+clinical+decision+support+systems%3A+benefits%2C+risks%2C+and+strategies+for+success&rft.jtitle=npj+Digital+Medicine&rft.aulast=Sutton&rft.aufirst=Reed+T.&rft.au=Sutton%2C%26%2332%3BReed+T.&rft.au=Pincock%2C%26%2332%3BDavid&rft.au=Baumgart%2C%26%2332%3BDaniel+C.&rft.au=Sadowski%2C%26%2332%3BDaniel+C.&rft.au=Fedorak%2C%26%2332%3BRichard+N.&rft.au=Kroeker%2C%26%2332%3BKaren+I.&rft.date=6+February+2020&rft.volume=3&rft.issue=1&rft.pages=17&rft_id=info:doi\/10.1038%2Fs41746-020-0221-y&rft.issn=2398-6352&rft_id=info:pmc\/PMC7005290&rft_id=info:pmid\/32047862&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41746-020-0221-y&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-3\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-3\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ceney, Adam; Tolond, Stephanie; Glowinski, Andrzej; Marks, Ben; Swift, Simon; Palser, Tom (15 July 2021). Wilson, Fernando A.. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/dx.plos.org\/10.1371\/journal.pone.0254088\" target=\"_blank\">\"Accuracy of online symptom checkers and the potential impact on service utilisation\"<\/a> (in en). <i>PLOS ONE<\/i> <b>16<\/b> (7): e0254088. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1371%2Fjournal.pone.0254088\" target=\"_blank\">10.1371\/journal.pone.0254088<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1932-6203\" target=\"_blank\">1932-6203<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC8282353\/\" target=\"_blank\">PMC8282353<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/34265845\" target=\"_blank\">34265845<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/dx.plos.org\/10.1371\/journal.pone.0254088\" target=\"_blank\">https:\/\/dx.plos.org\/10.1371\/journal.pone.0254088<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Accuracy+of+online+symptom+checkers+and+the+potential+impact+on+service+utilisation&rft.jtitle=PLOS+ONE&rft.aulast=Ceney&rft.aufirst=Adam&rft.au=Ceney%2C%26%2332%3BAdam&rft.au=Tolond%2C%26%2332%3BStephanie&rft.au=Glowinski%2C%26%2332%3BAndrzej&rft.au=Marks%2C%26%2332%3BBen&rft.au=Swift%2C%26%2332%3BSimon&rft.au=Palser%2C%26%2332%3BTom&rft.date=15+July+2021&rft.volume=16&rft.issue=7&rft.pages=e0254088&rft_id=info:doi\/10.1371%2Fjournal.pone.0254088&rft.issn=1932-6203&rft_id=info:pmc\/PMC8282353&rft_id=info:pmid\/34265845&rft_id=https%3A%2F%2Fdx.plos.org%2F10.1371%2Fjournal.pone.0254088&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-4\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-4\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Anastasiou, Olympia E.; K\u00e4lsch, Julia; Hakmouni, Mahdi; Kucukoglu, Ozlem; Heider, Dominik; Korth, Johannes; Manka, Paul; Sowa, Jan-Peter <i>et al.<\/i> (1 July 2017). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/liv.13369\" target=\"_blank\">\"Low transferrin and high ferritin concentrations are associated with worse outcome in acute liver failure\"<\/a> (in en). <i>Liver International<\/i> <b>37<\/b> (7): 1032\u20131041. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1111%2Fliv.13369\" target=\"_blank\">10.1111\/liv.13369<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/liv.13369\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/liv.13369<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Low+transferrin+and+high+ferritin+concentrations+are+associated+with+worse+outcome+in+acute+liver+failure&rft.jtitle=Liver+International&rft.aulast=Anastasiou&rft.aufirst=Olympia+E.&rft.au=Anastasiou%2C%26%2332%3BOlympia+E.&rft.au=K%C3%A4lsch%2C%26%2332%3BJulia&rft.au=Hakmouni%2C%26%2332%3BMahdi&rft.au=Kucukoglu%2C%26%2332%3BOzlem&rft.au=Heider%2C%26%2332%3BDominik&rft.au=Korth%2C%26%2332%3BJohannes&rft.au=Manka%2C%26%2332%3BPaul&rft.au=Sowa%2C%26%2332%3BJan-Peter&rft.au=Bechmann%2C%26%2332%3BLars&rft.date=1+July+2017&rft.volume=37&rft.issue=7&rft.pages=1032%E2%80%931041&rft_id=info:doi\/10.1111%2Fliv.13369&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1111%2Fliv.13369&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-5\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-5\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Riemenschneider, Mona; Hummel, Thomas; Heider, Dominik (1 December 2016). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/s12859-016-1179-2\" target=\"_blank\">\"SHIVA - a web application for drug resistance and tropism testing in HIV\"<\/a> (in en). <i>BMC Bioinformatics<\/i> <b>17<\/b> (1): 314. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1186%2Fs12859-016-1179-2\" target=\"_blank\">10.1186\/s12859-016-1179-2<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1471-2105\" target=\"_blank\">1471-2105<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC4994198\/\" target=\"_blank\">PMC4994198<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/27549230\" target=\"_blank\">27549230<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/s12859-016-1179-2\" target=\"_blank\">http:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/s12859-016-1179-2<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=SHIVA+-+a+web+application+for+drug+resistance+and+tropism+testing+in+HIV&rft.jtitle=BMC+Bioinformatics&rft.aulast=Riemenschneider&rft.aufirst=Mona&rft.au=Riemenschneider%2C%26%2332%3BMona&rft.au=Hummel%2C%26%2332%3BThomas&rft.au=Heider%2C%26%2332%3BDominik&rft.date=1+December+2016&rft.volume=17&rft.issue=1&rft.pages=314&rft_id=info:doi\/10.1186%2Fs12859-016-1179-2&rft.issn=1471-2105&rft_id=info:pmc\/PMC4994198&rft_id=info:pmid\/27549230&rft_id=http%3A%2F%2Fbmcbioinformatics.biomedcentral.com%2Farticles%2F10.1186%2Fs12859-016-1179-2&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-6\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-6\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Riemenschneider, Mona; Cashin, Kieran Y.; Budeus, Bettina; Sierra, Saleta; Shirvani-Dastgerdi, Elham; Bayanolhagh, Saeed; Kaiser, Rolf; Gorry, Paul R. <i>et al.<\/i> (29 April 2016). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/srep24883\" target=\"_blank\">\"Genotypic Prediction of Co-receptor Tropism of HIV-1 Subtypes A and C\"<\/a> (in en). <i>Scientific Reports<\/i> <b>6<\/b> (1): 24883. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fsrep24883\" target=\"_blank\">10.1038\/srep24883<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2045-2322\" target=\"_blank\">2045-2322<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC4850382\/\" target=\"_blank\">PMC4850382<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/27126912\" target=\"_blank\">27126912<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/srep24883\" target=\"_blank\">https:\/\/www.nature.com\/articles\/srep24883<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Genotypic+Prediction+of+Co-receptor+Tropism+of+HIV-1+Subtypes+A+and+C&rft.jtitle=Scientific+Reports&rft.aulast=Riemenschneider&rft.aufirst=Mona&rft.au=Riemenschneider%2C%26%2332%3BMona&rft.au=Cashin%2C%26%2332%3BKieran+Y.&rft.au=Budeus%2C%26%2332%3BBettina&rft.au=Sierra%2C%26%2332%3BSaleta&rft.au=Shirvani-Dastgerdi%2C%26%2332%3BElham&rft.au=Bayanolhagh%2C%26%2332%3BSaeed&rft.au=Kaiser%2C%26%2332%3BRolf&rft.au=Gorry%2C%26%2332%3BPaul+R.&rft.au=Heider%2C%26%2332%3BDominik&rft.date=29+April+2016&rft.volume=6&rft.issue=1&rft.pages=24883&rft_id=info:doi\/10.1038%2Fsrep24883&rft.issn=2045-2322&rft_id=info:pmc\/PMC4850382&rft_id=info:pmid\/27126912&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fsrep24883&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-7\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-7\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Rieke, Nicola; Hancox, Jonny; Li, Wenqi; Milletar\u00ec, Fausto; Roth, Holger R.; Albarqouni, Shadi; Bakas, Spyridon; Galtier, Mathieu N. <i>et al.<\/i> (14 September 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41746-020-00323-1\" target=\"_blank\">\"The future of digital health with federated learning\"<\/a> (in en). <i>npj Digital Medicine<\/i> <b>3<\/b> (1): 119. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41746-020-00323-1\" target=\"_blank\">10.1038\/s41746-020-00323-1<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2398-6352\" target=\"_blank\">2398-6352<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC7490367\/\" target=\"_blank\">PMC7490367<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/33015372\" target=\"_blank\">33015372<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41746-020-00323-1\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41746-020-00323-1<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+future+of+digital+health+with+federated+learning&rft.jtitle=npj+Digital+Medicine&rft.aulast=Rieke&rft.aufirst=Nicola&rft.au=Rieke%2C%26%2332%3BNicola&rft.au=Hancox%2C%26%2332%3BJonny&rft.au=Li%2C%26%2332%3BWenqi&rft.au=Milletar%C3%AC%2C%26%2332%3BFausto&rft.au=Roth%2C%26%2332%3BHolger+R.&rft.au=Albarqouni%2C%26%2332%3BShadi&rft.au=Bakas%2C%26%2332%3BSpyridon&rft.au=Galtier%2C%26%2332%3BMathieu+N.&rft.au=Landman%2C%26%2332%3BBennett+A.&rft.date=14+September+2020&rft.volume=3&rft.issue=1&rft.pages=119&rft_id=info:doi\/10.1038%2Fs41746-020-00323-1&rft.issn=2398-6352&rft_id=info:pmc\/PMC7490367&rft_id=info:pmid\/33015372&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41746-020-00323-1&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-8\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-8\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Hauschild, Anne-Christin; Lemanczyk, Marta; Matschinske, Julian; Frisch, Tobias; Zolotareva, Olga; Holzinger, Andreas; Baumbach, Jan; Heider, Dominik (12 April 2022). Wren, Jonathan. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/academic.oup.com\/bioinformatics\/article\/38\/8\/2278\/6525214\" target=\"_blank\">\"Federated Random Forests can improve local performance of predictive models for various healthcare applications\"<\/a> (in en). <i>Bioinformatics<\/i> <b>38<\/b> (8): 2278\u20132286. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1093%2Fbioinformatics%2Fbtac065\" target=\"_blank\">10.1093\/bioinformatics\/btac065<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1367-4803\" target=\"_blank\">1367-4803<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/academic.oup.com\/bioinformatics\/article\/38\/8\/2278\/6525214\" target=\"_blank\">https:\/\/academic.oup.com\/bioinformatics\/article\/38\/8\/2278\/6525214<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Federated+Random+Forests+can+improve+local+performance+of+predictive+models+for+various+healthcare+applications&rft.jtitle=Bioinformatics&rft.aulast=Hauschild&rft.aufirst=Anne-Christin&rft.au=Hauschild%2C%26%2332%3BAnne-Christin&rft.au=Lemanczyk%2C%26%2332%3BMarta&rft.au=Matschinske%2C%26%2332%3BJulian&rft.au=Frisch%2C%26%2332%3BTobias&rft.au=Zolotareva%2C%26%2332%3BOlga&rft.au=Holzinger%2C%26%2332%3BAndreas&rft.au=Baumbach%2C%26%2332%3BJan&rft.au=Heider%2C%26%2332%3BDominik&rft.date=12+April+2022&rft.volume=38&rft.issue=8&rft.pages=2278%E2%80%932286&rft_id=info:doi\/10.1093%2Fbioinformatics%2Fbtac065&rft.issn=1367-4803&rft_id=https%3A%2F%2Facademic.oup.com%2Fbioinformatics%2Farticle%2F38%2F8%2F2278%2F6525214&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-9\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-9\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Warnat-Herresthal, Stefanie; Schultze, Hartmut; Shastry, Krishnaprasad Lingadahalli; Manamohan, Sathyanarayanan; Mukherjee, Saikat; Garg, Vishesh; Sarveswara, Ravi; H\u00e4ndler, Kristian <i>et al.<\/i> (10 June 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/s41586-021-03583-3\" target=\"_blank\">\"Swarm Learning for decentralized and confidential clinical machine learning\"<\/a> (in en). <i>Nature<\/i> <b>594<\/b> (7862): 265\u2013270. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fs41586-021-03583-3\" target=\"_blank\">10.1038\/s41586-021-03583-3<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0028-0836\" target=\"_blank\">0028-0836<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC8189907\/\" target=\"_blank\">PMC8189907<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/34040261\" target=\"_blank\">34040261<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/s41586-021-03583-3\" target=\"_blank\">https:\/\/www.nature.com\/articles\/s41586-021-03583-3<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Swarm+Learning+for+decentralized+and+confidential+clinical+machine+learning&rft.jtitle=Nature&rft.aulast=Warnat-Herresthal&rft.aufirst=Stefanie&rft.au=Warnat-Herresthal%2C%26%2332%3BStefanie&rft.au=Schultze%2C%26%2332%3BHartmut&rft.au=Shastry%2C%26%2332%3BKrishnaprasad+Lingadahalli&rft.au=Manamohan%2C%26%2332%3BSathyanarayanan&rft.au=Mukherjee%2C%26%2332%3BSaikat&rft.au=Garg%2C%26%2332%3BVishesh&rft.au=Sarveswara%2C%26%2332%3BRavi&rft.au=H%C3%A4ndler%2C%26%2332%3BKristian&rft.au=Pickkers%2C%26%2332%3BPeter&rft.date=10+June+2021&rft.volume=594&rft.issue=7862&rft.pages=265%E2%80%93270&rft_id=info:doi\/10.1038%2Fs41586-021-03583-3&rft.issn=0028-0836&rft_id=info:pmc\/PMC8189907&rft_id=info:pmid\/34040261&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41586-021-03583-3&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-10\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-10\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Torkzadehmahani, Reihaneh; Nasirigerdeh, Reza; Blumenthal, David B.; Kacprowski, Tim; List, Markus; Matschinske, Julian; Spaeth, Julian; Wenke, Nina Kerstin <i>et al.<\/i> (1 June 2022). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.1055\/s-0041-1740630\" target=\"_blank\">\"Privacy-Preserving Artificial Intelligence Techniques in Biomedicine\"<\/a> (in en). <i>Methods of Information in Medicine<\/i> <b>61<\/b> (S 01): e12\u2013e27. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1055%2Fs-0041-1740630\" target=\"_blank\">10.1055\/s-0041-1740630<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0026-1270\" target=\"_blank\">0026-1270<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC9246509\/\" target=\"_blank\">PMC9246509<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/35062032\" target=\"_blank\">35062032<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.1055\/s-0041-1740630\" target=\"_blank\">http:\/\/www.thieme-connect.de\/DOI\/DOI?10.1055\/s-0041-1740630<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Privacy-Preserving+Artificial+Intelligence+Techniques+in+Biomedicine&rft.jtitle=Methods+of+Information+in+Medicine&rft.aulast=Torkzadehmahani&rft.aufirst=Reihaneh&rft.au=Torkzadehmahani%2C%26%2332%3BReihaneh&rft.au=Nasirigerdeh%2C%26%2332%3BReza&rft.au=Blumenthal%2C%26%2332%3BDavid+B.&rft.au=Kacprowski%2C%26%2332%3BTim&rft.au=List%2C%26%2332%3BMarkus&rft.au=Matschinske%2C%26%2332%3BJulian&rft.au=Spaeth%2C%26%2332%3BJulian&rft.au=Wenke%2C%26%2332%3BNina+Kerstin&rft.au=Baumbach%2C%26%2332%3BJan&rft.date=1+June+2022&rft.volume=61&rft.issue=S+01&rft.pages=e12%E2%80%93e27&rft_id=info:doi\/10.1055%2Fs-0041-1740630&rft.issn=0026-1270&rft_id=info:pmc\/PMC9246509&rft_id=info:pmid\/35062032&rft_id=http%3A%2F%2Fwww.thieme-connect.de%2FDOI%2FDOI%3F10.1055%2Fs-0041-1740630&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-11\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-11\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Nasirigerdeh, Reza; Torkzadehmahani, Reihaneh; Matschinske, Julian; Frisch, Tobias; List, Markus; Sp\u00e4th, Julian; Weiss, Stefan; V\u00f6lker, Uwe <i>et al.<\/i> (24 January 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/genomebiology.biomedcentral.com\/articles\/10.1186\/s13059-021-02562-1\" target=\"_blank\">\"sPLINK: a hybrid federated tool as a robust alternative to meta-analysis in genome-wide association studies\"<\/a> (in en). <i>Genome Biology<\/i> <b>23<\/b> (1): 32. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1186%2Fs13059-021-02562-1\" target=\"_blank\">10.1186\/s13059-021-02562-1<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1474-760X\" target=\"_blank\">1474-760X<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC8785575\/\" target=\"_blank\">PMC8785575<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/35073941\" target=\"_blank\">35073941<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/genomebiology.biomedcentral.com\/articles\/10.1186\/s13059-021-02562-1\" target=\"_blank\">https:\/\/genomebiology.biomedcentral.com\/articles\/10.1186\/s13059-021-02562-1<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=sPLINK%3A+a+hybrid+federated+tool+as+a+robust+alternative+to+meta-analysis+in+genome-wide+association+studies&rft.jtitle=Genome+Biology&rft.aulast=Nasirigerdeh&rft.aufirst=Reza&rft.au=Nasirigerdeh%2C%26%2332%3BReza&rft.au=Torkzadehmahani%2C%26%2332%3BReihaneh&rft.au=Matschinske%2C%26%2332%3BJulian&rft.au=Frisch%2C%26%2332%3BTobias&rft.au=List%2C%26%2332%3BMarkus&rft.au=Sp%C3%A4th%2C%26%2332%3BJulian&rft.au=Weiss%2C%26%2332%3BStefan&rft.au=V%C3%B6lker%2C%26%2332%3BUwe&rft.au=Pitk%C3%A4nen%2C%26%2332%3BEsa&rft.date=24+January+2022&rft.volume=23&rft.issue=1&rft.pages=32&rft_id=info:doi\/10.1186%2Fs13059-021-02562-1&rft.issn=1474-760X&rft_id=info:pmc\/PMC8785575&rft_id=info:pmid\/35073941&rft_id=https%3A%2F%2Fgenomebiology.biomedcentral.com%2Farticles%2F10.1186%2Fs13059-021-02562-1&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:1-12\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:1_12-0\">12.0<\/a><\/sup> <sup><a href=\"#cite_ref-:1_12-1\">12.1<\/a><\/sup> <sup><a href=\"#cite_ref-:1_12-2\">12.2<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"http:\/\/data.europa.eu\/eli\/reg\/2017\/745\/oj\" target=\"_blank\">\"Regulation (EU) 2017\/745 of the European Parliament and of the Council of 5 April 2017 on medical devices\"<\/a>. <i>EUR-Lex<\/i>. Publications Office of the European Union. 5 April 2017<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/data.europa.eu\/eli\/reg\/2017\/745\/oj\" target=\"_blank\">http:\/\/data.europa.eu\/eli\/reg\/2017\/745\/oj<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Regulation+%28EU%29+2017%2F745+of+the+European+Parliament+and+of+the+Council+of+5+April+2017+on+medical+devices&rft.atitle=EUR-Lex&rft.date=5+April+2017&rft.pub=Publications+Office+of+the+European+Union&rft_id=http%3A%2F%2Fdata.europa.eu%2Feli%2Freg%2F2017%2F745%2Foj&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:2-13\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:2_13-0\">13.0<\/a><\/sup> <sup><a href=\"#cite_ref-:2_13-1\">13.1<\/a><\/sup> <sup><a href=\"#cite_ref-:2_13-2\">13.2<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation web\">Medical Device Coordination Group (October 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/health.ec.europa.eu\/system\/files\/2020-09\/md_mdcg_2019_11_guidance_qualification_classification_software_en_0.pdf\" target=\"_blank\">\"MDCG 2019-11 Guidance on Qualification and Classification of Software in Regulation (EU) 2017\/745 \u2013 MDR and Regulation (EU) 2017\/746 \u2013 IVDR\"<\/a> (PDF)<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/health.ec.europa.eu\/system\/files\/2020-09\/md_mdcg_2019_11_guidance_qualification_classification_software_en_0.pdf\" target=\"_blank\">https:\/\/health.ec.europa.eu\/system\/files\/2020-09\/md_mdcg_2019_11_guidance_qualification_classification_software_en_0.pdf<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=MDCG+2019-11+Guidance+on+Qualification+and+Classification+of+Software+in+Regulation+%28EU%29+2017%2F745+%E2%80%93+MDR+and+Regulation+%28EU%29+2017%2F746+%E2%80%93+IVDR&rft.atitle=&rft.aulast=Medical+Device+Coordination+Group&rft.au=Medical+Device+Coordination+Group&rft.date=October+2019&rft_id=https%3A%2F%2Fhealth.ec.europa.eu%2Fsystem%2Ffiles%2F2020-09%2Fmd_mdcg_2019_11_guidance_qualification_classification_software_en_0.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-14\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-14\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">International Medical Device Regulators Forum SaMD Working Group (9 December 2013). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.imdrf.org\/sites\/default\/files\/docs\/imdrf\/final\/technical\/imdrf-tech-131209-samd-key-definitions-140901.pdf\" target=\"_blank\">\"Software as a Medical Device (SaMD): Key Definitions\"<\/a> (PDF). International Medical Device Regulators Forum<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.imdrf.org\/sites\/default\/files\/docs\/imdrf\/final\/technical\/imdrf-tech-131209-samd-key-definitions-140901.pdf\" target=\"_blank\">https:\/\/www.imdrf.org\/sites\/default\/files\/docs\/imdrf\/final\/technical\/imdrf-tech-131209-samd-key-definitions-140901.pdf<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Software+as+a+Medical+Device+%28SaMD%29%3A+Key+Definitions&rft.atitle=&rft.aulast=International+Medical+Device+Regulators+Forum+SaMD+Working+Group&rft.au=International+Medical+Device+Regulators+Forum+SaMD+Working+Group&rft.date=9+December+2013&rft.pub=International+Medical+Device+Regulators+Forum&rft_id=https%3A%2F%2Fwww.imdrf.org%2Fsites%2Fdefault%2Ffiles%2Fdocs%2Fimdrf%2Ffinal%2Ftechnical%2Fimdrf-tech-131209-samd-key-definitions-140901.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-15\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-15\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Oen, R.D.R. (2009). \"Software als medizinprodukt\". <i>Medizin Produkte Recht<\/i> <b>2<\/b>: 55\u20137.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Software+als+medizinprodukt&rft.jtitle=Medizin+Produkte+Recht&rft.aulast=Oen%2C+R.D.R.&rft.au=Oen%2C+R.D.R.&rft.date=2009&rft.volume=2&rft.pages=55%E2%80%937&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:3-16\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:3_16-0\">16.00<\/a><\/sup> <sup><a href=\"#cite_ref-:3_16-1\">16.01<\/a><\/sup> <sup><a href=\"#cite_ref-:3_16-2\">16.02<\/a><\/sup> <sup><a href=\"#cite_ref-:3_16-3\">16.03<\/a><\/sup> <sup><a href=\"#cite_ref-:3_16-4\">16.04<\/a><\/sup> <sup><a href=\"#cite_ref-:3_16-5\">16.05<\/a><\/sup> <sup><a href=\"#cite_ref-:3_16-6\">16.06<\/a><\/sup> <sup><a href=\"#cite_ref-:3_16-7\">16.07<\/a><\/sup> <sup><a href=\"#cite_ref-:3_16-8\">16.08<\/a><\/sup> <sup><a href=\"#cite_ref-:3_16-9\">16.09<\/a><\/sup> <sup><a href=\"#cite_ref-:3_16-10\">16.10<\/a><\/sup> <sup><a href=\"#cite_ref-:3_16-11\">16.11<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.iso.org\/standard\/64686.html\" target=\"_blank\">\"IEC 62304:2006\/Amd 1:2015 Medical device software \u2014 Software life cycle processes \u2014 Amendment 1\"<\/a>. International Organization for Standardization. June 2015<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.iso.org\/standard\/64686.html\" target=\"_blank\">https:\/\/www.iso.org\/standard\/64686.html<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=IEC+62304%3A2006%2FAmd+1%3A2015+Medical+device+software+%E2%80%94+Software+life+cycle+processes+%E2%80%94+Amendment+1&rft.atitle=&rft.date=June+2015&rft.pub=International+Organization+for+Standardization&rft_id=https%3A%2F%2Fwww.iso.org%2Fstandard%2F64686.html&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-17\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-17\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.accessdata.fda.gov\/scripts\/cdrh\/cfdocs\/cfstandards\/detail.cfm?standard__identification_no=38829\" target=\"_blank\">\"IEC 62304 Edition 1.1 2015-06 CONSOLIDATED VERSION Medical device software - Software life cycle processes\"<\/a>. <i>Recognized Consensus Standards<\/i>. U.S. Food and Drug Administration. 14 January 2019<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.accessdata.fda.gov\/scripts\/cdrh\/cfdocs\/cfstandards\/detail.cfm?standard__identification_no=38829\" target=\"_blank\">https:\/\/www.accessdata.fda.gov\/scripts\/cdrh\/cfdocs\/cfstandards\/detail.cfm?standard__identification_no=38829<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=IEC+62304+Edition+1.1+2015-06+CONSOLIDATED+VERSION+Medical+device+software+-+Software+life+cycle+processes&rft.atitle=Recognized+Consensus+Standards&rft.date=14+January+2019&rft.pub=U.S.+Food+and+Drug+Administration&rft_id=https%3A%2F%2Fwww.accessdata.fda.gov%2Fscripts%2Fcdrh%2Fcfdocs%2Fcfstandards%2Fdetail.cfm%3Fstandard__identification_no%3D38829&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:4-18\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:4_18-0\">18.0<\/a><\/sup> <sup><a href=\"#cite_ref-:4_18-1\">18.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Pinto, Gustavo; Wiese, Igor; Dias, Luiz Felipe (1 March 2018). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/ieeexplore.ieee.org\/document\/8330263\/\" target=\"_blank\">\"How do scientists develop scientific software? An external replication\"<\/a>. <i>2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER)<\/i> (Campobasso: IEEE): 582\u2013591. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FSANER.2018.8330263\" target=\"_blank\">10.1109\/SANER.2018.8330263<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-5386-4969-5<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/ieeexplore.ieee.org\/document\/8330263\/\" target=\"_blank\">http:\/\/ieeexplore.ieee.org\/document\/8330263\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=How+do+scientists+develop+scientific+software%3F+An+external+replication&rft.jtitle=2018+IEEE+25th+International+Conference+on+Software+Analysis%2C+Evolution+and+Reengineering+%28SANER%29&rft.aulast=Pinto&rft.aufirst=Gustavo&rft.au=Pinto%2C%26%2332%3BGustavo&rft.au=Wiese%2C%26%2332%3BIgor&rft.au=Dias%2C%26%2332%3BLuiz+Felipe&rft.date=1+March+2018&rft.pages=582%E2%80%93591&rft.place=Campobasso&rft.pub=IEEE&rft_id=info:doi\/10.1109%2FSANER.2018.8330263&rft.isbn=978-1-5386-4969-5&rft_id=http%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F8330263%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:5-19\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:5_19-0\">19.0<\/a><\/sup> <sup><a href=\"#cite_ref-:5_19-1\">19.1<\/a><\/sup> <sup><a href=\"#cite_ref-:5_19-2\">19.2<\/a><\/sup> <sup><a href=\"#cite_ref-:5_19-3\">19.3<\/a><\/sup> <sup><a href=\"#cite_ref-:5_19-4\">19.4<\/a><\/sup> <sup><a href=\"#cite_ref-:5_19-5\">19.5<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Brito, Jaqueline J; Li, Jun; Moore, Jason H; Greene, Casey S; Nogoy, Nicole A; Garmire, Lana X; Mangul, Serghei (1 June 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/academic.oup.com\/gigascience\/article\/doi\/10.1093\/gigascience\/giaa056\/5849489\" target=\"_blank\">\"Recommendations to enhance rigor and reproducibility in biomedical research\"<\/a> (in en). <i>GigaScience<\/i> <b>9<\/b> (6): giaa056. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1093%2Fgigascience%2Fgiaa056\" target=\"_blank\">10.1093\/gigascience\/giaa056<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2047-217X\" target=\"_blank\">2047-217X<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC7263079\/\" target=\"_blank\">PMC7263079<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/32479592\" target=\"_blank\">32479592<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/academic.oup.com\/gigascience\/article\/doi\/10.1093\/gigascience\/giaa056\/5849489\" target=\"_blank\">https:\/\/academic.oup.com\/gigascience\/article\/doi\/10.1093\/gigascience\/giaa056\/5849489<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Recommendations+to+enhance+rigor+and+reproducibility+in+biomedical+research&rft.jtitle=GigaScience&rft.aulast=Brito&rft.aufirst=Jaqueline+J&rft.au=Brito%2C%26%2332%3BJaqueline+J&rft.au=Li%2C%26%2332%3BJun&rft.au=Moore%2C%26%2332%3BJason+H&rft.au=Greene%2C%26%2332%3BCasey+S&rft.au=Nogoy%2C%26%2332%3BNicole+A&rft.au=Garmire%2C%26%2332%3BLana+X&rft.au=Mangul%2C%26%2332%3BSerghei&rft.date=1+June+2020&rft.volume=9&rft.issue=6&rft.pages=giaa056&rft_id=info:doi\/10.1093%2Fgigascience%2Fgiaa056&rft.issn=2047-217X&rft_id=info:pmc\/PMC7263079&rft_id=info:pmid\/32479592&rft_id=https%3A%2F%2Facademic.oup.com%2Fgigascience%2Farticle%2Fdoi%2F10.1093%2Fgigascience%2Fgiaa056%2F5849489&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:6-20\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:6_20-0\">20.0<\/a><\/sup> <sup><a href=\"#cite_ref-:6_20-1\">20.1<\/a><\/sup> <sup><a href=\"#cite_ref-:6_20-2\">20.2<\/a><\/sup> <sup><a href=\"#cite_ref-:6_20-3\">20.3<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Mangul, Serghei; Mosqueiro, Thiago; Abdill, Richard J.; Duong, Dat; Mitchell, Keith; Sarwal, Varuni; Hill, Brian; Brito, Jaqueline <i>et al.<\/i> (20 June 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/dx.plos.org\/10.1371\/journal.pbio.3000333\" target=\"_blank\">\"Challenges and recommendations to improve the installability and archival stability of omics computational tools\"<\/a> (in en). <i>PLOS Biology<\/i> <b>17<\/b> (6): e3000333. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1371%2Fjournal.pbio.3000333\" target=\"_blank\">10.1371\/journal.pbio.3000333<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1545-7885\" target=\"_blank\">1545-7885<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC6605654\/\" target=\"_blank\">PMC6605654<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/31220077\" target=\"_blank\">31220077<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/dx.plos.org\/10.1371\/journal.pbio.3000333\" target=\"_blank\">https:\/\/dx.plos.org\/10.1371\/journal.pbio.3000333<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Challenges+and+recommendations+to+improve+the+installability+and+archival+stability+of+omics+computational+tools&rft.jtitle=PLOS+Biology&rft.aulast=Mangul&rft.aufirst=Serghei&rft.au=Mangul%2C%26%2332%3BSerghei&rft.au=Mosqueiro%2C%26%2332%3BThiago&rft.au=Abdill%2C%26%2332%3BRichard+J.&rft.au=Duong%2C%26%2332%3BDat&rft.au=Mitchell%2C%26%2332%3BKeith&rft.au=Sarwal%2C%26%2332%3BVaruni&rft.au=Hill%2C%26%2332%3BBrian&rft.au=Brito%2C%26%2332%3BJaqueline&rft.au=Littman%2C%26%2332%3BRussell+Jared&rft.date=20+June+2019&rft.volume=17&rft.issue=6&rft.pages=e3000333&rft_id=info:doi\/10.1371%2Fjournal.pbio.3000333&rft.issn=1545-7885&rft_id=info:pmc\/PMC6605654&rft_id=info:pmid\/31220077&rft_id=https%3A%2F%2Fdx.plos.org%2F10.1371%2Fjournal.pbio.3000333&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:7-21\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:7_21-0\">21.0<\/a><\/sup> <sup><a href=\"#cite_ref-:7_21-1\">21.1<\/a><\/sup> <sup><a href=\"#cite_ref-:7_21-2\">21.2<\/a><\/sup> <sup><a href=\"#cite_ref-:7_21-3\">21.3<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Lee, Graham; Bacon, Sebastian; Bush, Ian; Fortunato, Laura; Gavaghan, David; Lestang, Thibault; Morton, Caroline; Robinson, Martin <i>et al.<\/i> (1 February 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2666389921000167\" target=\"_blank\">\"Barely sufficient practices in scientific computing\"<\/a> (in en). <i>Patterns<\/i> <b>2<\/b> (2): 100206. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.patter.2021.100206\" target=\"_blank\">10.1016\/j.patter.2021.100206<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC7892476\/\" target=\"_blank\">PMC7892476<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/33659915\" target=\"_blank\">33659915<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2666389921000167\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2666389921000167<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Barely+sufficient+practices+in+scientific+computing&rft.jtitle=Patterns&rft.aulast=Lee&rft.aufirst=Graham&rft.au=Lee%2C%26%2332%3BGraham&rft.au=Bacon%2C%26%2332%3BSebastian&rft.au=Bush%2C%26%2332%3BIan&rft.au=Fortunato%2C%26%2332%3BLaura&rft.au=Gavaghan%2C%26%2332%3BDavid&rft.au=Lestang%2C%26%2332%3BThibault&rft.au=Morton%2C%26%2332%3BCaroline&rft.au=Robinson%2C%26%2332%3BMartin&rft.au=Rocca-Serra%2C%26%2332%3BPhilippe&rft.date=1+February+2021&rft.volume=2&rft.issue=2&rft.pages=100206&rft_id=info:doi\/10.1016%2Fj.patter.2021.100206&rft_id=info:pmc\/PMC7892476&rft_id=info:pmid\/33659915&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2666389921000167&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:8-22\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:8_22-0\">22.0<\/a><\/sup> <sup><a href=\"#cite_ref-:8_22-1\">22.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Wiese, Igor; Polato, Ivanilton; Pinto, Gustavo (1 July 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/8664473\/\" target=\"_blank\">\"Naming the Pain in Developing Scientific Software\"<\/a>. <i>IEEE Software<\/i> <b>37<\/b> (4): 75\u201382. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FMS.2019.2899838\" target=\"_blank\">10.1109\/MS.2019.2899838<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0740-7459\" target=\"_blank\">0740-7459<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/8664473\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/8664473\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Naming+the+Pain+in+Developing+Scientific+Software&rft.jtitle=IEEE+Software&rft.aulast=Wiese&rft.aufirst=Igor&rft.au=Wiese%2C%26%2332%3BIgor&rft.au=Polato%2C%26%2332%3BIvanilton&rft.au=Pinto%2C%26%2332%3BGustavo&rft.date=1+July+2020&rft.volume=37&rft.issue=4&rft.pages=75%E2%80%9382&rft_id=info:doi\/10.1109%2FMS.2019.2899838&rft.issn=0740-7459&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F8664473%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:9-23\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:9_23-0\">23.0<\/a><\/sup> <sup><a href=\"#cite_ref-:9_23-1\">23.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Riemenschneider, Mona; Wienbeck, Joachim; Scherag, Andr\u00e9; Heider, Dominik (1 June 2018). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.liebertpub.com\/doi\/10.1089\/sysm.2018.0002\" target=\"_blank\">\"Data Science for Molecular Diagnostics Applications: From Academia to Clinic to Industry\"<\/a> (in en). <i>Systems Medicine<\/i> <b>1<\/b> (1): 13\u201317. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1089%2Fsysm.2018.0002\" target=\"_blank\">10.1089\/sysm.2018.0002<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2573-3370\" target=\"_blank\">2573-3370<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.liebertpub.com\/doi\/10.1089\/sysm.2018.0002\" target=\"_blank\">http:\/\/www.liebertpub.com\/doi\/10.1089\/sysm.2018.0002<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Data+Science+for+Molecular+Diagnostics+Applications%3A+From+Academia+to+Clinic+to+Industry&rft.jtitle=Systems+Medicine&rft.aulast=Riemenschneider&rft.aufirst=Mona&rft.au=Riemenschneider%2C%26%2332%3BMona&rft.au=Wienbeck%2C%26%2332%3BJoachim&rft.au=Scherag%2C%26%2332%3BAndr%C3%A9&rft.au=Heider%2C%26%2332%3BDominik&rft.date=1+June+2018&rft.volume=1&rft.issue=1&rft.pages=13%E2%80%9317&rft_id=info:doi\/10.1089%2Fsysm.2018.0002&rft.issn=2573-3370&rft_id=http%3A%2F%2Fwww.liebertpub.com%2Fdoi%2F10.1089%2Fsysm.2018.0002&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-24\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-24\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Coiera, Enrico; Ammenwerth, Elske; Georgiou, Andrew; Magrabi, Farah (1 August 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/academic.oup.com\/jamia\/article\/25\/8\/963\/4970161\" target=\"_blank\">\"Does health informatics have a replication crisis?\"<\/a> (in en). <i>Journal of the American Medical Informatics Association<\/i> <b>25<\/b> (8): 963\u2013968. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1093%2Fjamia%2Focy028\" target=\"_blank\">10.1093\/jamia\/ocy028<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1067-5027\" target=\"_blank\">1067-5027<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC6077781\/\" target=\"_blank\">PMC6077781<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/29669066\" target=\"_blank\">29669066<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/academic.oup.com\/jamia\/article\/25\/8\/963\/4970161\" target=\"_blank\">https:\/\/academic.oup.com\/jamia\/article\/25\/8\/963\/4970161<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Does+health+informatics+have+a+replication+crisis%3F&rft.jtitle=Journal+of+the+American+Medical+Informatics+Association&rft.aulast=Coiera&rft.aufirst=Enrico&rft.au=Coiera%2C%26%2332%3BEnrico&rft.au=Ammenwerth%2C%26%2332%3BElske&rft.au=Georgiou%2C%26%2332%3BAndrew&rft.au=Magrabi%2C%26%2332%3BFarah&rft.date=1+August+2018&rft.volume=25&rft.issue=8&rft.pages=963%E2%80%93968&rft_id=info:doi\/10.1093%2Fjamia%2Focy028&rft.issn=1067-5027&rft_id=info:pmc\/PMC6077781&rft_id=info:pmid\/29669066&rft_id=https%3A%2F%2Facademic.oup.com%2Fjamia%2Farticle%2F25%2F8%2F963%2F4970161&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:10-25\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:10_25-0\">25.0<\/a><\/sup> <sup><a href=\"#cite_ref-:10_25-1\">25.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Wilkinson, Mark D.; Dumontier, Michel; Aalbersberg, IJsbrand Jan; Appleton, Gabrielle; Axton, Myles; Baak, Arie; Blomberg, Niklas; Boiten, Jan-Willem <i>et al.<\/i> (15 March 2016). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/sdata201618\" target=\"_blank\">\"The FAIR Guiding Principles for scientific data management and stewardship\"<\/a> (in en). <i>Scientific Data<\/i> <b>3<\/b> (1): 160018. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fsdata.2016.18\" target=\"_blank\">10.1038\/sdata.2016.18<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2052-4463\" target=\"_blank\">2052-4463<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC4792175\/\" target=\"_blank\">PMC4792175<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/26978244\" target=\"_blank\">26978244<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/sdata201618\" target=\"_blank\">https:\/\/www.nature.com\/articles\/sdata201618<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+FAIR+Guiding+Principles+for+scientific+data+management+and+stewardship&rft.jtitle=Scientific+Data&rft.aulast=Wilkinson&rft.aufirst=Mark+D.&rft.au=Wilkinson%2C%26%2332%3BMark+D.&rft.au=Dumontier%2C%26%2332%3BMichel&rft.au=Aalbersberg%2C%26%2332%3BIJsbrand+Jan&rft.au=Appleton%2C%26%2332%3BGabrielle&rft.au=Axton%2C%26%2332%3BMyles&rft.au=Baak%2C%26%2332%3BArie&rft.au=Blomberg%2C%26%2332%3BNiklas&rft.au=Boiten%2C%26%2332%3BJan-Willem&rft.au=da+Silva+Santos%2C%26%2332%3BLuiz+Bonino&rft.date=15+March+2016&rft.volume=3&rft.issue=1&rft.pages=160018&rft_id=info:doi\/10.1038%2Fsdata.2016.18&rft.issn=2052-4463&rft_id=info:pmc\/PMC4792175&rft_id=info:pmid\/26978244&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fsdata201618&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:11-26\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:11_26-0\">26.0<\/a><\/sup> <sup><a href=\"#cite_ref-:11_26-1\">26.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Sharma, Arjun; Blank, Anthony; Patel, Parashar; Stein, Kenneth (1 March 2013). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/s10840-013-9781-y\" target=\"_blank\">\"Health care policy and regulatory implications on medical device innovations: a cardiac rhythm medical device industry perspective\"<\/a> (in en). <i>Journal of Interventional Cardiac Electrophysiology<\/i> <b>36<\/b> (2): 107\u2013117. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2Fs10840-013-9781-y\" target=\"_blank\">10.1007\/s10840-013-9781-y<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1383-875X\" target=\"_blank\">1383-875X<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC3606523\/\" target=\"_blank\">PMC3606523<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/23474980\" target=\"_blank\">23474980<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/s10840-013-9781-y\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/s10840-013-9781-y<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Health+care+policy+and+regulatory+implications+on+medical+device+innovations%3A+a+cardiac+rhythm+medical+device+industry+perspective&rft.jtitle=Journal+of+Interventional+Cardiac+Electrophysiology&rft.aulast=Sharma&rft.aufirst=Arjun&rft.au=Sharma%2C%26%2332%3BArjun&rft.au=Blank%2C%26%2332%3BAnthony&rft.au=Patel%2C%26%2332%3BParashar&rft.au=Stein%2C%26%2332%3BKenneth&rft.date=1+March+2013&rft.volume=36&rft.issue=2&rft.pages=107%E2%80%93117&rft_id=info:doi\/10.1007%2Fs10840-013-9781-y&rft.issn=1383-875X&rft_id=info:pmc\/PMC3606523&rft_id=info:pmid\/23474980&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2Fs10840-013-9781-y&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-27\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-27\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Manghani, Kishu (2011). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.picronline.org\/text.asp?2011\/2\/1\/34\/76288\" target=\"_blank\">\"Quality assurance: Importance of systems and standard operating procedures\"<\/a> (in en). <i>Perspectives in Clinical Research<\/i> <b>2<\/b> (1): 34. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.4103%2F2229-3485.76288\" target=\"_blank\">10.4103\/2229-3485.76288<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2229-3485\" target=\"_blank\">2229-3485<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC3088954\/\" target=\"_blank\">PMC3088954<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/21584180\" target=\"_blank\">21584180<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.picronline.org\/text.asp?2011\/2\/1\/34\/76288\" target=\"_blank\">http:\/\/www.picronline.org\/text.asp?2011\/2\/1\/34\/76288<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Quality+assurance%3A+Importance+of+systems+and+standard+operating+procedures&rft.jtitle=Perspectives+in+Clinical+Research&rft.aulast=Manghani&rft.aufirst=Kishu&rft.au=Manghani%2C%26%2332%3BKishu&rft.date=2011&rft.volume=2&rft.issue=1&rft.pages=34&rft_id=info:doi\/10.4103%2F2229-3485.76288&rft.issn=2229-3485&rft_id=info:pmc\/PMC3088954&rft_id=info:pmid\/21584180&rft_id=http%3A%2F%2Fwww.picronline.org%2Ftext.asp%3F2011%2F2%2F1%2F34%2F76288&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-28\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-28\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">McHugh, Martin; Ali, Abder-Rahman; McCaffery, Fergal (2013). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/arrow.tudublin.ie\/scschcomcon\/126\/\" target=\"_blank\"><i>The Significance of Requirements in Medical Device Software Development<\/i><\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.21427%2F7ADY-5269\" target=\"_blank\">10.21427\/7ADY-5269<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/arrow.tudublin.ie\/scschcomcon\/126\/\" target=\"_blank\">https:\/\/arrow.tudublin.ie\/scschcomcon\/126\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=The+Significance+of+Requirements+in+Medical+Device+Software+Development&rft.aulast=McHugh&rft.aufirst=Martin&rft.au=McHugh%2C%26%2332%3BMartin&rft.au=Ali%2C%26%2332%3BAbder-Rahman&rft.au=McCaffery%2C%26%2332%3BFergal&rft.date=2013&rft_id=info:doi\/10.21427%2F7ADY-5269&rft_id=https%3A%2F%2Farrow.tudublin.ie%2Fscschcomcon%2F126%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-29\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-29\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Memon, M.; Jalbani, A.A.; Menghwar, G.D. et al. (2016). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.sci-int.com\/pdf\/637305175103306413.pdf\" target=\"_blank\">\"I2A: An Interoperability & Integration Architecture for Medical Device Software and eHealth Systems\"<\/a> (PDF). <i>Science International<\/i> <b>28<\/b> (4): 3783\u201387. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1013-5316\" target=\"_blank\">1013-5316<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.sci-int.com\/pdf\/637305175103306413.pdf\" target=\"_blank\">http:\/\/www.sci-int.com\/pdf\/637305175103306413.pdf<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=I2A%3A+An+Interoperability+%26+Integration+Architecture+for+Medical+Device+Software+and+eHealth+Systems&rft.jtitle=Science+International&rft.aulast=Memon%2C+M.%3B+Jalbani%2C+A.A.%3B+Menghwar%2C+G.D.+et+al.&rft.au=Memon%2C+M.%3B+Jalbani%2C+A.A.%3B+Menghwar%2C+G.D.+et+al.&rft.date=2016&rft.volume=28&rft.issue=4&rft.pages=3783%E2%80%9387&rft.issn=1013-5316&rft_id=http%3A%2F%2Fwww.sci-int.com%2Fpdf%2F637305175103306413.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-30\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-30\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">McHugh, Martin; Cawley, Oisin; McCaffcry, Fergal; Richardson, Ita; Wang, Xiaofeng (1 May 2013). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/6602471\/\" target=\"_blank\">\"An agile V-model for medical device software development to overcome the challenges with plan-driven software development lifecycles\"<\/a>. <i>2013 5th International Workshop on Software Engineering in Health Care (SEHC)<\/i> (San Francisco, CA, USA: IEEE): 12\u201319. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FSEHC.2013.6602471\" target=\"_blank\">10.1109\/SEHC.2013.6602471<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-4673-6282-5<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/6602471\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/6602471\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=An+agile+V-model+for+medical+device+software+development+to+overcome+the+challenges+with+plan-driven+software+development+lifecycles&rft.jtitle=2013+5th+International+Workshop+on+Software+Engineering+in+Health+Care+%28SEHC%29&rft.aulast=McHugh&rft.aufirst=Martin&rft.au=McHugh%2C%26%2332%3BMartin&rft.au=Cawley%2C%26%2332%3BOisin&rft.au=McCaffcry%2C%26%2332%3BFergal&rft.au=Richardson%2C%26%2332%3BIta&rft.au=Wang%2C%26%2332%3BXiaofeng&rft.date=1+May+2013&rft.pages=12%E2%80%9319&rft.place=San+Francisco%2C+CA%2C+USA&rft.pub=IEEE&rft_id=info:doi\/10.1109%2FSEHC.2013.6602471&rft.isbn=978-1-4673-6282-5&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F6602471%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-31\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-31\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation book\">Davis, Alan Mark (1993). <i>Software requirements: objects, functions, and states<\/i> (Rev ed.). Englewood Cliffs, N.J: PTR Prentice Hall. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-0-13-805763-3.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=Software+requirements%3A+objects%2C+functions%2C+and+states&rft.aulast=Davis&rft.aufirst=Alan+Mark&rft.au=Davis%2C%26%2332%3BAlan+Mark&rft.date=1993&rft.edition=Rev&rft.place=Englewood+Cliffs%2C+N.J&rft.pub=PTR+Prentice+Hall&rft.isbn=978-0-13-805763-3&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-32\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-32\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Leffingwell, D. (1997). \"Calculating your return on investment from more effective requirements management\". <i>American Programmer<\/i> <b>10<\/b> (4): 13\u201316.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Calculating+your+return+on+investment+from+more+effective+requirements+management&rft.jtitle=American+Programmer&rft.aulast=Leffingwell%2C+D.&rft.au=Leffingwell%2C+D.&rft.date=1997&rft.volume=10&rft.issue=4&rft.pages=13%E2%80%9316&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-33\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-33\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation book\">Booch, Grady; Rumbaugh, James; Jacobson, Ivar (1999). <i>The unified modeling language user guide<\/i>. The Addison-Wesley object technology series. Reading Mass: Addison-Wesley. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-0-201-57168-4.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=The+unified+modeling+language+user+guide&rft.aulast=Booch&rft.aufirst=Grady&rft.au=Booch%2C%26%2332%3BGrady&rft.au=Rumbaugh%2C%26%2332%3BJames&rft.au=Jacobson%2C%26%2332%3BIvar&rft.date=1999&rft.series=The+Addison-Wesley+object+technology+series&rft.place=Reading+Mass&rft.pub=Addison-Wesley&rft.isbn=978-0-201-57168-4&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-34\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-34\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Booch, G.; Jacobson, I.; Rumbaugh, J. et al. (4 January 2005). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.omg.org\/spec\/UML\/ISO\/19501\/PDF\" target=\"_blank\">\"Unified Modeling Language Specification Version 1.4.2\"<\/a> (PDF). Object Management Group<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.omg.org\/spec\/UML\/ISO\/19501\/PDF\" target=\"_blank\">https:\/\/www.omg.org\/spec\/UML\/ISO\/19501\/PDF<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Unified+Modeling+Language+Specification+Version+1.4.2&rft.atitle=&rft.aulast=Booch%2C+G.%3B+Jacobson%2C+I.%3B+Rumbaugh%2C+J.+et+al.&rft.au=Booch%2C+G.%3B+Jacobson%2C+I.%3B+Rumbaugh%2C+J.+et+al.&rft.date=4+January+2005&rft.pub=Object+Management+Group&rft_id=https%3A%2F%2Fwww.omg.org%2Fspec%2FUML%2FISO%2F19501%2FPDF&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-35\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-35\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Fairley, Richard E. Dick; Bourque, Pierre; Keppler, John (1 April 2014). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/6816804\/\" target=\"_blank\">\"The impact of SWEBOK Version 3 on software engineering education and training\"<\/a>. <i>2014 IEEE 27th Conference on Software Engineering Education and Training (CSEE&T)<\/i>: 192\u2013200. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FCSEET.2014.6816804\" target=\"_blank\">10.1109\/CSEET.2014.6816804<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/6816804\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/6816804\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+impact+of+SWEBOK+Version+3+on+software+engineering+education+and+training&rft.jtitle=2014+IEEE+27th+Conference+on+Software+Engineering+Education+and+Training+%28CSEE%26T%29&rft.aulast=Fairley&rft.aufirst=Richard+E.+Dick&rft.au=Fairley%2C%26%2332%3BRichard+E.+Dick&rft.au=Bourque%2C%26%2332%3BPierre&rft.au=Keppler%2C%26%2332%3BJohn&rft.date=1+April+2014&rft.pages=192%E2%80%93200&rft_id=info:doi\/10.1109%2FCSEET.2014.6816804&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F6816804%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-36\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-36\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.iso.org\/standard\/35733.html\" target=\"_blank\">\"ISO\/IEC 25010:2011 Systems and software engineering \u2014 Systems and software Quality Requirements and Evaluation (SQuaRE) \u2014 System and software quality models\"<\/a>. International Organization for Standardization. March 2011<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.iso.org\/standard\/35733.html\" target=\"_blank\">https:\/\/www.iso.org\/standard\/35733.html<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=ISO%2FIEC+25010%3A2011+Systems+and+software+engineering+%E2%80%94+Systems+and+software+Quality+Requirements+and+Evaluation+%28SQuaRE%29+%E2%80%94+System+and+software+quality+models&rft.atitle=&rft.date=March+2011&rft.pub=International+Organization+for+Standardization&rft_id=https%3A%2F%2Fwww.iso.org%2Fstandard%2F35733.html&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-37\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-37\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Mosleh, Mohsen; Dalili, Kia; Heydari, Babak (1 March 2018). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/ieeexplore.ieee.org\/document\/7539535\/\" target=\"_blank\">\"Distributed or Monolithic? A Computational Architecture Decision Framework\"<\/a>. <i>IEEE Systems Journal<\/i> <b>12<\/b> (1): 125\u2013136. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FJSYST.2016.2594290\" target=\"_blank\">10.1109\/JSYST.2016.2594290<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1932-8184\" target=\"_blank\">1932-8184<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/ieeexplore.ieee.org\/document\/7539535\/\" target=\"_blank\">http:\/\/ieeexplore.ieee.org\/document\/7539535\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Distributed+or+Monolithic%3F+A+Computational+Architecture+Decision+Framework&rft.jtitle=IEEE+Systems+Journal&rft.aulast=Mosleh&rft.aufirst=Mohsen&rft.au=Mosleh%2C%26%2332%3BMohsen&rft.au=Dalili%2C%26%2332%3BKia&rft.au=Heydari%2C%26%2332%3BBabak&rft.date=1+March+2018&rft.volume=12&rft.issue=1&rft.pages=125%E2%80%93136&rft_id=info:doi\/10.1109%2FJSYST.2016.2594290&rft.issn=1932-8184&rft_id=http%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F7539535%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-38\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-38\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation book\">Richards, Mark; Ford, Neal (2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/on1089438191\" target=\"_blank\"><i>Fundamentals of software architecture: an engineering approach<\/i><\/a> (First edition ed.). Sebastopol, CA: O'Reilly Media, Inc. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-4920-4345-4. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Online_Computer_Library_Center\" data-key=\"b53206e2204c7e657858a88b56c8ac4a\">OCLC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/oclc\/on1089438191\" target=\"_blank\">on1089438191<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/on1089438191\" target=\"_blank\">https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/on1089438191<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=Fundamentals+of+software+architecture%3A+an+engineering+approach&rft.aulast=Richards&rft.aufirst=Mark&rft.au=Richards%2C%26%2332%3BMark&rft.au=Ford%2C%26%2332%3BNeal&rft.date=2020&rft.edition=First+edition&rft.place=Sebastopol%2C+CA&rft.pub=O%27Reilly+Media%2C+Inc&rft.isbn=978-1-4920-4345-4&rft_id=info:oclcnum\/on1089438191&rft_id=https%3A%2F%2Fwww.worldcat.org%2Ftitle%2Fmediawiki%2Foclc%2Fon1089438191&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-39\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-39\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Jiang, Zhen Ming; Hassan, Ahmed E.; Holt, Richard C. (1 December 2006). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/4137451\/\" target=\"_blank\">\"Visualizing Clone Cohesion and Coupling\"<\/a>. <i>2006 13th Asia Pacific Software Engineering Conference (APSEC'06)<\/i>: 467\u2013476. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FAPSEC.2006.63\" target=\"_blank\">10.1109\/APSEC.2006.63<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/4137451\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/4137451\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Visualizing+Clone+Cohesion+and+Coupling&rft.jtitle=2006+13th+Asia+Pacific+Software+Engineering+Conference+%28APSEC%2706%29&rft.aulast=Jiang&rft.aufirst=Zhen+Ming&rft.au=Jiang%2C%26%2332%3BZhen+Ming&rft.au=Hassan%2C%26%2332%3BAhmed+E.&rft.au=Holt%2C%26%2332%3BRichard+C.&rft.date=1+December+2006&rft.pages=467%E2%80%93476&rft_id=info:doi\/10.1109%2FAPSEC.2006.63&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F4137451%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-40\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-40\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation\" id=\"rdp-ebb-CITEREFHofmeisterNordSoni1999\">Hofmeister, C.; Nord, R. L.; Soni, D. (1999), Donohoe, Patrick, ed., <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/978-0-387-35563-4_9\" target=\"_blank\">\"Describing Software Architecture with UML\"<\/a>, <i>Software Architecture<\/i> (Boston, MA: Springer US) <b>12<\/b>: 145\u2013159, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2F978-0-387-35563-4_9\" target=\"_blank\">10.1007\/978-0-387-35563-4_9<\/a>, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-4757-6536-6<span class=\"printonly\">, <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/978-0-387-35563-4_9\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/978-0-387-35563-4_9<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 2023-07-04<\/span><\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Describing+Software+Architecture+with+UML&rft.jtitle=Software+Architecture&rft.aulast=Hofmeister&rft.aufirst=C.&rft.au=Hofmeister%2C%26%2332%3BC.&rft.au=Nord%2C%26%2332%3BR.+L.&rft.au=Soni%2C%26%2332%3BD.&rft.date=1999&rft.volume=12&rft.pages=145%E2%80%93159&rft.place=Boston%2C+MA&rft.pub=Springer+US&rft_id=info:doi\/10.1007%2F978-0-387-35563-4_9&rft.isbn=978-1-4757-6536-6&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2F978-0-387-35563-4_9&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-41\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-41\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation\" id=\"rdp-ebb-CITEREFSp.C3.A4th2021\">Sp\u00e4th, Peter (2021), <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/978-1-4842-6280-1_1\" target=\"_blank\">\"About MVC: Model, View, Controller\"<\/a> (in en), <i>Beginning Java MVC 1.0<\/i> (Berkeley, CA: Apress): 1\u201318, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2F978-1-4842-6280-1_1\" target=\"_blank\">10.1007\/978-1-4842-6280-1_1<\/a>, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-4842-6279-5<span class=\"printonly\">, <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/978-1-4842-6280-1_1\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/978-1-4842-6280-1_1<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 2023-07-04<\/span><\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=About+MVC%3A+Model%2C+View%2C+Controller&rft.jtitle=Beginning+Java+MVC+1.0&rft.aulast=Sp%C3%A4th&rft.aufirst=Peter&rft.au=Sp%C3%A4th%2C%26%2332%3BPeter&rft.date=2021&rft.pages=1%E2%80%9318&rft.place=Berkeley%2C+CA&rft.pub=Apress&rft_id=info:doi\/10.1007%2F978-1-4842-6280-1_1&rft.isbn=978-1-4842-6279-5&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2F978-1-4842-6280-1_1&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-42\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-42\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\"> <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/servlet\/opac?punumber=2238\" target=\"_blank\"><i>IEEE Standard Glossary of Software Engineering Terminology<\/i><\/a>. IEEE. 31 December 1990. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2Fieeestd.1990.101064\" target=\"_blank\">10.1109\/ieeestd.1990.101064<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-0-7381-0391-4<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/servlet\/opac?punumber=2238\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/servlet\/opac?punumber=2238<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=IEEE+Standard+Glossary+of+Software+Engineering+Terminology&rft.date=31+December+1990&rft.pub=IEEE&rft_id=info:doi\/10.1109%2Fieeestd.1990.101064&rft.isbn=978-0-7381-0391-4&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fservlet%2Fopac%3Fpunumber%3D2238&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-43\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-43\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation book\">Abran, Alain; Moore, James W., eds. (2001). <i>Guide to the software engineering body of knowledge: trial version [version 0.95]; SWEBOK*; a project of the Software Engineering Coordinating Commettee<\/i>. Los Alamitos, Calif.: IEEE Computer Society. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-0-7695-1000-2.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=Guide+to+the+software+engineering+body+of+knowledge%3A+trial+version+%5Bversion+0.95%5D%3B+SWEBOK%2A%3B+a+project+of+the+Software+Engineering+Coordinating+Commettee&rft.date=2001&rft.place=Los+Alamitos%2C+Calif.&rft.pub=IEEE+Computer+Society&rft.isbn=978-0-7695-1000-2&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-44\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-44\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Nidhra, Srinivas (30 June 2012). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.airccse.org\/journal\/ijesa\/papers\/2212ijesa04.pdf\" target=\"_blank\">\"Black Box and White Box Testing Techniques - A Literature Review\"<\/a>. <i>International Journal of Embedded Systems and Applications<\/i> <b>2<\/b> (2): 29\u201350. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.5121%2Fijesa.2012.2204\" target=\"_blank\">10.5121\/ijesa.2012.2204<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.airccse.org\/journal\/ijesa\/papers\/2212ijesa04.pdf\" target=\"_blank\">http:\/\/www.airccse.org\/journal\/ijesa\/papers\/2212ijesa04.pdf<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Black+Box+and+White+Box+Testing+Techniques+-+A+Literature+Review&rft.jtitle=International+Journal+of+Embedded+Systems+and+Applications&rft.aulast=Nidhra&rft.aufirst=Srinivas&rft.au=Nidhra%2C%26%2332%3BSrinivas&rft.date=30+June+2012&rft.volume=2&rft.issue=2&rft.pages=29%E2%80%9350&rft_id=info:doi\/10.5121%2Fijesa.2012.2204&rft_id=http%3A%2F%2Fwww.airccse.org%2Fjournal%2Fijesa%2Fpapers%2F2212ijesa04.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-45\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-45\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Murnane, T.; Reed, K. (2001). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/ieeexplore.ieee.org\/document\/948492\/\" target=\"_blank\">\"On the effectiveness of mutation analysis as a black box testing technique\"<\/a>. <i>Proceedings 2001 Australian Software Engineering Conference<\/i> (Canberra, ACT, Australia: IEEE Comput. Soc): 12\u201320. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FASWEC.2001.948492\" target=\"_blank\">10.1109\/ASWEC.2001.948492<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-0-7695-1254-9<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/ieeexplore.ieee.org\/document\/948492\/\" target=\"_blank\">http:\/\/ieeexplore.ieee.org\/document\/948492\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=On+the+effectiveness+of+mutation+analysis+as+a+black+box+testing+technique&rft.jtitle=Proceedings+2001+Australian+Software+Engineering+Conference&rft.aulast=Murnane&rft.aufirst=T.&rft.au=Murnane%2C%26%2332%3BT.&rft.au=Reed%2C%26%2332%3BK.&rft.date=2001&rft.pages=12%E2%80%9320&rft.place=Canberra%2C+ACT%2C+Australia&rft.pub=IEEE+Comput.+Soc&rft_id=info:doi\/10.1109%2FASWEC.2001.948492&rft.isbn=978-0-7695-1254-9&rft_id=http%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F948492%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-46\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-46\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation book\">Myers, Glenford J.; Sandler, Corey; Badgett, Tom (2012). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/728656684\" target=\"_blank\"><i>The art of software testing<\/i><\/a> (3rd ed ed.). Hoboken, N.J: John Wiley & Sons. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-118-03196-4. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Online_Computer_Library_Center\" data-key=\"b53206e2204c7e657858a88b56c8ac4a\">OCLC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/oclc\/728656684\" target=\"_blank\">728656684<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/728656684\" target=\"_blank\">https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/728656684<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=The+art+of+software+testing&rft.aulast=Myers&rft.aufirst=Glenford+J.&rft.au=Myers%2C%26%2332%3BGlenford+J.&rft.au=Sandler%2C%26%2332%3BCorey&rft.au=Badgett%2C%26%2332%3BTom&rft.date=2012&rft.edition=3rd+ed&rft.place=Hoboken%2C+N.J&rft.pub=John+Wiley+%26+Sons&rft.isbn=978-1-118-03196-4&rft_id=info:oclcnum\/728656684&rft_id=https%3A%2F%2Fwww.worldcat.org%2Ftitle%2Fmediawiki%2Foclc%2F728656684&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-47\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-47\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.iso.org\/standard\/63179.html\" target=\"_blank\">\"IEC 62366-1:2015 Medical devices \u2014 Part 1: Application of usability engineering to medical devices\"<\/a>. International Organization for Standardization. February 2015<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.iso.org\/standard\/63179.html\" target=\"_blank\">https:\/\/www.iso.org\/standard\/63179.html<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=IEC+62366-1%3A2015+Medical+devices+%E2%80%94+Part+1%3A+Application+of+usability+engineering+to+medical+devices&rft.atitle=&rft.date=February+2015&rft.pub=International+Organization+for+Standardization&rft_id=https%3A%2F%2Fwww.iso.org%2Fstandard%2F63179.html&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-48\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-48\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Hunter, Philip (1 September 2017). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.embopress.org\/doi\/10.15252\/embr.201744876\" target=\"_blank\">\"The reproducibility \u201ccrisis\u201d: Reaction to replication crisis should not stifle innovation\"<\/a> (in en). <i>EMBO reports<\/i> <b>18<\/b> (9): 1493\u20131496. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.15252%2Fembr.201744876\" target=\"_blank\">10.15252\/embr.201744876<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1469-221X\" target=\"_blank\">1469-221X<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC5579390\/\" target=\"_blank\">PMC5579390<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/28794201\" target=\"_blank\">28794201<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.embopress.org\/doi\/10.15252\/embr.201744876\" target=\"_blank\">https:\/\/www.embopress.org\/doi\/10.15252\/embr.201744876<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+reproducibility+%E2%80%9Ccrisis%E2%80%9D%3A+Reaction+to+replication+crisis+should+not+stifle+innovation&rft.jtitle=EMBO+reports&rft.aulast=Hunter&rft.aufirst=Philip&rft.au=Hunter%2C%26%2332%3BPhilip&rft.date=1+September+2017&rft.volume=18&rft.issue=9&rft.pages=1493%E2%80%931496&rft_id=info:doi\/10.15252%2Fembr.201744876&rft.issn=1469-221X&rft_id=info:pmc\/PMC5579390&rft_id=info:pmid\/28794201&rft_id=https%3A%2F%2Fwww.embopress.org%2Fdoi%2F10.15252%2Fembr.201744876&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-49\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-49\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Boulesteix, Anne-Laure; Hoffmann, Sabine; Charlton, Alethea; Seibold, Heidi (1 October 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/academic.oup.com\/jrssig\/article\/17\/5\/18\/7038554\" target=\"_blank\">\"A Replication Crisis in Methodological Research?\"<\/a> (in en). <i>Significance<\/i> <b>17<\/b> (5): 18\u201321. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1111%2F1740-9713.01444\" target=\"_blank\">10.1111\/1740-9713.01444<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1740-9705\" target=\"_blank\">1740-9705<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/academic.oup.com\/jrssig\/article\/17\/5\/18\/7038554\" target=\"_blank\">https:\/\/academic.oup.com\/jrssig\/article\/17\/5\/18\/7038554<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Replication+Crisis+in+Methodological+Research%3F&rft.jtitle=Significance&rft.aulast=Boulesteix&rft.aufirst=Anne-Laure&rft.au=Boulesteix%2C%26%2332%3BAnne-Laure&rft.au=Hoffmann%2C%26%2332%3BSabine&rft.au=Charlton%2C%26%2332%3BAlethea&rft.au=Seibold%2C%26%2332%3BHeidi&rft.date=1+October+2020&rft.volume=17&rft.issue=5&rft.pages=18%E2%80%9321&rft_id=info:doi\/10.1111%2F1740-9713.01444&rft.issn=1740-9705&rft_id=https%3A%2F%2Facademic.oup.com%2Fjrssig%2Farticle%2F17%2F5%2F18%2F7038554&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-50\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-50\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Hauschild, Anne-Christin; Eick, Lisa; Wienbeck, Joachim; Heider, Dominik (1 July 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2589004221007719\" target=\"_blank\">\"Fostering reproducibility, reusability, and technology transfer in health informatics\"<\/a> (in en). <i>iScience<\/i> <b>24<\/b> (7): 102803. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.isci.2021.102803\" target=\"_blank\">10.1016\/j.isci.2021.102803<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC8282945\/\" target=\"_blank\">PMC8282945<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/34296072\" target=\"_blank\">34296072<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2589004221007719\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2589004221007719<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Fostering+reproducibility%2C+reusability%2C+and+technology+transfer+in+health+informatics&rft.jtitle=iScience&rft.aulast=Hauschild&rft.aufirst=Anne-Christin&rft.au=Hauschild%2C%26%2332%3BAnne-Christin&rft.au=Eick%2C%26%2332%3BLisa&rft.au=Wienbeck%2C%26%2332%3BJoachim&rft.au=Heider%2C%26%2332%3BDominik&rft.date=1+July+2021&rft.volume=24&rft.issue=7&rft.pages=102803&rft_id=info:doi\/10.1016%2Fj.isci.2021.102803&rft_id=info:pmc\/PMC8282945&rft_id=info:pmid\/34296072&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2589004221007719&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-51\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-51\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Sapunar, Damir (28 May 2016). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/ama.ba\/index.php\/ama\/article\/view\/268\/pdf\" target=\"_blank\">\"The business process management software for successful quality management and organization: case study from the University of Split School of medicine\"<\/a>. <i>Acta Medica Academica<\/i> <b>45<\/b> (1): 26\u201333. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.5644%2Fama2006-124.153\" target=\"_blank\">10.5644\/ama2006-124.153<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/ama.ba\/index.php\/ama\/article\/view\/268\/pdf\" target=\"_blank\">http:\/\/ama.ba\/index.php\/ama\/article\/view\/268\/pdf<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+business+process+management+software+for+successful+quality+management+and+organization%3A+case+study+from+the+University+of+Split+School+of+medicine&rft.jtitle=Acta+Medica+Academica&rft.aulast=Sapunar&rft.aufirst=Damir&rft.au=Sapunar%2C%26%2332%3BDamir&rft.date=28+May+2016&rft.volume=45&rft.issue=1&rft.pages=26%E2%80%9333&rft_id=info:doi\/10.5644%2Fama2006-124.153&rft_id=http%3A%2F%2Fama.ba%2Findex.php%2Fama%2Farticle%2Fview%2F268%2Fpdf&rfr_id=info:sid\/en.wikipedia.org:Journal:Guideline_for_software_life_cycle_in_health_informatics\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<\/ol><\/div><\/div>\n<h2><span class=\"mw-headline\" id=\"Notes\">Notes<\/span><\/h2>\n<p>This presentation is faithful to the original, with only a few minor changes to presentation, though grammar and word usage was substantially updated for improved readability. In some cases important information was missing from the references, and that information was added.\n<\/p>\n<!-- \nNewPP limit report\nCached time: 20231215052528\nCache expiry: 86400\nDynamic content: false\nComplications: []\nCPU time usage: 0.730 seconds\nReal time usage: 0.872 seconds\nPreprocessor visited node count: 50841\/1000000\nPost\u2010expand include size: 432684\/2097152 bytes\nTemplate argument size: 131825\/2097152 bytes\nHighest expansion depth: 25\/40\nExpensive parser function count: 0\/100\nUnstrip recursion depth: 0\/20\nUnstrip post\u2010expand size: 122687\/5000000 bytes\n-->\n<!--\nTransclusion expansion time report (%,ms,calls,template)\n100.00% 595.234 1 -total\n 87.41% 520.272 1 Template:Reflist\n 68.34% 406.797 51 Template:Citation\/core\n 55.99% 333.283 36 Template:Cite_journal\n 11.79% 70.187 48 Template:Date\n 8.65% 51.463 8 Template:Cite_web\n 8.15% 48.527 103 Template:Citation\/identifier\n 7.07% 42.095 5 Template:Cite_book\n 5.78% 34.405 1 Template:Infobox_journal_article\n 5.11% 30.414 1 Template:Infobox\n-->\n\n<!-- Saved in parser cache with key limswiki:pcache:idhash:14276-0!canonical and timestamp 20231215052527 and revision id 52442. Serialized with JSON.\n -->\n<\/div><\/div><div class=\"printfooter\">Source: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.limswiki.org\/index.php\/Journal:Guideline_for_software_life_cycle_in_health_informatics\">https:\/\/www.limswiki.org\/index.php\/Journal:Guideline_for_software_life_cycle_in_health_informatics<\/a><\/div>\n<!-- end content --><div class=\"visualClear\"><\/div><\/div><\/div><div class=\"visualClear\"><\/div><\/div><!-- end of the left (by default at least) column --><div class=\"visualClear\"><\/div><\/div>\n\n\n\n<\/body>","3a2816a67d7d45f854c1e2fb9ec00f31_images":["https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/1\/1a\/GA_Hauschild_iScience2022_25-12.jpg","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/5\/5b\/Fig1_Hauschild_iScience2022_25-12.jpg"],"3a2816a67d7d45f854c1e2fb9ec00f31_timestamp":1702682171,"96ca1abdbcc7bf60389fe942b678382d_type":"article","96ca1abdbcc7bf60389fe942b678382d_title":"Transforming healthcare analytics with FHIR: A framework for standardizing and analyzing clinical data (Ayaz et al. 2023)","96ca1abdbcc7bf60389fe942b678382d_url":"https:\/\/www.limswiki.org\/index.php\/Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data","96ca1abdbcc7bf60389fe942b678382d_plaintext":"\n\nJournal:Transforming healthcare analytics with FHIR: A framework for standardizing and analyzing clinical dataFrom LIMSWikiJump to navigationJump to searchFull article title\n \nTransforming healthcare analytics with FHIR: A framework for standardizing and analyzing clinical dataJournal\n \nHealthcareAuthor(s)\n \nAyaz, Muhammad; Pasha, Muhammad F.; Alahmadi, Tahani J.; Abdullah, Nik N.B.; Alkahtani, Hend K.Author affiliation(s)\n \nMonash University, Princess Nourah bint Abdulrahman UniversityPrimary contact\n \nEmail: muhammad dot ayaz at monash dot eduYear published\n \n2023Volume and issue\n \n11(12)Article #\n \n1729DOI\n \n10.3390\/healthcare11121729ISSN\n \n2227-9032Distribution license\n \nCreative Commons Attribution 4.0 InternationalWebsite\n \nhttps:\/\/www.mdpi.com\/2227-9032\/11\/12\/1729Download\n \nhttps:\/\/www.mdpi.com\/2227-9032\/11\/12\/1729\/pdf?version=1686738253 (PDF)\n\nContents \n\n1 Abstract \n2 Background \n\n2.1 Healthcare data analytics \n2.2 Healthcare data analytics using the FHIR data standard \n\n\n3 Literature review \n4 Materials \n\n4.1 Required outcomes \n4.2 User research and inputs \n4.3 Challenges \n4.4 The clinical data analysis workflow design \n4.5 FHIR REST API's working mechanism \n\n\n5 FHIR data analytics framework \n\n5.1 FHIR database \n5.2 FHIR query engine layer \n5.3 Mapping agent\/algorithm \n\n5.3.1 Need of mapping algorithm \n5.3.2 Role of mapping algorithm \n\n\n5.4 FHIR-compliant database \n5.5 Data analytics engine layer \n5.6 User interface \n\n\n6 Methods\/Implementation \n7 Experiments \n8 Results \n\n8.1 Use case 1 \n8.2 Use case 2 \n8.3 Use case 3 \n8.4 Use case 4 \n8.5 Use case 5 \n\n\n9 Limitations \n10 Discussion \n11 Conclusions \n12 Acknowledgements \n\n12.1 Author contributions \n12.2 Funding \n12.3 Informed consent \n12.4 Data availability statement \n12.5 Conflict of interest \n\n\n13 References \n14 Notes \n\n\n\nAbstract \nIn this study, we discuss our contribution to building a data analytic framework that supports clinical statistics and analysis by leveraging a scalable standards-based data model named Fast Healthcare Interoperability Resources (FHIR). We developed an intelligent algorithm that is used to facilitate the clinical data analytics process on FHIR-based data. We designed several workflows for patient clinical data used in two hospital information systems (HISs), namely patient registration systems (PRSs) and laboratory information systems (LIS). These workflows exploit various FHIR application programming interfaces (API) to facilitate patient-centered and cohort-based interactive analyses. We developed a FHIR database implementation that utilizes FHIR APIs and a range of operations to facilitate descriptive data analytics (DDA) and patient cohort selection. A prototype user interface for DDA was developed with support for visualizing healthcare data analysis results in various forms. Healthcare professionals and researchers would use the developed framework to perform analytics on clinical data used in healthcare settings. Our experimental results demonstrate the proposed framework\u2019s ability to generate various analytics from clinical data represented in the FHIR resources.\nKeywords: data analytics, data analysis, FHIR, EMR, EHR\n\nBackground \nTo provide a comprehensive idea to readers about the applications of data analytics in the healthcare industry, this section introduces the data analytics concept employed in the healthcare sector. We also discuss the data analytics concept in the clinical data represented in the latest healthcare data standard, Fast Healthcare Interoperability Resources (FHIR).\n\nHealthcare data analytics \nHealthcare data analytics is the process of analyzing and interpreting large sets of healthcare data to gain insights and improve healthcare outcomes. It involves using a range of analytical techniques and tools to process data from various sources, such as electronic health records (EHRs), electronic medical records (EMRs), medical devices, claims data, patient-generated data, etc. The rapid advancements in hardware and software technologies in recent years have ushered in a new era of data collection and processing, resulting in remarkable progress in the field of healthcare data analytics. In the realm of healthcare organizations, clinical data serve a dual purpose. Firstly, it is utilized for the delivery of healthcare services to patients. Secondly, it is used for secondary purposes such as research, analysis, quality improvement, and more. In particular, the secondary use of clinical data has emerged as a critical component of healthcare data analytics. This has resulted in a paradigm shift in recent healthcare settings, where the secondary use of healthcare data is deemed just as important as its primary use.\nEHR systems are leveraged to facilitate the secondary use of healthcare data, for activities such as quality improvement, safety measurement, payments, provider certification, marketing, and research.[1] Moreover, the secondary use of healthcare data has the potential to significantly enhance the healthcare experiences of individuals. It can facilitate the learning of diseases and their effective treatments, deepen people\u2019s knowledge and understanding of the effectiveness and efficiency of healthcare systems, and aid in supporting public health initiatives.[1] However, the secondary use of healthcare data also raises complex ethical, social, and technical issues; for example, questions regarding data ownership and access privileges continue to challenge the field.[2]\nThe healthcare industry has witnessed a remarkable surge in the volume of healthcare data in recent times, primarily driven by the widespread adoption of EHR systems worldwide.[3] In addition, there has been an unprecedented growth in other types of healthcare data, such as genome sequencing and other biological structures.[3] The analysis of this clinical data is commonly referred to as analytics or healthcare data analytics, which falls under the category of secondary use of clinical data. While the term \"data analytics\" is extensively used in and outside of healthcare[3], our focus in this study is on its application in the healthcare industry.\nAnalytics has been deployed across various domains, including healthcare. However, experts from different fields offer diverse definitions of analytics. Nonetheless, the ultimate objective of analytics, as perceived by all experts, remains consistent. Data analytics experts characterize analytics as \u201cthe comprehensive exploitation of data, statistical and quantitative analysis, explanatory and predictive models, fact-based management to drive decisions, actions, and much more.\u201d[4] Similarly, IBM defines analytics as \u201cthe methodical use of data and associated business insights developed through applied analytical disciplines (e.g., statistical, predictive, contextual, quantitative, cognitive, and other models) to drive evidence-based decision making for planning, management, measurement, and learning. Analytics can be descriptive, predictive, or prescriptive.\u201d[5]\nMoreover, the two eminent healthcare data analytics experts, Adams and Klein, outline three distinct levels and applications of analytics in the healthcare domain.[6][dead link ] Each level is associated with increasing functionality and value:\n\nDescriptive: This level refers to standard reporting types that depict current situations and problems.\nPredictive: This level refers to simulation and modeling techniques that forecast trends and anticipate the outcomes of implemented actions.\nPrescriptive: This level concerns financial, clinical optimization, and other outcomes.\nAll three levels of healthcare data analytics are of paramount importance. However, predictive analytics has gained more attention in the current healthcare landscape[3], as medical experts seek to predict various clinical-related variables in healthcare data to enhance healthcare delivery services and optimize health and financial outcomes.\nWith the advent of digital medical records, hospitals and other healthcare organizations are accumulating vast amounts of data at an unprecedented rate. The clinical data captured by these organizations take multifarious forms, ranging from structured data (such as laboratory results and images) to unstructured data (such as textual notes comprising clinical narratives, reports, and various other documents). For example, the well-known US healthcare company Kaiser-Permanente has a current data store for over nine million members that surpasses a staggering 30 petabytes of data.[7] Another notable example is the American Society for Clinical Oncology (ASCO), which is developing its Cancer Learning Intelligence Network for Quality (CancerLinQ).[8] The clinical data accumulated by CancerLinQ serve myriad healthcare data analytics purposes, providing clinicians and researchers with an extensive platform for EHR data collection, data mining, and visualization, as well as the application of clinical decision support, among others.\nThe ultimate goal of healthcare data analytics is to use data to make informed decisions and identify patterns and trends that can help improve patient outcomes, optimize operational efficiency, and reduce costs. By analyzing data, healthcare providers can identify areas for improvement, predict health outcomes, and personalize care for individual patients.\nSome common applications of healthcare data analytics include population health management, clinical decision support, disease surveillance and monitoring, and quality improvement initiatives. The field of healthcare data analytics is constantly evolving as new technologies and approaches emerge, and it is a critical area of focus for healthcare organizations looking to improve their performance and deliver better care to patients.\nTo summarize, data analytics has become a pivotal aspect of current healthcare settings, a core requirement for both the industry and its experts.[3] Moreover, the future of healthcare holds tremendous promise when it comes to data analytics. With the burgeoning volume of clinical and research data, coupled with the methods employed to analyze and put it to use, there is tremendous potential for improving healthcare delivery, personal health, and biomedical research. However, there is also a continuing need to improve the quality of clinical data and conduct research aimed at demonstrating how best to apply data analytics to address healthcare challenges.\n\nHealthcare data analytics using the FHIR data standard \nFHIR is the latest healthcare data standard that is gaining popularity in the healthcare sector.[9] FHIR provides a standardized way to represent and exchange healthcare information electronically.[10] This avant-garde standard has captured the imagination of healthcare providers due to its unparalleled ability to reduce the costs of interoperability and its potential to catalyze a new ecosystem of third-party applications.[11] FHIR\u2019s revolutionary interoperability capabilities have surpassed the antiquated data standards of yore, such as Health Level 7 (HL7; v2, v3, CDA).\nIn a recent survey conducted by Australian and New Zealand healthcare executives, the adoption of FHIR was found to increase interoperability from a measly 11% to a staggering 66%.[12] Consequently, its adaptable nature for data exchange is increasing at a rapid pace within the healthcare industry as it garners favor among stakeholders for data exchange. The survey further revealed that 55% of healthcare providers are willing to make the shift to a FHIR-based interoperability platform. Additionally, it is estimated that FHIR will be widespread in the world healthcare industry by 2024.[13] This showed the popularity of FHIR-based interoperability in the healthcare industry and healthcare providers\u2019 interest in its adaptability.\nHowever, the healthcare industry\u2019s needs go beyond mere clinical data exchange. Clinical data need to be processed for other purposes, such as data analysis, data analytics, research, and so forth. Thus, the clinical data represented in the FHIR standard need to fulfill these requirements. FHIR\u2019s adoption is expected to increase data availability for analytics and solve the data exchange and analytics problems faced by the healthcare industry.[12] Nevertheless, the adoption of FHIR in the analytics domain remains relatively low, as the standard is still young.[14] Moreover, the tools supporting FHIR data analytics are still relatively immature.[15] However, the healthcare providers argue that they are not only interested in sharing clinical data across healthcare organizations to improve data interoperability but are more excited to process clinical data for other purposes, such as data analysis and research, to provide real-time medical services to patients. Therefore, the tools provided these services are essential in the healthcare industry.\nOn the flip side, the cutting-edge FHIR standard for patient clinical information presents plenty of new opportunities for visualizing, analyzing, and automating various types of healthcare data. With each passing day, fresh use cases for FHIR data analytics are building in the healthcare industry, such as real-time alerts for patient satisfaction, identifying patterns in patients\u2019 medical records across datasets, real-time visibility into patient readmission rates, cost savings while upholding top-notch care quality, and countless more.[16][17][18][19] However, analyzing and implementing these use cases can prove challenging owing to the young stage and practicality of FHIR.\nTo facilitate data processing and exchange, FHIR employs REST APIs. Nonetheless, for the domain of FHIR data analytics, the FHIR APIs must possess a dynamic nature regarding data queries and processing. As data analytics are based on diverse types of data housed in varied FHIR resources, the FHIR APIs must query this data in various ways to enable effective data analysis. Additionally, FHIR has accelerated the swift delivery of a massive volume of new healthcare applications that can integrate with EHR or EMR data via the FHIR APIs. However, most of these applications are limited to perusing data relevant to a single patient.[14] One contributing factor, among many others, could be that the FHIR APIs are not optimally suited to queries that aggregate and categorize data across a vast clinical dataset.\nA related and parallel trend within the realm of health information systems involves investing in higher-quality structured data via the coding of clinical records at the point of care. With the implementation of EMRs, healthcare providers are now able to incorporate a multitude of concepts into medical records using advanced terminologies, including ICD-10, LOINC, and SNOMED CT.[20][21] This affords the opportunity for more detailed analysis by enabling access to specific clinical concepts as well as the ability to query the ontology based on additional attributes and relationships to other clinical concepts.\nWhile this technique is highly effective when analyzing clinical data based on specific codes or terminologies, it proves to be less fruitful in general concept analysis. Therefore, other scenarios, including modifications to FHIR APIs, must be considered to enable various ways of analyzing medical data for deep clinical data analysis. However, this technique is extremely challenging and requires an individual with extensive skill and experience to change the core implementation mechanisms of FHIR APIs.\nCurrently, the level of expertise required to make the best use of FHIR and other clinical terminology within a data analysis workflow is relatively rare in the healthcare domain.[22][23][24] The applications of data analytics and analysis in healthcare settings using the FHIR data standard are also a relatively new concept and have scarcely been applied. However, due to the rapid adoption of FHIR for medical data exchange, data analytics and analysis are now a core demand of the healthcare industry to process patient medical data in various ways and provide real-time medication to improve healthcare delivery. In summary, the standardization of healthcare data plays a crucial role in clinical and translational data analysis systems, especially when large-scale data are involved. Moreover, healthcare applications for clinical statistics and analysis can significantly enhance healthcare by connecting clinical data with analytic tools, thereby engaging practitioners or clinicians in the process of medical data analysis.[25][26]\nIn response to the pressing need to address the complex and multifaceted challenges of data analytics in the healthcare industry, this research study puts forth a cutting-edge and innovative FHIR standard-based data analytics framework. This platform is designed to tackle the healthcare industry\u2019s data analytics issues and provide them with a scalable, standards-based data model. At present, this pioneering framework is tailored to work with workflows specifically designed for patient clinical data originating from two distinct hospital information systems: patient registration systems (PRSs) and laboratory information systems (LISs). Other possible data analysis workflows and customized research scenarios on the patient data from other HISs could be performed on FHIR-based data but are not currently directly supported by our framework without any modification.\nThe developed framework utilizes a FHIR database as its dataset, with FHIR RESTful APIs that query different types of FHIR resources from the database algorithmically. The mapping algorithm and analytic engine then process the retrieved data and generate various data analytics from patient clinical data, presenting the results to end-users via a user-friendly interface.\nIn short, this research study provides a state-of-the-art solution for healthcare data analytics, offering healthcare professionals an innovative platform to conduct data analysis on clinical data using FHIR. With the FHIR Data Analytics Framework, healthcare professionals can now extract meaningful insights from patient data and leverage these insights to enhance patient care delivery, promote better health outcomes, and drive healthcare industry advancements forward.\nThis research work has three main contributions: First, the entire framework and workflow design follow the FHIR data standard, which could be reused for any other clinical data domains and could provide support for any clinical data that follow the FHIR standard. Second, the data analysis workflow and tools incorporate the experience of clinical researchers and statisticians, which could provide a starting point for FHIR researchers in this cutting-edge standard. Third, the intelligent mapping algorithm is artfully designed to facilitate the sublime process of data analytics or data analysis within the realm of FHIR-based data. The mapping algorithm could be reused for any other clinical data that follow the FHIR specification and need to process the FHIR-based data for other purposes, such as research, developing an artificial intelligence (AI) model or machine learning (ML) model, etc.\nThe FHIR Data Analytics Framework comprises six layers: the FHIR database, the FHIR query engine layer, the mapping algorithm\/agent layer, the FHIR-compliant database layer, the analytics engine layer, and the user interface. The rest of this manuscript is structured accordingly. The next section provides a comprehensive literature review, followed by a discussion of the five major materials used in this study. Then, the framework\u2019s architecture is described in detail, followed by the implementation details, an explanation of the experiment setup, and the results. We close by describing the limitations of this approach, as well as a discussion, future plans, and finally a conclusion.\n\nLiterature review \nThroughout the years, financial and administrative data were deemed essential attributes for planning purposes. However, in recent times, comprehensive healthcare data have become crucial to institutional strategic planning and self-analysis.[27] The healthcare industry heavily relies on various data sources, such as EHR analysis (EHRA), biomedical image analysis (BIA), sensor data analysis (SDA), biomedical signal analysis (BSA), genomic data analysis (GDA), clinical text mining (CTM), and other analytics methods to process and analyze clinical data.[28] Analyzing and performing data analytics on clinical data in the healthcare settings is a fundamental requirement in the healthcare industry. Despite this, the literature scarcely acknowledges the use of data analytics in the healthcare industry.\nIn our thorough literature review, we noticed some efforts that utilized various clinical data sources in the data analytics domain. For example, the Observational Health Data Sciences and Informatics (OHDSI) program has generated an enormous volume of work in the field of health data analytics, including the creation of the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM).[29] The OMOP provides a target data model for health data analytics, along with analytic routines and common vocabularies that could be run over the common data model.[30]\nFurthermore, the OMOP has a rich ecosystem of applications that have been developed to assist in its implementation and use, such as the ATLAS user interface designed by the OHDSI community[31] to facilitate analytic queries over the OMOP data model. Moreover, researchers have also explored the use of the openEHR model within health data analytics, as exemplified by the work of Chunlan et al.[32] in developing the Archetype Query Language (AQL), which is a standard way of querying data from openEHR-based systems.[33] The AQL has been implemented in many EHRs and analytics software tools and provides important design features for this type of capability.[14]\nHowever, while these attempts have been applied to EHR datasets, the application of such techniques to data represented in the FHIR standard is a relatively new and challenging concept. Therefore, the researchers are looking for new techniques with which they can apply data analytics to the clinical data represented in the FHIR-based standard. However, as aforementioned, FHIR is a young data standard[14], and limited research related to FHIR analytics has been reported.[34] A recent scientific literature review study reveals that only a few studies have been reported in the literature that discussed FHIR analytics.[15] Thus, the concept of FHIR data analytics is extremely new, and so far, the state of FHIR analytics is at an early stage. Therefore, applying data analytics or data analysis is challenging and an extremely new concept in this domain. However, some researchers have made some initial efforts in FHIR-based analytical circumstances, such as the prediction of sepsis based on the FHIR standard by Lakshman et al.[35] and the deployment of clinical predictive models via FHIR in Web Services explained by Khalilia et al.[36]\nFurthermore, the use of FHIR to store and analyze medical data on a large scale has also been implemented and integrated into the Google Cloud and Microsoft Azure cloud platforms.[20][25] In addition, the tech company Startups has recognized the analytical capabilities of FHIR and utilized the doc.ai application to provide personalized medicine, automate the process of controlling audit files, and store data in a structured way.[37]\nMoreover, FHIR was used to support clinical decisions and to build a distributed phenotyping analytics platform.[38] Kreuzthaler et al. discussed the use and benefits of standardized data in analytical approaches.[39] In addition, Franz et al. developed a monitoring system with the FHIR data standard.[40] Liu et al.[41] explained many ways to make bulk FHIR data available for analytic queries. The authors concluded that Apache Parquet[42] is the ideal tool for storing and querying FHIR data in the context of large-scale analytics using Apache Spark.\nGrimes et al.[14] discussed the use of FHIR data analytics using the pathling concept. However, it works in a limited domain because some operations are not easily or even currently possible to achieve via the FHIR REST API specification, such as data aggregation, searching the data, etc. Therefore, it is extremely challenging to implement. Furthermore, Dunn et al.[43] explained genomic data analysis using FHIR in a cloud framework. However, it only applies to the analysis of genomic data using a cloud framework and would be challenging to apply to clinical data represented in FHIR and implement in traditional healthcare settings. Similarly, Gruendner et al.[44] described the FHIR data formatting for statistical analysis. However, this technique only generated the FHIR data but failed to provide any platform for clinical data analysis or data analytics using REST APIs. Therefore, it is extremely challenging to generalize the concept and provide a platform for medical software developers and researchers to perform any data analytics on the clinical data or use the resulting data for research purposes.\nMoreover, the notable health information technology (HIT) services provider Cerner Corporation produces the Bunsen[45] library that encodes FHIR resources within Apache Spark[46] datasets. This work facilitates loading, transforming, and analyzing FHIR data. Cerner Corporation has also been involved with implementation of Structured Query Language (SQL) on the FHIR proposal[47], which is a projection of the FHIR data model onto the relational query model and SQL language. Additionally, Google also discussed and implemented a method for encoding FHIR data using the Buffers Protocol.[48] Furthermore, Google also developed many tools and techniques[49] for using FHIR with the BigQuery analytics platform, integrating with the FHIR Bulk Data API, and using FHIR data within cloud-based data processing and machine learning pipelines.\nDespite the various initial attempts at data analytics on clinical data represented in the FHIR data standard, there has been no user-friendly data analytics framework or visualized tool to help healthcare users such as practitioners, providers, and patients perform various data analytics on patient clinical data. To address this gap, our research study developed a framework with a user-friendly interface that enables healthcare practitioners, providers, and patients to perform data analytics on the clinical data used in two HISs and represented in the FHIR-based standard.\n\nMaterials \nIn this section, we are discussing various materials that will help us develop our framework. This information is helpful for the readers to know about the challenges and framework pre-development procedures involved in this undertaking.\n\nRequired outcomes \nOur ambition was to develop a data analytics framework that could perform various types of analytics on clinical data typically used in healthcare facilities and represented in the FHIR standard. Our esteemed endeavor has borne fruit, and we have proudly fashioned a data analytics framework for healthcare environments, performing an array of analytical procedures on clinical data and elegantly visualizing the resulting insights.\n\nUser research and inputs \nAs previously discussed, support for FHIR and related data analytics is still in its infancy. Moreover, the clinical data flow in healthcare settings and the data analytics concept on clinical data represented in the FHIR format remain unclear at this stage. Particularly for individuals outside of the medical field, comprehending this concept can prove to be a challenging task. Therefore, to better grasp the FHIR data analytics concept and its workflow in the healthcare environment, we decided to take input from various professionals working in healthcare settings. We conducted numerous interviews with doctors, practitioners, patients, pharmacists, and others in the healthcare industry to obtain a more comprehensive understanding of workflow and user requirements. These interviews consisted of both open-ended and closed questions related to the current challenges within the healthcare data analytics domain. Furthermore, we sought to understand the data analytics needs of various stakeholders, including patients, practitioners, and healthcare providers, regarding the healthcare industry.\nThis process helped us validate our assumptions about adopting FHIR data analytics in the healthcare industry and provided insight into the views of users (practitioners, patients, providers, etc.) regarding the adoption of FHIR data analytics, as well as their opinions on workflow with this new technique in this domain. It also identified a range of use case scenarios for various analytics that we could implement in this prototype. Based on what we learned from this process, we selected the following two parameters (use cases) to serve as the focal point of our work:\n\nPatient cohort selection: This involves the selection and retrieval of patient information\/records based on complex inclusion and exclusion criteria.\nData preparation: This includes processing and reshaping data in preparation for use with statistical models or tools.\nChallenges \nFHIR has a highly nested, complex, and graph-like data format that represents clinical data in a resource structure in JSON\/XML format. With a hierarchical tree structure, the data elements are nested, making it difficult to represent within traditional relational data models, especially when simplifying query logic is a primary goal. Representing the data in a traditional relational data model is essential for data analytics and analysis. However, the graphical FHIR resource structure poses a significant challenge, and optimizing the data structure for analytics and analysis queries across a wide range of use cases also raises performance issues.\nTo handle these challenges, we have developed a cutting-edge mapping algorithm\/agent to convert FHIR resource data into a sample EMR format and store it in a relational data model before conducting any data analytics. Our mapping algorithm was used to transform the clinical data stored in FHIR resources into a relational data model.\n\nThe clinical data analysis workflow design \nPerforming data analytics on the data present in the dataset is challenging, as it relies totally on workflows (business use case scenarios). The design of such workflows is quite difficult, particularly for non-medical experts, because they have issues identifying various parameters for the clinical data used in the healthcare settings, which include user requirements, data constraints, and more. Therefore, we had discussions with medical experts and, on the basis of their inputs and our common clinical data analysis requirements, we designed two general analysis workflows: patient-centered data analysis and cohort-based data analysis. Furthermore, we elaborated on the workflows and designed five primary workflows that are used in healthcare settings on the patient data and are suitable for performing data analytics on our dataset (see Table 1).\n\n\n\n\n\n\n\nTable 1. List of data analysis workflows (business use cases) for patient data in the healthcare setting.\n\n\nNumber\n\nDescription\n\n\n1\n\nInvestigate registered patients in healthcare settings\n\n\n2\n\nInvestigate registered patients in healthcare settings within a specified timeframe\n\n\n3\n\nInvestigate patients having various types of allergies\n\n\n4\n\nInvestigate various types of tests ordered by a physician, organization, etc.\n\n\n5\n\nInvestigate various types of tests ordered by a physician, organization, etc., within a specified timeframe\n\n\n\nThe patient-centered data analysis workflow facilitates the browsing of various pieces of information focused on the individual. Patient-specific data derived from multiple sources are integrated into a single identifier. In the FHIR data model, the patient is an independent resource, while other resources such as observation and practitioner have a property \u201csubject\u201d that links them to a specific patient object, representing patient-centered relationships.\nThe cohort-based analysis workflow refers to more common data analysis needs in clinical statistics and studies. In this workflow, the Condition\/AllergyIntolerance\/Observation\/Practitioner of a cohort is largely measured by the distribution of patient characteristics in different dimensions. The workflow is designed to support a wide range of clinical data analysis tasks, including patient registration analysis, patient allergy timeline analysis, patient laboratory test analysis, cohort gender\/age distribution statistics, and more. Overall, our workflows provide a robust framework for performing data analytics on healthcare datasets.\n\n FHIR REST API's working mechanism \nThe FHIR specification defines standard REST APIs to exchange a variety of healthcare data and perform a range of operations on the clinical data represented in the FHIR resource structure. These APIs are also known as the core FHIR REST APIs. The power of these APIs lies in their use of the widely accepted HTTP (GET, POST, PUT, DELETE) protocol to perform pre-defined operations such as CRUD (Create, Read, Update, Delete) on any FHIR resource. For example, with just a few clicks, one can access and retrieve the update history, view information, delete, create, or update any instance of a FHIR resource. Figure 1 illustrates the view of these operations.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 1. CRUD operations of the patient resource in the FHIR server.\n\n\n\nFurthermore, every FHIR API conforms to a common signature and format, ensuring that FHIR-compliant systems can retrieve specific healthcare data using the same API signature and format. For example, to retrieve patient demographic information based on the patient\u2019s name and date of birth, one can use the following API:\nGET http:\/\/baseURL\/Patient?given=[patient given name]&birthDate=[date of birth]\nThis API will retrieve the patient\u2019s name and date of birth. In this API, \u201cPatient\u201d is the FHIR patient resource, while \u201cgiven\u201d and \u201cbirthDate\u201d are the given parameters. The output of this API will be in standard JSON\/XML format, with tags and elements following strict standards. Figure 2 provides an illustrative view of a sample API operation mechanism in which the APIs access the clinical data, and we performed data analytics on that data.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 2. REST APIs Operations: The operations performed on clinical data represented in FHIR resources.\n\n\n\nFHIR data analytics framework \nWe developed a FHIR data analytics framework used to perform various data analytics on the clinical data represented in the FHIR resource structure. In our use case scenario, the FHIR resources are stored in the Mango database that we developed in our previous porotype. We developed various APIs on top of this database to retrieve the data stored in the FHIR resources format and perform data analytics. Figure 3 explains various sections of this framework and their connections. This framework has the following six major parts:\n\nFHIR database\nFHIR query engine\nMapping algorithm\nFHIR-compliant database (relational database model)\nAnalytic engine\nUser interface\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 3. Block diagram of the proposed data analytic framework: Explains various sections of the framework and their connections.\n\n\n\nFHIR database \nThe FHIR database is the collection of FHIR resources that we already developed in our previous prototype and would be used as a dataset. Therefore, we are not discussing the creation of this database in this study. Within our database, we have different types of resources, each comprising a grand total of 100 individual resources, but we utilized only those resources that are used for our data analysis.\n\nFHIR query engine layer \nThe FHIR query engine is a collection of FHIR queries, executing only FHIR queries based on core FHIR CRUD operations. Our query engine is responsible for accessing a list of available FHIR resources from the FHIR databases and preparing them for further processing. For this purpose, it uses the core FHIR RESTful APIs. Therefore, our query engine adeptly employs these RESTful APIs to extract and gather all FHIR resources in bulk out of the FHIR database and do some processing, filtering, and transformation within client-side code (in our case, the query engine). We leveraged the core FHIR GET and search APIs to access all resources from the FHIR database. The resulting data (FHIR resources) are assumed to be available in JSON format, the standard format for bulk FHIR data interchange. Table 2 shows the resulting data that have been retrieved from the FHIR database using REST APIs. For this purpose, we used an algorithm (see Algorithm 1) to access all FHIR resources housed within the database. Each type of resource has its own unique title and access parameters; therefore, for different FHIR resources, we used different resource names and search and access parameters within the resource URL to access each resource type. Figure 4 shows the block diagram of the query engine.\n\n\n\n\n\n\n\nTable 2. FHIR resources retrieved from FHIR database using APIs.\n\n\nNumber\n\nResource type\n\nTotal resources\n\n\n1\n\nPatient\n\n100\n\n\n2\n\nAllergyIntolerance\n\n100\n\n\n3\n\nPractitioner\n\n100\n\n\n4\n\nServiceRequest\n\n100\n\n\n5\n\nDiagnosticReport\n\n100\n\n\n6\n\nCondition\n\n100\n\n\n7\n\nAppointment\n\n100\n\n\n\n\n\n\n\n\n\nAlgorithm 1. Algorithm to retrieve resources from FHIR database.\n\n\n\n1. <b>Function<\/b> Retrive_Resources()<br \/>\n2. define resource type, e.g., patient<br \/>\n3. define search parameters, e.g., resource id or any other attribute(s)<br \/>\n4. value = Read resource id<br \/>\n5.\u2003\u2003 <b>while<\/b> (resources are available) <b>do<\/b><br \/>\n6. \u2003\u2003 <b>GET<\/b> [base-url]\/RsourceName?id = value<br \/>\n7.\u2003\u2003 <b>end while<\/b><br \/>\n8. <b>end function<\/b> ** Retrive_Resources function **\n\n\n\n\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 4. Query engine working mechanism: Query engine read FHIR resources in bulk.\n\n\n\nThe first feature provided by REST API is the read (GET) operation. This operation provided a way to access data and prepared it for further operations via various sub-operations. This standard FHIR API reads the FHIR resources from the databases or servers and transfers to the clients in the form of JSON.\nThe GET operation is designed to accept data extracted from the database or server via FHIR APIs operations. One of the primary functions of the GET request as a data request is a method to provide the data to the client. During the GET request operation, the clients (we) must provide the server or database with URLs indicating which data (data from resources) we seek to retrieve.\nThese URLs also enable us to receive updates on the operation\u2019s progress and valuable information about retrieving the final results.\nThe retrieved data are made available to the client in a JSON format (in our case, the query engine). Figure 5 demonstrates that the query engine part reads the FHIR resources from the database using these APIs. \n\r\n\n\n\n\n\n\n\n\n\n\nFigure 5. The GET APIs to extract resources from FHIR database.\n\n\n\nHere is an example of the API query:\nGET [base-url]\/resource-type? parameters\nFor example, we could obtain data from a patient resource with identifier 23 using this query:\nGET [base-url]\/patient? identifier = 23\n\n Mapping agent\/algorithm \nNeed of mapping algorithm \nThe FHIR REST APIs are currently in their nascent stage, offering limited functionalities and operations that can be leveraged for healthcare data analytics applications. The FHIR REST APIs can only perform the core CRUD (Create, Read, Update, and Delete) operations, alongside a handful of other basic functionalities, on data stored in the FHIR resources. These operations are executed using standard mechanisms provided by FHIR, and the REST APIs are happy to execute these operations on various FHIR resources while exchanging data between the FHIR server and the client.\nHowever, the healthcare landscape is rapidly evolving, and there is an increasing demand for more advanced and complex operations on patient data in the healthcare environment, particularly in the FHIR data analytics domain. Additionally, healthcare analytics applications need to be improved to reduce the data processing burden and enhance the quality of data analyses.[14] Consequently, the REST APIs must evolve to prepare themselves for these challenges by incorporating more complex functionalities and executing more complicated queries. For this purpose, FHIR offers standard mechanisms for extending API functionality, such as extension operations and search profiles.\nCertain types of operations, including data transformation, aggregations, search operations, and many more, are unfortunately unachievable or impossible using the core FHIR APIs specification.[14] This limitation implies that executing more complex queries to perform advanced operations, such as any data analytics or analysis operations on clinical data stored in FHIR resources, is quite challenging and limited at this stage of REST. In other words, the core FHIR APIs encounter difficulties while performing data analytics directly on the patient data stored in the FHIR resources structure in the FHIR server or database. However, the use of data analytics in healthcare information systems is essential in the modern healthcare environment. As a result, we leveraged the FHIR core API functionalities and implemented a specialized intermediate layer known as a mapping algorithm\/agent to simplify data analytics operations on the data stored in the FHIR resources.\n\nRole of mapping algorithm \nThe FHIR APIs present us with a wealth of resources, returned in the JSON format, which is a complex, hierarchical structure that nests data elements within tags. However, this structure is unsuitable for data analytics operations, which typically require structured or unstructured data, not data in a hierarchal structure.[50] Therefore, we must preprocess the JSON data by converting it into a tabular format and storing it in a FHIR-compliant relational database before applying any analytics.\nFor this purpose, we used a special agent that mapped the FHIR resource data into a format more suitable for data analytics. This mapping agent was responsible for converting the retrieved FHIR resource data via core FHIR APIs into a flat data format. The resulting data elements were then stored in a FHIR-compliant database, ready for analytics. The mapping algorithm is presented in Algorithm 2. Our mapping algorithm worked as a mapping agent between the FHIR API and FHIR-compliant database for data conversion. This mapping algorithm worked seamlessly for all types of resources in our dataset, for example, Patient, AllergyIntolerance, Practitioner, Condition, DiagnosticReport, ServiceRequest, Appointment, etc. Whenever we retrieved FHIR resource data from the FHIR centralized database, we applied the mapping algorithm on the way during FHIR API operations to retrieve and transform the data into the FHIR-compliant database; we named this data-mapping mechanism \u201cData Retrieval on Fly (DRF).\u201d The working mechanisms of this algorithm are illustrated in Figure 6, which depicts how it acted as a mediator between the FHIR API and a FHIR-compliant database, thus enabling the efficient conversion of hierarchical data into tabular data for analytics purposes.\n\n\n\n\n\n\n\nAlgorithm 2. Mapping Algorithm (Transform JSON data to EMR format).\n\n\n\n1. <b>Function<\/b> void main ()<br \/>\n2.\u2003\u2003 Create Tables in MySQL database, once table for each resources type data and link these tables<br \/>\n3.\u2003\u2003 Resource = Read (FHIR API resource)<br \/>\n4.\u2003\u2003 Templet = Resource-Templet (Resource)<br \/>\n5.\u2003\u2003 counter = Count(Temple)<br \/>\n6.\u2003\u2003 <b>while<\/b> (counter > 0) <b>do<\/b><br \/>\n7. \u2003\u2003 <b>If<\/b> (Templet.Tag == Resource.Tag) <b>then<\/b><br \/>\n8.\u2003\u2003 \u2003\u2003 Table. attribute = Resource.Tag.Value<br \/>\n9. \u2003\u2003 <b>end if<\/b><br \/>\n10.\u2003\u2003 counter = counter \u2212 1<br \/>\n11.\u2003\u2003 <b>end while<\/b><br \/>\n12. <b>end function<\/b> ** main function **<br \/>\n13. ** This function used to compare Resource type **<br \/>\n14. <b>Function<\/b> string Resource-Templet (Resource type)<br \/>\n15. ** Create one dimension array for all resources and stored their tags. This is pre-defined templet for all resources **<br \/>\n16. define string Result<br \/>\n17. String Array List = [Patient, Condition, AllergyIntolerance, Practitioner, ServiceRequest, DiagnosticReport, Appointment, \u2026\u2026\u2026]<br \/>\n18. String Patient [] = [\u201cidentifier\u201d, \u201cname\u201d, \u201ctelecom\u201d, \u201caddress\u201c, \u201cgender\u201d \u2026\u2026\u2026\u2026]<br \/>\n19. String Condition [] = [\u201cidentifier\u201d, \u201cclinical status\u201d, \u201ccategory\u201d, \u201ccode\u201d \u2026\u2026\u2026\u2026]<br \/>\n20. String AllergyIntolerance [] = [\u201cidentifier\u201d, \u201cclinical status\u201d, \u201ccode\u201d, \u2026\u2026\u2026\u2026]<br \/>\n21. String Practitioner [] = [\u201cidentifier\u201d, \u201cname\u201d, \u201caddress\u201d, \u201cqualification\u201d, \u2026\u2026\u2026\u2026]<br \/>\n22. String DiagnosticReport [] = [\u201cidentifier\u201d, \u201cbaseOn\u201d status\u201d, \u201ccategory\u201d, \u201ccode\u201d,\u2026\u2026.\u2026]<br \/>\n23. String ServiceRequest [] = [\u201cidentifier\u201d, \u201cbaseOn\u201d status\u201d, \u201ccategory\u201d, \u201crequester\u201d,\u2026\u2026]<br \/>\n24. String Appointment [] = [\u201cidentifier\u201d, \u201cstatus\u201d, \u201cappointmentType\u201d, \u201cpriority\u201d, \u2026\u2026\u2026\u2026]<br \/>\n25.\u2003\u2003 <b>If<\/b> (type == Patient) <b>then<\/b><br \/>\n26. \u2003\u2003 Result = \u201cPatient\u201d<br \/>\n27.\u2003\u2003 \u2003\u2003 <b>else if<\/b> (type == Condition) <b>then<\/b><br \/>\n28.\u2003\u2003\u2003\u2003 \u2003\u2003 Result = \u201cCondition\u201d<br \/>\n29.\u2003 \u2003\u2003\u2003\u2003 \u2003\u2003 <b>else if<\/b> (type == AllergyIntolerance) <b>then<\/b><br \/>\n30.\u2003 \u2003 \u2003\u2003\u2003\u2003 \u2003\u2003 Result = \u201cAllergyIntolerance\u201d<br \/>\n31.\u2003 \u2003 \u2003 \u2003\u2003\u2003\u2003 \u2003\u2003 <b>else if<\/b> (type == Practitioner) <b>then<\/b><br \/>\n32.\u2003 \u2003 \u2003 \u2003 \u2003\u2003\u2003\u2003 \u2003\u2003 Result = \u201cPractitioner\u201d<br \/>\n33.\u2003 \u2003 \u2003 \u2003 \u2003 \u2003\u2003\u2003\u2003 \u2003\u2003 <b>else if<\/b> (type == DiagnosticReport) <b>then<\/b><br \/>\n34.\u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003\u2003\u2003\u2003 \u2003\u2003 Result = \u201d DiagnosticReport\u201d<br \/>\n35.\u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003\u2003\u2003\u2003 \u2003\u2003 <b>else if<\/b> (type == ServiceRequest) <b>then<\/b><br \/>\n36.\u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003\u2003\u2003\u2003 \u2003\u2003 Result = \u201cServiceRequest\u201d<br \/>\n37.\u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003\u2003\u2003\u2003 \u2003\u2003 <b>else<\/b><br \/>\n38.\u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003\u2003\u2003\u2003 \u2003\u2003 Result = \u201cAppointment\u201d<br \/>\n39.\u2003\u2003 <b>end if<\/b><br \/>\n40. return (Result)<br \/>\n41. <b>end function<\/b> ** Resource-Templet function **<br \/>\n42. ** This function used to count the total number of tags in the resource **<br \/>\n43. <b>Function<\/b> int Count(String Templet)<br \/>\n44.\u2003\u2003 int counter = Templet.length<br \/>\n45. return (counter)<br \/>\n46. <b>end function<\/b> ** Count function **\n\n\n\n\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 6. Mapping algorithm working mechanism.\n\n\n\nFHIR-compliant database \nWe created a special database called the FHIR-compliant database. This is a relational database schema with a collection of tables that have been designed to store the data represented in FHIR resources. The tables are connected with each other, and each table stores clinical data represented in the FHIR resources.\nWe have multiple resources, and each resource represents different types of clinical data. Therefore, first we created a table schema according to the data represented in the FHIR resources and logically connected these tables to facilitate the analytic query engine to query the data from multiple tables according to the workflows in the result generation process. Each resource was stored in a single table or spread across multiple tables, for example, the patient resource data spread across multiple tables, etc. The table\u2019s creation and connection were specifically designed to cater to the needs of the proposed workflows and required result generation.\nSecond, we applied a mapping algorithm that enabled us to retrieve the data elements from the FHIR resources and store them accurately in the corresponding tables in the FHIR-compliant database. The algorithm retrieved the data from the FHIR resources and then pushed it to the corresponding table. When querying the data from the FHIR database, the FHIR query engine utilizes RESTful APIs to read the resources in JSON format. On the way, the mapping algorithm seamlessly pre-processed this JSON data and transformed it to the relational database schema. This process is completed automatically, and all data from all FHIR resources are transformed into the sample EMR data format and stored in the relational tables. Figure 7 presents the FHIR-compliant database.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 7. FHIR-compliant database: A sample schema of compliant database.\n\n\n\nData analytics engine layer \nThe data analytics layer plays a key role in this prototype. Once the FHIR resource data are seamlessly mapped to the relational database tables, they become ready for any data analytics operations. The data analytics is based on workflows (use cases scenarios) that we have already designed for optimal results.\nOur data analytics engine (DAE) is a collection of selective SQL queries proficient in merging data from multiple tables, thereby providing unparalleled data analysis. We have created a series of distinct SQL queries, catering to our business use cases, which are then executed on the data stored in the SQL database to generate exceptional results. The queries have been designed in alignment with our workflows and expected outcomes.\nThe resulting data are unequivocally valuable and accessible to the end-users via an intuitive and efficient user interface. The detail-oriented results generated by the data analytics engine are undoubtedly the backbone of our prototype, providing insights into the data.\n\nUser interface \nThe user interface of our framework is an elegant and sophisticated section, where the end-users access their desired data and obtain results catered specifically to their unique requirements. We developed a user-friendly graphical user interface (GUI) to efficiently process data and generate results.\nAs a demonstration of the utility of our prototype, we developed an experimental data analysis GUI that shows the use of the search operations within the generic tool for exploring FHIR data sets. We created a number of FHIR data sets and a graphic visualization of these data sets that allowed for the demonstration of the data analytics on the clinical data used in healthcare settings and represented in the FHIR-based standard. The user interface of our prototype is presented in Figure 8.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 8. Experimental user interface for data analytics.\n\n\n\n Methods\/Implementation \nIn this prototype or research work, we have stored our FHIR resource datasets in our NoSQL database (Mongo DB), which we had developed in our previous prototype. Therefore, we leverage the core FHIR APIs to perform data analytics on the data stored in these FHIR resources. We have employed the technique to download FHIR data into the FHIR-compliant database (SQL DB) and then applied data analytics to this FHIR data. For this purpose, we have utilized our developed mapping algorithm to transfer the FHIR resource data into the relational database tables. This has made it effortless to query data using standard SQL queries or tools and perform data analytics tasks on the data stored in these FHIR resources. It is essential to note that all the retrieval data from the FHIR resources require merging and formatting to support data analysis. As the patient\u2019s unique clinical identifier is the key to connecting these objects, we have utilized this number to merge the data into a group of tables in a relational database to support further querying and analysis.\nWe have an extensive array of FHIR resources stored in our database; therefore, we have utilized the core FHIR GET and Search APIs to retrieve all the resources from the database. These APIs have seamlessly accessed the FHIR resources from the FHIR database, and we have performed various data analytics tasks depending on the defined use cases. To provide our esteemed readers with a clear understanding of these APIs\u2019 working mechanisms, we have discussed how FHIR APIs work for data analytics. The prior Figure 2 illustrates a sample API operation mechanism in which the APIs access the clinical data and then perform the data analytics. This refers to the specialization of the FHIR API that focuses on providing the API\u2019s functionality that is useful for healthcare data analytics applications.\nThis implementation has been executed in two phases:\nPhase 1: We have developed various FHIR APIs to retrieve the FHIR resources from the FHIR database and then pre-process these resources using our developed algorithm to map the clinical data elements stored in the FHIR resource tags to a relational data model or schema and store the resulting data into the MySQL database. We have magnificently processed the FHIR resources via our algorithm, retrieved all data elements from these resources, and stored the result in database tables; we called it the FHIR-compliant database. For this purpose, we have crafted a database schema (tables) in the MySQL database (see the prior Figure 7). Each resource type requires different parameters in the REST API URL to retrieve the FHIR resource from the database. Therefore, for each resource, we have provided a resource name and parameters depending on the resource type and data retrieval. For this purpose, we have executed an algorithm to perform this job for us. When the FHIR APIs retrieve resources from the FHIR database, on the way, the mapping agent\/algorithm pre-processes the JSON format of FHIR resources and maps the data stored in various tags of JSON structure into the various MySQL database tables.\nPhase 2: When the data were converted from a graph structure to a relational data model format, we applied various data analytics techniques to the data stored in the MySQL database. For this purpose, we have developed various types of SQL queries to generate our results. These SQL queries have impeccably matched the requirements of our use cases, defined for our required data analytics. The output of these data analytics use cases is shown in the Section 7. Figure 9 shows the implementation process, while Figure 10 describes the complete framework process, including the techniques and computational tools applied in each step.\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 9. Described the implementation components and process.\n\n\n\n\n\n\n\n\n\n\n\n\nFigure 10. Framework working process: Describes each step working and implementation processing.\n\n\n\nExperiments \nWe implemented our data analytics prototype\/concept leveraging the FHIR database (Mongo DB), Python 3.9.8 programming language, and MySQL 5.6 database. We developed FHIR APIs, which enabled us to seamlessly retrieve various resources stored in the Mongo DB, consisting of a dataset size of 700 resources, inclusive of 100 resources of each resource type. Furthermore, before applying data analytics, we implemented our mapping algorithm\/agent, enabling the smooth transformation of FHIR resource tags to FHIR-compliant database (MYSQL) tables. We used the following:\n\nDataset size: 700 resources (including 100 resources of each resource type)\nHardware: 4 Cores, 32 GB of RAM\nSoftware: Windows 10 OS, Python 3.9.8 programming language, Mongo DB 4.4, MySQL 5.6 DB\nOur experiment consisted of two phases:\nPhase 1: In this step, we implemented our FHIR APIs and executed algorithms to retrieve the FHIR resources from the Mongo DB. Furthermore, we also executed a mapping algorithm to transform the FHIR resource data into the relational database tables.\nPhase 2: In this step, we executed various SQL queries to perform highly precise data analytics based on the defined use cases and generate the required results.\n\nResults \nTo provide the underlying data for our esteemed results, we used the data stored in the relational data model, generated from the FHIR dataset stored in Table 2. The results are based on the use cases we defined in our previous step. We have a number of use cases, each based on a dataset different from others. Therefore, we executed various queries based on the use cases. We generated various results from the FHIR dataset. We are discussing these use cases and their results in detail here.\n\nUse case 1 \nIn this scenario, the queries used within the patient\u2019s scalability count the number of patients that have been dutifully registered in the healthcare unit. These unparalleled queries seamlessly retrieve data from the patient table, which are associated with the esteemed patient resource in the FHIR dataset. Table 3 shows the retrieval data associated with patients gender-wise, and Figure 11 shows the graphical representation of this data.\n\n\n\n\n\n\n\nTable 3. Registered patients gender-wise (patient-centered-based analysis).\n\n\nMale\n\nFemale\n\n\n55\n\n45\n\n\n\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 11. Registered patients Gender-wise (patient-centered-based analysis).\n\n\n\nUse case 2 \nIn this scenario, the queries used within the patient\u2019s scalability count the number of patients registered within the healthcare unit across a variety of years. These queries are specifically designed to retrieve relevant data from the patient table, which are closely associated with the patient resource within the FHIR dataset. The queries retrieved the data related to patients who have been registered within the healthcare system over a span of several years, ranging from the year 1950 to the year 2021. Table 4 presents the retrieval of the registered patients\u2019 data in various years, while Figure 12 describes a graphical representation of this data.\n\n\n\n\n\n\n\nTable 4. Registered patients within a specified timeframe (patient-centered-based analysis).\n\n\nYear\n\n1950\n\n1951\n\n1952\n\n1953\n\n1955\n\n-----\n\n2013\n\n2018\n\n2021\n\n\nPatient's number\n\n3\n\n1\n\n3\n\n2\n\n1\n\n-----\n\n3\n\n1\n\n2\n\n\n\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 12. Registered patients within a specified timeframe (patient-centered-based analysis).\n\n\n\nUse case 3 \nIn this particular scenario, the patient\u2019s scalability has been measured by employing sophisticated queries aimed at counting the multitude of patients afflicted with diverse types of allergies. These queries were designed to extract relevant data from both the allergy and patient tables, which are associated with the \"Patient\" and \"AllergyIntolerance\" resources within the FHIR dataset. It joined data from these two tables because they belong to \"Patient\" and \"AllergyIntolerance\" resources and are spread across multiple tables and FHIR resources. Via these queries, relevant information relating to patients suffering from various allergies has been successfully retrieved. Table 5 presents the success of these queries, providing a comprehensive breakdown of the number of patients affected by different types of allergies. Furthermore, Figure 13 shows the graphical representation of this result.\n\n\n\n\n\n\n\nTable 5. Number of patients having various types of allergies (cohort-based interactive analyses).\n\n\nNumber\n\nAllergy\n\nNumber of patients\n\n\n1\n\nShellfish\n\n9\n\n\n2\n\nGlyburide\n\n8\n\n\n3\n\nLatex\n\n5\n\n\n4\n\nCoal tar\n\n6\n\n\n5\n\nNeomycin\n\n12\n\n\n6\n\nCodeine\n\n8\n\n\n7\n\nIVP dye\n\n10\n\n\n8\n\nCaffeine\n\n5\n\n\n9\n\nLevaquin\n\n5\n\n\n10\n\nSeafood\n\n6\n\n\n11\n\nRifampin\n\n3\n\n\n12\n\nNorco\n\n6\n\n\n13\n\nPenicillium\n\n5\n\n\n14\n\nBenztropine\n\n6\n\n\n15\n\nWatermelon\n\n3\n\n\n16\n\nMetoprolol\n\n2\n\n\n17\n\nIV dye\n\n1\n\n\n\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 13. Patients and various types of allergies association (cohort-based interactive analyses).\n\n\n\nUse case 4 \nIn this particular scenario, the queries employed in the patient\u2019s scalability quantify the number of distinct medical tests that have been requested by either a healthcare organization or a practitioner. These queries procure data from various tables, including patient, order, provider, practitioner, etc., which are associated with the \"Patient,\" \"Practitioner,\" \"DiagnosticReport,\" and \"ServiceRequest\" resources in the FHIR dataset. The queries retrieved information related to various types of test orders that are present within the healthcare system. Table 6 represents the various types of medical tests undertaken by the patient. Additionally, the graphical representation of this data is illustrated in Figure 14.\n\n\n\n\n\n\n\nTable 6. Patient various types of medical test orders (cohort-based interactive analyses).\n\n\nTest name\n\nHIV\n\nCBC\n\nCT scan\n\nX-ray, ankle\n\nMRI\n\nBlood culture\n\nCOVID\n\nSGPT\n\n\nTest order percentage\n\n16\n\n15\n\n15\n\n14\n\n12\n\n10\n\n9\n\n9\n\n\n\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 14. Patient various types of medical test orders (cohort-based interactive analyses).\n\n\n\n\r\n\n\nUse case 5 \nIn this scenario, the queries employed within the patient\u2019s scalability counts the number of sundry categories of medical tests ordered by a healthcare organization or practitioner in different years. These queries procure data from various tables, including patient, order, provider, practitioner, etc., which are associated with the \"Patient,\" \"Practitioner,\" \"DiagnosticReport,\" and \"ServiceRequest\" resources in the FHIR dataset. These queries have retrieved relevant information regarding assorted test orders in the healthcare system spanning a timeline from 1950 to 2021. The resulting outcome of these queries has been presented in Table 7, summarizing the diverse medical tests undertaken by the patient. Additionally, Figure 15 describes the graphical representation of this information.\n\n\n\n\n\n\n\nTable 7. Patient various types of medical test orders within a specified timeframe (cohort-based interactive analyses).\n\n\nYear\n\n1951\n\n1952\n\n1953\n\n1955\n\n-----\n\n2010\n\n2015\n\n2019\n\n2020\n\n\nNumber of tests ordered\n\n4\n\n4\n\n3\n\n2\n\n-----\n\n5\n\n3\n\n6\n\n5\n\n\n\n\r\n\n\n\n\n\n\n\n\n\n\nFigure 15. Patient various types of medical test orders within a specified timeframe (cohort-based interactive analyses).\n\n\n\nLimitations \nOur developed framework is capable of performing various types of descriptive data analytics on clinical data used in healthcare settings and represented in the FHIR-based standard. However, it is important to note that our study is limited in that it focuses solely on business use cases for patient clinical data belonging to two HID: PRSs and LISs. Other possible data analysis workflows and customized research scenarios based on patient data from other HISs could be performed on FHIR-based data, but our current framework or tool does not directly support them without modification. In addition, there are some technical challenges in this research work:\n\nOur framework is currently developed under the FHIR R4 version and needs to be upgraded to the official FHIR R5 version when it gets finalized and released by HL7.\nOur framework might face issues in the coming FHIR version. HL7 FHIR specification requirements are changing over time, and the current resources might be replaced with any other new resources in the coming FHIR version. Additionally, the resource nature (from non-normative to normative) is changing over time. In this case, our framework might face challenges. Therefore, it needs to be updated in the coming FHIR versions if any of the mentioned cases happen. However, if none of these changes happen in the FHIR R5 version, it will work perfectly.\nOur framework executed multiple algorithms, such as the algorithm for accessing the FHIR resources via the RESTful APIs and the algorithm to map data from the FHIR resources to the EMR data format, and executed queries to perform data analytics for the end users. Therefore, the performance might not be ideal for every dataset. It worked excellently for our dataset (which is small), but the performance might be affected when dealing with large datasets, for example, when the number of resources and data elements in the dataset is in the billions or trillions.\nThe interface of our framework works for our dataset (patient data used in PRSs and LISs); therefore, it would update if the workflow changed and included the data from other HISs.\nDiscussion \nIn this study, we have developed an integrated framework or visual tool leveraging the cutting-edge FHIR standard, with prototype implementation and evaluation, aiming to empower standardized clinical statistics and analysis applications. This research work has three main contributions: First, the entire framework and workflow design follow the FHIR data standards, which could be reused for any other clinical data domain and could provide support for any clinical data that follows the FHIR standard. Second, the data analysis workflow and tools incorporate the experience of clinical researchers and statisticians and leverage powerful Python analytics, which could provide a starting point for FHIR researchers in this cutting-edge standard. Third, the intelligent mapping algorithm, artfully designed to facilitate the sublime process of data analytics or data analysis within the realm of FHIR-based data. The mapping algorithm could be reused for any other clinical data that follow the FHIR specification and need to process the FHIR-based data for other purposes, such as research or developing an AI or ML model, etc.\nOur research effectively used the data-mapping algorithm for FHIR-based data to facilitate the data analytics process. Furthermore, mapped data could be utilized for other purposes, such as research, etc. Although recently, another technique, namely pathling[14], has been used for data analytics on FHIR-based data. However, it works in a limited domain because some operations are not easily or even currently possible to achieve via the FHIR REST API specification, such as data aggregation, searching the data, etc. Therefore, it is extremely challenging to implement. Furthermore, this technique is language-specific. Therefore, it needs to redesign the entire framework for a new language. Our technique is easy to implement and generally could be used for all FHIR-based data types and FHIR resources with minor modifications. Furthermore, the implementation process would work for every language.\nOur developed framework or tool provides a user-friendly GUI to the end-users, such as healthcare professionals and researchers. The developed interface is used for FHIR data mapping and analytics purposes. Therefore, we developed two sub-menus, one for data mapping and a second for data analytic purposes (see the prior Figure 8). However, we only discussed the data analytics sub-menu in this research work. The data mapping sub-menu is out of the scope of this study. Our data analytics sub-menu provided all options for our required results based on the defined use-cases. For example, the \u201cRegistered Patients\u201d option provided results for all registered patients in the patient information systems. Similarly, \u201cTest Order\u201d generates the results of various types of patient laboratory tests ordered by any practitioner, healthcare organization, laboratory, etc. All the remaining options work accordingly. In short, it could greatly facilitate interactive, user-friendly data analysis.\nIn the future, we have a plan to extend our framework by adding data from other HISs and updating the framework, including the data workflows and user interface, to make it more generic for users and researchers. Furthermore, we also intend to adopt the FHIR R5 version with particular COVID-19 and cancer-related resource definitions to represent COVID-19 and cancer data in our framework. This will help people working in the healthcare industry to enhance the consistency and quality of data analysis for COVID-19 and cancer data. Moreover, it will open more research dimensions for healthcare data analytic researchers in these areas.\n\nConclusions \nIn this study, we discussed the need for a data analytics tool to improve data analysis and reduce the skill burden in the healthcare industry. We have designed a comprehensive framework that empowers healthcare users (patients, practitioners, healthcare providers, etc.) to perform advanced data analysis on patient data used in healthcare settings and represented in the FHIR-based standard. The framework incorporates different data workflows based on patient data derived from two HISs, namely PRSs and LISs, represented in the FHIR-based standard. Our use cases facilitate both patient-centered and cohort-based analysis and address common clinical user and researcher requirements. Although currently limited to two HISs, the framework is flexible and can be extended to include data from other systems represented in the FHIR-based standard. With ongoing improvements, our framework will be valuable for healthcare applications in statistics and analytics. Overall, the goal of developing a state-of-the-art data analytics framework for clinical data in healthcare settings has been achieved.\n\nAcknowledgements \nAuthor contributions \nConceptualization, M.A.; Methodology, M.A. and H.K.A.; software, M.A.; validation, M.A.; Formal analysis, T.J.A. and H.K.A.; Investigation, N.N.B.A., M.A., and M.F.P.; Data curation, N.N.B.A. and T.J.A.; Writing\u2014original draft, M.A.; Writing\u2014review & editing, M.A.; visualization, M.A.; Supervision, M.F.P. All authors have read and agreed to the published version of the manuscript.\n\nFunding \nThis research supported by Princess Nourah bint Abdulrahman University Researchers Supporting Project number (PNURSP2023R384), Princess Nourah bint Abdulrahman University, and P.O. Box 84428, Riyadh 11671, Saudi Arabia.\n\nInformed consent \nInformed consent was obtained from all subjects involved in the study. In this manuscript, we used data from the MIMIC-III database. The establishment of this database was approved by the Massachusetts Institute of Technology (Cambridge, MA, USA) and Beth Israel Deaconess Medical Center (Boston, MA, USA), and consent was obtained for the original data collection. Therefore, the ethical approval statement and the need for informed consent were waived for this manuscript.\n\nData availability statement \nThe datasets used or analyzed in this study are available from the corresponding author on reasonable request.\n\nConflict of interest \nAll authors declare that they have no conflict of interest.\n\nReferences \n\n\n\u2191 1.0 1.1 Safran, C.; Bloomrosen, M.; Hammond, W. E.; Labkoff, S.; Markel-Fox, S.; Tang, P. C.; Detmer, D. E. (1 January 2007). \"Toward a National Framework for the Secondary Use of Health Data: An American Medical Informatics Association White Paper\" (in en). Journal of the American Medical Informatics Association 14 (1): 1\u20139. doi:10.1197\/jamia.M2273. ISSN 1067-5027. PMC PMC2329823. PMID 17077452. https:\/\/academic.oup.com\/jamia\/article-lookup\/doi\/10.1197\/jamia.M2273 .   \n \n\n\u2191 Trotter, F. (20 August 2012). \"Who Owns Patient Data?\". The Health Care Blog. Holt, M.. https:\/\/thehealthcareblog.com\/blog\/2012\/08\/20\/who-owns-patient-data\/ . Retrieved 04 December 2022 .   \n \n\n\u2191 3.0 3.1 3.2 3.3 3.4 Hersh, W.R. (2014). \"Chapter 3: Healthcare Data Analytics\". In Hoyt, Robert E.; Yoshihashi, Ann. Health informatics: practical guide for healthcare and information technology professionals (Fifth ed.). Informatics Education. ISBN 978-1-304-79110-8. https:\/\/dmice.ohsu.edu\/hersh\/hoyt-14-analytics.pdf .   \n \n\n\u2191 Davenport, Thomas H.; Harris, Jeanne G. (2007). Competing on analytics: the new science of winning. Boston, Mass: Harvard Business School Press. ISBN 978-1-4221-0332-6. OCLC ocm74649067. https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/ocm74649067 .   \n \n\n\u2191 Cortada, J.W.; Gordon, D.; Leniban, B. (January 2012). \"The value of analytics in healthcare: From insights to outcomes\" (PDF). IBM Institute for Business Value. Archived from the original on 11 August 2023. https:\/\/web.archive.org\/web\/20230811171408\/https:\/\/dl200.dokumen.tips\/file_download\/file1\/55cf864e550346484b9653d8?ext=pdf&vaild=0dd1e37e9f43ee23f6c4498f7ea6068f889867446631bb6d00f103b2be23026403af450c3f0533792a3d70fefb74d4a3224e33087229ddc130762dfe529d5d3eGwX9zBiXxNLqkR1dRP8JhSIvoVq8eA7QwvpZf4WRyR8XqCxum0FIGy%2FtXjig%2BNXcyJEVYfvEgLNvMpc7ieUN%2BiIwx2Xor%2F0hLDjO1tU%2FGAk%3D . Retrieved 04 December 2022 .   \n \n\n\u2191 Adams, J.; Klein, J. (2011). \"Business Intelligence and Analytics in Health Care\u2014A Primer\". The Advisory Board Company. https:\/\/www.advisory.com\/Research\/IT-Strategy-Council\/Research-Notes\/2011\/Business-Intelligence-and-Analytics-in-Health-C . Retrieved 06 December 2022 .   \n \n\n\u2191 Gardner, E. (1 March 2023). \"The HIT Approach to Big Data\". HealthData Management. SourceMedia. Archived from the original on 02 March 2013. https:\/\/web.archive.org\/web\/20130302053700\/http:\/\/www.healthdatamanagement.com\/issues\/21_3\/The-HIT-Approach-to-Big-Data-Anayltics-45735-1.html . Retrieved 30 November 2022 .   \n \n\n\u2191 Sledge, G.W.; Miller, R.S.; Hauser. R. (3 June 2013). \"CancerLinQ and the Future of Cancer Care\". ASCO Meeting Library. ASCO University. Archived from the original on 02 June 2018. https:\/\/web.archive.org\/web\/20180602212656\/https:\/\/meetinglibrary.asco.org\/record\/78971\/edbook . Retrieved 02 December 2022 .   \n \n\n\u2191 Ayaz, Muhammad; Pasha, Muhammad F.; Alzahrani, Mohammed Y.; Budiarto, Rahmat; Stiawan, Deris (30 July 2021). \"The Fast Health Interoperability Resources (FHIR) Standard: Systematic Literature Review of Implementations, Applications, Challenges and Opportunities\" (in EN). JMIR Medical Informatics 9 (7): e21929. doi:10.2196\/21929. https:\/\/medinform.jmir.org\/2021\/7\/e21929 .   \n \n\n\u2191 Centers for Medicare & Medicaid Services (2023). \"FHIR - Fast Healthcare Interoperability Resources\". eCQI Resource Center. Centers for Medicare & Medicaid Services. https:\/\/ecqi.healthit.gov\/fhir . Retrieved 10 February 2023 .   \n \n\n\u2191 Braunstein, Mark L. (2018), \"SMART on FHIR\" (in en), Health Informatics on FHIR: How HL7's New API is Transforming Healthcare (Cham: Springer International Publishing): 205\u2013225, doi:10.1007\/978-3-319-93414-3_10, ISBN 978-3-319-93413-6, http:\/\/link.springer.com\/10.1007\/978-3-319-93414-3_10 . Retrieved 2023-08-11   \n \n\n\u2191 12.0 12.1 Staff Reporter, APAC (28 November 2022). \"Analytics and Data-Driven Healthcare to Be Fuelled by FHIR Interoperability Boost: InterSystems ANZ Study\". HealthcareAsia. https:\/\/healthcareasiamagazine.com\/co-written-partner\/analytics-and-data-driven-healthcare-be-fuelled-fhir-interoperability-boost-intersystems-anz-study . Retrieved 04 December 2022 .   \n \n\n\u2191 Ostrovskiy, S. (10 November 2021). \"What Is FHIR: A Brief Overview of Its Role in Interoperability\". Edenlab. https:\/\/edenlab.io\/blog\/what-is-fhir-a-brief-overview-of-its-role-in-interoperability . Retrieved 04 December 2022 .   \n \n\n\u2191 14.0 14.1 14.2 14.3 14.4 14.5 14.6 14.7 Grimes, John; Szul, Piotr; Metke-Jimenez, Alejandro; Lawley, Michael; Loi, Kylynn (8 September 2022). \"Pathling: analytics on FHIR\" (in en). Journal of Biomedical Semantics 13 (1): 23. doi:10.1186\/s13326-022-00277-1. ISSN 2041-1480. PMC PMC9455941. PMID 36076268. https:\/\/jbiomedsem.biomedcentral.com\/articles\/10.1186\/s13326-022-00277-1 .   \n \n\n\u2191 15.0 15.1 Lehne, Moritz; Luijten, Sandra; Vom Felde Genannt Imbusch, Paulina; Thun, Sylvia (2019). \"The Use of FHIR in Digital Health \u2013 A Review of the Scientific Literature\". German Medical Data Sciences: Shaping Change \u2013 Creative Solutions for Innovative Medicine: 52\u201358. doi:10.3233\/SHTI190805. https:\/\/ebooks.iospress.nl\/doi\/10.3233\/SHTI190805 .   \n \n\n\u2191 \"FHIR Analytics in Healthcare\". Qrvey, Inc. https:\/\/qrvey.com\/fhir-healthcare-analytics\/ . Retrieved 05 December 2022 .   \n \n\n\u2191 Ajibade, Samuel-Soma M.; Ayaz, Muhammad; Ngo-Hoang, Dai-Long; Tabuena, Almighty C.; Rabbi, Fazle; Tilaye, Getahun; Bassey, Mbiatke Anthony (25 June 2022). \"Analysis of Improved Evolutionary Algorithms Using Students\u2019 Datasets\". 2022 IEEE International Conference on Automatic Control and Intelligent Systems (I2CACIS) (Shah Alam, Malaysia: IEEE): 180\u2013185. doi:10.1109\/I2CACIS54679.2022.9815272. ISBN 978-1-6654-9581-3. https:\/\/ieeexplore.ieee.org\/document\/9815272\/ .   \n \n\n\u2191 Rabbi, Fazle; Ayaz, Muhammad; Dayupay, Johnry P.; Oyebode, Oluwadare Joshua; Gido, Nathaniel G.; Adhikari, Nirmal; Tabuena, Almighty C.; Ajibade, Samuel-Soma M. et al. (23 July 2022). \"Gaussian Map to Improve Firefly Algorithm Performance\". 2022 IEEE 13th Control and System Graduate Research Colloquium (ICSGRC) (Shah Alam, Malaysia: IEEE): 88\u201392. doi:10.1109\/ICSGRC55096.2022.9845171. ISBN 978-1-6654-6806-0. https:\/\/ieeexplore.ieee.org\/document\/9845171\/ .   \n \n\n\u2191 Ajibade, Samuel-Soma M.; Zaidi, Abdelhamid; Tapales, Catherine P.; Ngo-Hoang, Dai-Long; Ayaz, Muhammad; Dayupay, Johnry P.; Aminu Dodo, Yakubu; Chaudhury, Sushovan et al. (17 December 2022). \"Data Mining Analysis of Online Drug Reviews\". 2022 IEEE 10th Conference on Systems, Process & Control (ICSPC) (Malacca, Malaysia: IEEE): 247\u2013251. doi:10.1109\/ICSPC55597.2022.10001810. ISBN 978-1-6654-7098-8. https:\/\/ieeexplore.ieee.org\/document\/10001810\/ .   \n \n\n\u2191 20.0 20.1 Giannangelo, Kathy; Fenton, Susan H. (20 May 2008). \"SNOMED CT Survey: An Assessment of Implementation in EMR\/EHR Applications\". Perspectives in Health Information Management \/ AHIMA, American Health Information Management Association 5: 7. ISSN 1559-4122. PMC 2396499. PMID 18509501. https:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC2396499\/ .   \n \n\n\u2191 Ayaz, M. (2017). \"Cloud Computing Base Electronic Health Record System Architecture for Disabled Children\". International Journal of Multidisciplinary Sciences and Engineering 8 (2): 24\u201328. http:\/\/www.ijmse.org\/Volume8\/Issue2.html .   \n \n\n\u2191 Bresnick, J. (8 May 2017). \"48% of Businesses, Including Healthcare, Face Big Data Skills Gap\". Health IT Analytics. TechTarget. https:\/\/healthitanalytics.com\/news\/48-of-businesses-including-healthcare-face-big-data-skills-gap . Retrieved 06 December 2022 .   \n \n\n\u2191 Ayaz, M. (2017). \"A Novel Model of Software Process Improvements for Small and Medium Scale Enterprises by using the Big Data Analytics Approach\". International Journal of Multidisciplinary Sciences and Engineering 8 (3): 1\u201310. http:\/\/www.ijmse.org\/Volume8\/Issue3.html .   \n \n\n\u2191 Ayaz. M. (2017). \"A Seminal Hybrid Business Process Management Model\". International Journal of Multidisciplinary Sciences and Engineering 8 (2): 38\u201342. http:\/\/www.ijmse.org\/Volume8\/Issue2.html .   \n \n\n\u2191 25.0 25.1 Hong, Na; Prodduturi, Naresh; Wang, Chen; Jiang, Guoqian (2017). \"Shiny FHIR: An Integrated Framework Leveraging Shiny R and HL7 FHIR to Empower Standards-Based Clinical Data Applications\". MEDINFO 2017: Precision Healthcare through Informatics: 868\u2013872. doi:10.3233\/978-1-61499-830-3-868. https:\/\/ebooks.iospress.nl\/doi\/10.3233\/978-1-61499-830-3-868 .   \n \n\n\u2191 Ayaz, Muhammad; Pasha, Muhammad Fermi; Le, Tham Yu; Alahmadi, Tahani Jaser; Abdullah, Nik Nailah Binti; Alhababi, Zaid Ali (30 January 2023). \"A Framework for Automatic Clustering of EHR Messages Using a Spatial Clustering Approach\" (in en). Healthcare 11 (3): 390. doi:10.3390\/healthcare11030390. ISSN 2227-9032. PMC PMC9914110. PMID 36766965. https:\/\/www.mdpi.com\/2227-9032\/11\/3\/390 .   \n \n\n\u2191 Shortliffe, Edward Hance; Cimino, James J.; Chiang, Michael F., eds. (2021). Biomedical Informatics: Computer applications in health care and biomedicine (5th edition ed.). Cham, Switzerland: Springer. ISBN 978-3-030-58720-8.   \n \n\n\u2191 Reddy, Chandan K.; Aggarwal, Charu C., eds. (23 June 2015) (in en). Healthcare Data Analytics (0 ed.). Chapman and Hall\/CRC. doi:10.1201\/b18588. ISBN 978-1-4822-3212-7. https:\/\/www.taylorfrancis.com\/books\/9781482232127 .   \n \n\n\u2191 Hripcsak, George; Duke, Jon D.; Shah, Nigam H.; Reich, Christian G.; Huser, Vojtech; Schuemie, Martijn J.; Suchard, Marc A.; Park, Rae Woong et al. (2015). \"Observational Health Data Sciences and Informatics (OHDSI): Opportunities for Observational Researchers\". MEDINFO 2015: eHealth-enabled Health: 574\u2013578. doi:10.3233\/978-1-61499-564-7-574. https:\/\/ebooks.iospress.nl\/doi\/10.3233\/978-1-61499-564-7-574 .   \n \n\n\u2191 Hripcsak, George; Duke, Jon D.; Shah, Nigam H.; Reich, Christian G.; Huser, Vojtech; Schuemie, Martijn J.; Suchard, Marc A.; Park, Rae Woong et al. (2015). \"Observational Health Data Sciences and Informatics (OHDSI): Opportunities for Observational Researchers\". MEDINFO 2015: eHealth-enabled Health: 574\u2013578. doi:10.3233\/978-1-61499-564-7-574. https:\/\/ebooks.iospress.nl\/doi\/10.3233\/978-1-61499-564-7-574 .   \n \n\n\u2191 \"ATLAS - A unified interface for the OHDSI tools\". GitHub. 30 May 2019. https:\/\/github.com\/OHDSI\/Atlas\/wiki .   \n \n\n\u2191 Ma, Chunlan; Frankel, Heath; Beale, Thomas; Heard, Sam (2007). \"EHR query language (EQL)--a query language for archetype-based health records\". Studies in Health Technology and Informatics 129 (Pt 1): 397\u2013401. ISSN 0926-9630. PMID 17911747. https:\/\/pubmed.ncbi.nlm.nih.gov\/17911747 .   \n \n\n\u2191 The openEHR Foundation (4 February 2021). \"openEHR - Archetype Query Language (AQL)\". https:\/\/specifications.openehr.org\/releases\/QUERY\/latest\/AQL.html . Retrieved 10 August 2022 .   \n \n\n\u2191 Karim, Md Rezaul; Nguyen, Binh-Phi; Zimmermann, Lukas; Kirsten, Toralf; L\u00f6be, Matthias; Meineke, Frank; Stenzhorn, Holger; Kohlbacher, Oliver et al. (2018) (in en). A Distributed Analytics Platform to Execute FHIR based Phenotyping Algorithms. doi:10.15496\/publikation-28068. https:\/\/publikationen.uni-tuebingen.de\/xmlui\/handle\/10900\/86681 .   \n \n\n\u2191 Lakshman, V.; Amrollahi, F.; Koppisetty, V.S. et al. (2018). \"DeepAISE on FHIR\u2014An Interoperable Real-Time Predictive Analytic Platform for Early Prediction of Sepsis\". Proceedings of the AMIA Annual Symposium. https:\/\/par.nsf.gov\/servlets\/purl\/10084140 . Retrieved 12 December 2022 .   \n \n\n\u2191 Khalilia, Mohammed; Choi, Myung; Henderson, Amelia; Iyengar, Sneha; Braunstein, Mark; Sun, Jimeng (2015). \"Clinical Predictive Modeling Development and Deployment through FHIR Web Services\". AMIA ... Annual Symposium proceedings. AMIA Symposium 2015: 717\u2013726. ISSN 1942-597X. PMC 4765683. PMID 26958207. https:\/\/pubmed.ncbi.nlm.nih.gov\/26958207 .   \n \n\n\u2191 doc.ai (10 May 2018). \"doc.ai is on fire. Oops we mean FHIR:)\". Medium. https:\/\/medium.com\/@_doc_ai\/doc-ai-is-on-fire-oops-we-mean-fhir-ea2912b2864b . Retrieved 30 January 2023 .   \n \n\n\u2191 Semler, Sebastian; Wissing, Frank; Heyder, Ralf (1 July 2018). \"German Medical Informatics Initiative: A National Approach to Integrating Health Data from Patient Care and Medical Research\" (in en). Methods of Information in Medicine 57 (S 01): e50\u2013e56. doi:10.3414\/ME18-03-0003. ISSN 0026-1270. PMC PMC6178199. PMID 30016818. http:\/\/www.thieme-connect.de\/DOI\/DOI?10.3414\/ME18-03-0003 .   \n \n\n\u2191 Kreuzthaler, Markus; Mart&#237; nez-Costa, Catalina; Kaiser, Peter; Schulz, Stefan (2017). \"Semantic Technologies for Re-Use of Clinical Routine Data\". Health Informatics Meets eHealth: 24\u201331. doi:10.3233\/978-1-61499-759-7-24. https:\/\/ebooks.iospress.nl\/doi\/10.3233\/978-1-61499-759-7-24 .   \n \n\n\u2191 Franz, Barbara (2015). \"Applying FHIR in an Integrated Health Monitoring System\". European Journal for Biomedical Informatics 11 (02). doi:10.24105\/ejbi.2015.11.2.8. https:\/\/www.ejbi.org\/scholarly-articles\/applying-fhir-in-an-integrated-health-monitoring-system.pdf .   \n \n\n\u2191 Liu, Dianbo; Sahu, Ricky; Ignatov, Vlad; Gottlieb, Dan; Mandl, Kenneth D. (4 March 2020). \"High Performance Computing on Flat FHIR Files Created with the New SMART\/HL7 Bulk Data Access Standard\". AMIA Annual Symposium Proceedings 2019: 592\u2013596. ISSN 1942-597X. PMC 7153160. PMID 32308853. https:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC7153160\/ .   \n \n\n\u2191 \"Apache Parquet\". Google. https:\/\/parquet.apache.org\/ . Retrieved 10 August 2022 .   \n \n\n\u2191 Dunn, Tim; Cosgun, Erdal (5 January 2023). Arighi, Cecilia. ed. \"A cloud-based pipeline for analysis of FHIR and long-read data\" (in en). Bioinformatics Advances 3 (1): vbac095. doi:10.1093\/bioadv\/vbac095. ISSN 2635-0041. PMC PMC9872570. PMID 36726729. https:\/\/academic.oup.com\/bioinformaticsadvances\/article\/doi\/10.1093\/bioadv\/vbac095\/6994207 .   \n \n\n\u2191 Gruendner, Julian; Gulden, Christian; Kampf, Marvin; Mate, Sebastian; Prokosch, Hans-Ulrich; Zierk, Jakob (1 April 2021). \"A Framework for Criteria-Based Selection and Processing of Fast Healthcare Interoperability Resources (FHIR) Data for Statistical Analysis: Design and Implementation Study\" (in en). JMIR Medical Informatics 9 (4): e25645. doi:10.2196\/25645. ISSN 2291-9694. PMC PMC8050750. PMID 33792554. https:\/\/medinform.jmir.org\/2021\/4\/e25645 .   \n \n\n\u2191 \"cerner \/ bunsen\". GitHub. 20 November 2020. https:\/\/github.com\/cerner\/bunsen . Retrieved 10 August 2022 .   \n \n\n\u2191 Zaharia, M.; Chowdhury, M.; Frankling, M.J. et al. (2010). \"Spark: Cluster Computing with Working Sets\" (PDF). pp. 1\u20137. https:\/\/www1.icsi.berkeley.edu\/pubs\/networking\/ICSI_sparkclustercomputing10.pdf . Retrieved 21 December 2022 .   \n \n\n\u2191 Brush, R.; Mandel, J. (2023). \"SQL on FHIR\". GitHub. https:\/\/github.com\/FHIR\/sql-on-fhir . Retrieved 10 August 2022 .   \n \n\n\u2191 \"Protocol Buffers\". Protocol Buffers Documentation. Google, LLC. 2022. https:\/\/protobuf.dev\/ . Retrieved 10 August 2022 .   \n \n\n\u2191 \"google \/ fhir\". GitHub. 2022. https:\/\/github.com\/google\/fhir . Retrieved 10 August 2022 .   \n \n\n\u2191 Chong, Dazhi; Shi, Hui (3 July 2015). \"Big data analytics: a literature review\" (in en). Journal of Management Analytics 2 (3): 175\u2013201. doi:10.1080\/23270012.2015.1082449. ISSN 2327-0012. http:\/\/www.tandfonline.com\/doi\/full\/10.1080\/23270012.2015.1082449 .   \n \n\n\nNotes \nThis presentation is faithful to the original, with only a few minor changes to presentation, though grammar and word usage was substantially updated for improved readability. In some cases important information was missing from the references, and that information was added. In the original, citations three and four are identical; for this version, those citations were combined, making the total citation count one less than the original 50. Numerous cited URLs from the original were broken; suitable archived versions were found for this version. In some cases, a suitable archived version could not be found.\n\n\n\n\n\nSource: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.limswiki.org\/index.php\/Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\">https:\/\/www.limswiki.org\/index.php\/Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data<\/a>\nCategories: LIMSwiki journal articles (added in 2023)LIMSwiki journal articles (all)LIMSwiki journal articles on clinical researchLIMSwiki journal articles on data analysisLIMSwiki journal articles on health informaticsNavigation menuPage actionsJournalDiscussionView sourceHistoryPage actionsJournalDiscussionMoreToolsIn other languagesPersonal toolsLog inNavigationMain pageEncyclopedic articlesRecent changesRandom pageHelp about MediaWikiSearch\u00a0 ToolsWhat links hereRelated changesSpecial pagesPermanent linkPage informationPopular publications\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\n\n\t\r\nPrint\/exportCreate a bookDownload as PDFDownload as PDFDownload as Plain textPrintable version This page was last edited on 11 August 2023, at 18:40.Content is available under a Creative Commons Attribution-ShareAlike 4.0 International License unless otherwise noted.This page has been accessed 1,059 times.Privacy policyAbout LIMSWikiDisclaimers\n\n\n\n","96ca1abdbcc7bf60389fe942b678382d_html":"<body class=\"mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-206 ns-subject page-Journal_Transforming_healthcare_analytics_with_FHIR_A_framework_for_standardizing_and_analyzing_clinical_data rootpage-Journal_Transforming_healthcare_analytics_with_FHIR_A_framework_for_standardizing_and_analyzing_clinical_data skin-monobook action-view skin--responsive\"><div id=\"rdp-ebb-globalWrapper\"><div id=\"rdp-ebb-column-content\"><div id=\"rdp-ebb-content\" class=\"mw-body\" role=\"main\"><a id=\"rdp-ebb-top\"><\/a>\n<h1 id=\"rdp-ebb-firstHeading\" class=\"firstHeading\" lang=\"en\">Journal:Transforming healthcare analytics with FHIR: A framework for standardizing and analyzing clinical data<\/h1><div id=\"rdp-ebb-bodyContent\" class=\"mw-body-content\"><!-- start content --><div id=\"rdp-ebb-mw-content-text\" lang=\"en\" dir=\"ltr\" class=\"mw-content-ltr\"><div class=\"mw-parser-output\">\n\n\n<h2><span class=\"mw-headline\" id=\"Abstract\">Abstract<\/span><\/h2>\n<p>In this study, we discuss our contribution to building a <a href=\"https:\/\/www.limswiki.org\/index.php\/Data_analysis\" title=\"Data analysis\" class=\"wiki-link\" data-key=\"545c95e40ca67c9e63cd0a16042a5bd1\">data analytic<\/a> framework that supports clinical statistics and analysis by leveraging a scalable standards-based data model named <a href=\"https:\/\/www.limswiki.org\/index.php\/Fast_Healthcare_Interoperability_Resources\" title=\"Fast Healthcare Interoperability Resources\" class=\"wiki-link\" data-key=\"65dd2a848285f9151006e17e036d596d\">Fast Healthcare Interoperability Resources<\/a> (FHIR). We developed an intelligent algorithm that is used to facilitate the clinical data analytics process on FHIR-based data. We designed several <a href=\"https:\/\/www.limswiki.org\/index.php\/Workflow\" title=\"Workflow\" class=\"wiki-link\" data-key=\"92bd8748272e20d891008dcb8243e8a8\">workflows<\/a> for patient clinical data used in two <a href=\"https:\/\/www.limswiki.org\/index.php\/Hospital_information_system\" title=\"Hospital information system\" class=\"wiki-link\" data-key=\"d8385de7b1f39a39d793f8ce349b448d\">hospital information systems<\/a> (HISs), namely patient registration systems (PRSs) and <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory_information_system\" title=\"Laboratory information system\" class=\"wiki-link\" data-key=\"37add65b4d1c678b382a7d4817a9cf64\">laboratory information systems<\/a> (LIS). These workflows exploit various FHIR <a href=\"https:\/\/www.limswiki.org\/index.php\/Application_programming_interface\" title=\"Application programming interface\" class=\"wiki-link\" data-key=\"36fc319869eba4613cb0854b421b0934\">application programming interfaces<\/a> (API) to facilitate patient-centered and cohort-based interactive analyses. We developed a FHIR database implementation that utilizes FHIR APIs and a range of operations to facilitate descriptive data analytics (DDA) and patient cohort selection. A prototype user interface for DDA was developed with support for visualizing healthcare data analysis results in various forms. Healthcare professionals and researchers would use the developed framework to perform analytics on clinical data used in healthcare settings. Our experimental results demonstrate the proposed framework\u2019s ability to generate various analytics from clinical data represented in the FHIR resources.\n<\/p><p><b>Keywords<\/b>: data analytics, data analysis, FHIR, EMR, EHR\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Background\">Background<\/span><\/h2>\n<p>To provide a comprehensive idea to readers about the applications of <a href=\"https:\/\/www.limswiki.org\/index.php\/Data_analysis\" title=\"Data analysis\" class=\"wiki-link\" data-key=\"545c95e40ca67c9e63cd0a16042a5bd1\">data analytics<\/a> in the healthcare industry, this section introduces the data analytics concept employed in the healthcare sector. We also discuss the data analytics concept in the clinical data represented in the latest healthcare data standard, <a href=\"https:\/\/www.limswiki.org\/index.php\/Fast_Healthcare_Interoperability_Resources\" title=\"Fast Healthcare Interoperability Resources\" class=\"wiki-link\" data-key=\"65dd2a848285f9151006e17e036d596d\">Fast Healthcare Interoperability Resources<\/a> (FHIR).\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Healthcare_data_analytics\">Healthcare data analytics<\/span><\/h3>\n<p>Healthcare data analytics is the process of analyzing and interpreting large sets of healthcare data to gain insights and improve healthcare outcomes. It involves using a range of analytical techniques and tools to process data from various sources, such as <a href=\"https:\/\/www.limswiki.org\/index.php\/Electronic_health_record\" title=\"Electronic health record\" class=\"wiki-link\" data-key=\"f2e31a73217185bb01389404c1fd5255\">electronic health records<\/a> (EHRs), <a href=\"https:\/\/www.limswiki.org\/index.php\/Electronic_medical_record\" title=\"Electronic medical record\" class=\"wiki-link\" data-key=\"99a695d2af23397807da0537d29d0be7\">electronic medical records<\/a> (EMRs), <a href=\"https:\/\/www.limswiki.org\/index.php\/Medical_device\" title=\"Medical device\" class=\"wiki-link\" data-key=\"8e821122daa731f0fa8782fae57831fa\">medical devices<\/a>, claims data, patient-generated data, etc. The rapid advancements in hardware and software technologies in recent years have ushered in a new era of data collection and processing, resulting in remarkable progress in the field of healthcare data analytics. In the realm of healthcare organizations, clinical data serve a dual purpose. Firstly, it is utilized for the delivery of healthcare services to patients. Secondly, it is used for secondary purposes such as <a href=\"https:\/\/www.limswiki.org\/index.php\/Research\" title=\"Research\" class=\"wiki-link\" data-key=\"409634fd90113f119362927fe222f549\">research<\/a>, analysis, <a href=\"https:\/\/www.limswiki.org\/index.php\/Quality_(business)\" title=\"Quality (business)\" class=\"wiki-link\" data-key=\"c4ac43430d1c3a3a15d1255257aaea37\">quality<\/a> improvement, and more. In particular, the secondary use of clinical data has emerged as a critical component of healthcare data analytics. This has resulted in a paradigm shift in recent healthcare settings, where the secondary use of healthcare data is deemed just as important as its primary use.\n<\/p><p>EHR systems are leveraged to facilitate the secondary use of healthcare data, for activities such as quality improvement, safety measurement, payments, provider certification, marketing, and research.<sup id=\"rdp-ebb-cite_ref-:0_1-0\" class=\"reference\"><a href=\"#cite_note-:0-1\">[1]<\/a><\/sup> Moreover, the secondary use of healthcare data has the potential to significantly enhance the healthcare experiences of individuals. It can facilitate the learning of diseases and their effective treatments, deepen people\u2019s knowledge and understanding of the effectiveness and efficiency of healthcare systems, and aid in supporting public health initiatives.<sup id=\"rdp-ebb-cite_ref-:0_1-1\" class=\"reference\"><a href=\"#cite_note-:0-1\">[1]<\/a><\/sup> However, the secondary use of healthcare data also raises complex ethical, social, and technical issues; for example, questions regarding data ownership and access privileges continue to challenge the field.<sup id=\"rdp-ebb-cite_ref-2\" class=\"reference\"><a href=\"#cite_note-2\">[2]<\/a><\/sup>\n<\/p><p>The healthcare industry has witnessed a remarkable surge in the volume of healthcare data in recent times, primarily driven by the widespread adoption of EHR systems worldwide.<sup id=\"rdp-ebb-cite_ref-:1_3-0\" class=\"reference\"><a href=\"#cite_note-:1-3\">[3]<\/a><\/sup> In addition, there has been an unprecedented growth in other types of healthcare data, such as genome <a href=\"https:\/\/www.limswiki.org\/index.php\/Sequencing\" class=\"mw-disambig wiki-link\" title=\"Sequencing\" data-key=\"e36167a9eb152ca16a0c4c4e6d13f323\">sequencing<\/a> and other biological structures.<sup id=\"rdp-ebb-cite_ref-:1_3-1\" class=\"reference\"><a href=\"#cite_note-:1-3\">[3]<\/a><\/sup> The analysis of this clinical data is commonly referred to as analytics or healthcare data analytics, which falls under the category of secondary use of clinical data. While the term \"data analytics\" is extensively used in and outside of healthcare<sup id=\"rdp-ebb-cite_ref-:1_3-2\" class=\"reference\"><a href=\"#cite_note-:1-3\">[3]<\/a><\/sup>, our focus in this study is on its application in the healthcare industry.\n<\/p><p>Analytics has been deployed across various domains, including healthcare. However, experts from different fields offer diverse definitions of analytics. Nonetheless, the ultimate objective of analytics, as perceived by all experts, remains consistent. Data analytics experts characterize analytics as \u201cthe comprehensive exploitation of data, statistical and quantitative analysis, explanatory and predictive models, fact-based management to drive decisions, actions, and much more.\u201d<sup id=\"rdp-ebb-cite_ref-4\" class=\"reference\"><a href=\"#cite_note-4\">[4]<\/a><\/sup> Similarly, IBM defines analytics as \u201cthe methodical use of data and associated business insights developed through applied analytical disciplines (e.g., statistical, predictive, contextual, quantitative, cognitive, and other models) to drive evidence-based decision making for planning, management, measurement, and learning. Analytics can be descriptive, predictive, or prescriptive.\u201d<sup id=\"rdp-ebb-cite_ref-5\" class=\"reference\"><a href=\"#cite_note-5\">[5]<\/a><\/sup>\n<\/p><p>Moreover, the two eminent healthcare data analytics experts, Adams and Klein, outline three distinct levels and applications of analytics in the healthcare domain.<sup id=\"rdp-ebb-cite_ref-6\" class=\"reference\"><a href=\"#cite_note-6\">[6]<\/a><\/sup><sup class=\"noprint Inline-Template\"><span style=\"white-space: nowrap;\">[<i><a href=\"https:\/\/en.wikipedia.org\/wiki\/Wikipedia:Link_rot\" class=\"extiw wiki-link\" title=\"wikipedia:Wikipedia:Link rot\" data-key=\"8e73a2ff6f82d88817bdd8ee8b302ab7\"><span title=\" Dead link since 11 August 2023\">dead link<\/span><\/a><\/i>]<\/span><\/sup> Each level is associated with increasing functionality and value:\n<\/p>\n<ol><li>Descriptive: This level refers to standard reporting types that depict current situations and problems.<\/li>\n<li>Predictive: This level refers to simulation and modeling techniques that forecast trends and anticipate the outcomes of implemented actions.<\/li>\n<li>Prescriptive: This level concerns financial, clinical optimization, and other outcomes.<\/li><\/ol>\n<p>All three levels of healthcare data analytics are of paramount importance. However, predictive analytics has gained more attention in the current healthcare landscape<sup id=\"rdp-ebb-cite_ref-:1_3-3\" class=\"reference\"><a href=\"#cite_note-:1-3\">[3]<\/a><\/sup>, as medical experts seek to predict various clinical-related variables in healthcare data to enhance healthcare delivery services and optimize health and financial outcomes.\n<\/p><p>With the advent of digital medical records, <a href=\"https:\/\/www.limswiki.org\/index.php\/Hospital\" title=\"Hospital\" class=\"wiki-link\" data-key=\"b8f070c66d8123fe91063594befebdff\">hospitals<\/a> and other healthcare organizations are accumulating vast amounts of data at an unprecedented rate. The clinical data captured by these organizations take multifarious forms, ranging from structured data (such as <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory\" title=\"Laboratory\" class=\"wiki-link\" data-key=\"c57fc5aac9e4abf31dccae81df664c33\">laboratory<\/a> results and images) to unstructured data (such as textual notes comprising clinical narratives, reports, and various other documents). For example, the well-known US healthcare company Kaiser-Permanente has a current data store for over nine million members that surpasses a staggering 30 petabytes of data.<sup id=\"rdp-ebb-cite_ref-7\" class=\"reference\"><a href=\"#cite_note-7\">[7]<\/a><\/sup> Another notable example is the American Society for Clinical Oncology (ASCO), which is developing its Cancer Learning Intelligence Network for Quality (CancerLinQ).<sup id=\"rdp-ebb-cite_ref-8\" class=\"reference\"><a href=\"#cite_note-8\">[8]<\/a><\/sup> The clinical data accumulated by CancerLinQ serve myriad healthcare data analytics purposes, providing clinicians and researchers with an extensive platform for EHR data collection, <a href=\"https:\/\/www.limswiki.org\/index.php\/Data_mining\" title=\"Data mining\" class=\"wiki-link\" data-key=\"be09d3680fe1608addedf6f62692ee47\">data mining<\/a>, and <a href=\"https:\/\/www.limswiki.org\/index.php\/Data_visualization\" title=\"Data visualization\" class=\"wiki-link\" data-key=\"4a3b86cba74bc7bb7471aa3fc2fcccc3\">visualization<\/a>, as well as the application of <a href=\"https:\/\/www.limswiki.org\/index.php\/Clinical_decision_support_system\" title=\"Clinical decision support system\" class=\"wiki-link\" data-key=\"095141425468d057aa977016869ca37d\">clinical decision support<\/a>, among others.\n<\/p><p>The ultimate goal of healthcare data analytics is to use data to make informed decisions and identify patterns and trends that can help improve patient outcomes, optimize operational efficiency, and reduce costs. By analyzing data, healthcare providers can identify areas for improvement, predict health outcomes, and personalize care for individual patients.\n<\/p><p>Some common applications of healthcare data analytics include population health management, clinical decision support, disease surveillance and monitoring, and quality improvement initiatives. The field of healthcare data analytics is constantly evolving as new technologies and approaches emerge, and it is a critical area of focus for healthcare organizations looking to improve their performance and deliver better care to patients.\n<\/p><p>To summarize, data analytics has become a pivotal aspect of current healthcare settings, a core requirement for both the industry and its experts.<sup id=\"rdp-ebb-cite_ref-:1_3-4\" class=\"reference\"><a href=\"#cite_note-:1-3\">[3]<\/a><\/sup> Moreover, the future of healthcare holds tremendous promise when it comes to data analytics. With the burgeoning volume of clinical and research data, coupled with the methods employed to analyze and put it to use, there is tremendous potential for improving healthcare delivery, personal health, and biomedical research. However, there is also a continuing need to improve the quality of clinical data and conduct research aimed at demonstrating how best to apply data analytics to address healthcare challenges.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Healthcare_data_analytics_using_the_FHIR_data_standard\">Healthcare data analytics using the FHIR data standard<\/span><\/h3>\n<p>FHIR is the latest healthcare data standard that is gaining popularity in the healthcare sector.<sup id=\"rdp-ebb-cite_ref-9\" class=\"reference\"><a href=\"#cite_note-9\">[9]<\/a><\/sup> FHIR provides a standardized way to represent and exchange healthcare information electronically.<sup id=\"rdp-ebb-cite_ref-10\" class=\"reference\"><a href=\"#cite_note-10\">[10]<\/a><\/sup> This avant-garde standard has captured the imagination of healthcare providers due to its unparalleled ability to reduce the costs of interoperability and its potential to catalyze a new ecosystem of third-party applications.<sup id=\"rdp-ebb-cite_ref-11\" class=\"reference\"><a href=\"#cite_note-11\">[11]<\/a><\/sup> FHIR\u2019s revolutionary interoperability capabilities have surpassed the antiquated data standards of yore, such as <a href=\"https:\/\/www.limswiki.org\/index.php\/Health_Level_7\" title=\"Health Level 7\" class=\"wiki-link\" data-key=\"e0bf845fb58d2bae05a846b47629e86f\">Health Level 7<\/a> (HL7; v2, v3, CDA).\n<\/p><p>In a recent survey conducted by Australian and New Zealand healthcare executives, the adoption of FHIR was found to increase interoperability from a measly 11% to a staggering 66%.<sup id=\"rdp-ebb-cite_ref-:2_12-0\" class=\"reference\"><a href=\"#cite_note-:2-12\">[12]<\/a><\/sup> Consequently, its adaptable nature for data exchange is increasing at a rapid pace within the healthcare industry as it garners favor among stakeholders for data exchange. The survey further revealed that 55% of healthcare providers are willing to make the shift to a FHIR-based interoperability platform. Additionally, it is estimated that FHIR will be widespread in the world healthcare industry by 2024.<sup id=\"rdp-ebb-cite_ref-13\" class=\"reference\"><a href=\"#cite_note-13\">[13]<\/a><\/sup> This showed the popularity of FHIR-based interoperability in the healthcare industry and healthcare providers\u2019 interest in its adaptability.\n<\/p><p>However, the healthcare industry\u2019s needs go beyond mere clinical data exchange. Clinical data need to be processed for other purposes, such as data analysis, data analytics, research, and so forth. Thus, the clinical data represented in the FHIR standard need to fulfill these requirements. FHIR\u2019s adoption is expected to increase data availability for analytics and solve the data exchange and analytics problems faced by the healthcare industry.<sup id=\"rdp-ebb-cite_ref-:2_12-1\" class=\"reference\"><a href=\"#cite_note-:2-12\">[12]<\/a><\/sup> Nevertheless, the adoption of FHIR in the analytics domain remains relatively low, as the standard is still young.<sup id=\"rdp-ebb-cite_ref-:3_14-0\" class=\"reference\"><a href=\"#cite_note-:3-14\">[14]<\/a><\/sup> Moreover, the tools supporting FHIR data analytics are still relatively immature.<sup id=\"rdp-ebb-cite_ref-:4_15-0\" class=\"reference\"><a href=\"#cite_note-:4-15\">[15]<\/a><\/sup> However, the healthcare providers argue that they are not only interested in sharing clinical data across healthcare organizations to improve data interoperability but are more excited to process clinical data for other purposes, such as data analysis and research, to provide real-time medical services to patients. Therefore, the tools provided these services are essential in the healthcare industry.\n<\/p><p>On the flip side, the cutting-edge FHIR standard for patient clinical information presents plenty of new opportunities for visualizing, analyzing, and automating various types of healthcare data. With each passing day, fresh use cases for FHIR data analytics are building in the healthcare industry, such as real-time alerts for patient satisfaction, identifying patterns in patients\u2019 medical records across datasets, real-time visibility into patient readmission rates, cost savings while upholding top-notch care quality, and countless more.<sup id=\"rdp-ebb-cite_ref-16\" class=\"reference\"><a href=\"#cite_note-16\">[16]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-17\" class=\"reference\"><a href=\"#cite_note-17\">[17]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-18\" class=\"reference\"><a href=\"#cite_note-18\">[18]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-19\" class=\"reference\"><a href=\"#cite_note-19\">[19]<\/a><\/sup> However, analyzing and implementing these use cases can prove challenging owing to the young stage and practicality of FHIR.\n<\/p><p>To facilitate data processing and exchange, FHIR employs REST APIs. Nonetheless, for the domain of FHIR data analytics, the FHIR APIs must possess a dynamic nature regarding data queries and processing. As data analytics are based on diverse types of data housed in varied FHIR resources, the FHIR APIs must query this data in various ways to enable effective data analysis. Additionally, FHIR has accelerated the swift delivery of a massive volume of new healthcare applications that can integrate with EHR or EMR data via the FHIR APIs. However, most of these applications are limited to perusing data relevant to a single patient.<sup id=\"rdp-ebb-cite_ref-:3_14-1\" class=\"reference\"><a href=\"#cite_note-:3-14\">[14]<\/a><\/sup> One contributing factor, among many others, could be that the FHIR APIs are not optimally suited to queries that aggregate and categorize data across a vast clinical dataset.\n<\/p><p>A related and parallel trend within the realm of <a href=\"https:\/\/www.limswiki.org\/index.php\/Health_information_technology\" title=\"Health information technology\" class=\"wiki-link\" data-key=\"9c8ef822470559f757db89f3fa234cc0\">health information systems<\/a> involves investing in higher-quality structured data via the coding of clinical records at the point of care. With the implementation of EMRs, healthcare providers are now able to incorporate a multitude of concepts into medical records using advanced terminologies, including <a href=\"https:\/\/www.limswiki.org\/index.php\/International_Statistical_Classification_of_Diseases_and_Related_Health_Problems#ICD-10\" title=\"International Statistical Classification of Diseases and Related Health Problems\" class=\"wiki-link\" data-key=\"eb4d523c235de789cef9c5ddadca615e\">ICD-10<\/a>, <a href=\"https:\/\/www.limswiki.org\/index.php\/LOINC\" title=\"LOINC\" class=\"wiki-link\" data-key=\"b20c83dbfab36194bab6c223e31ebfdc\">LOINC<\/a>, and <a href=\"https:\/\/www.limswiki.org\/index.php\/SNOMED_CT\" title=\"SNOMED CT\" class=\"wiki-link\" data-key=\"a04000c818ae954bf52cfea5efdf020d\">SNOMED CT<\/a>.<sup id=\"rdp-ebb-cite_ref-:5_20-0\" class=\"reference\"><a href=\"#cite_note-:5-20\">[20]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-21\" class=\"reference\"><a href=\"#cite_note-21\">[21]<\/a><\/sup> This affords the opportunity for more detailed analysis by enabling access to specific clinical concepts as well as the ability to query the ontology based on additional attributes and relationships to other clinical concepts.\n<\/p><p>While this technique is highly effective when analyzing clinical data based on specific codes or terminologies, it proves to be less fruitful in general concept analysis. Therefore, other scenarios, including modifications to FHIR APIs, must be considered to enable various ways of analyzing medical data for deep clinical data analysis. However, this technique is extremely challenging and requires an individual with extensive skill and experience to change the core implementation mechanisms of FHIR APIs.\n<\/p><p>Currently, the level of expertise required to make the best use of FHIR and other clinical terminology within a data analysis workflow is relatively rare in the healthcare domain.<sup id=\"rdp-ebb-cite_ref-22\" class=\"reference\"><a href=\"#cite_note-22\">[22]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-23\" class=\"reference\"><a href=\"#cite_note-23\">[23]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-24\" class=\"reference\"><a href=\"#cite_note-24\">[24]<\/a><\/sup> The applications of data analytics and analysis in healthcare settings using the FHIR data standard are also a relatively new concept and have scarcely been applied. However, due to the rapid adoption of FHIR for medical data exchange, data analytics and analysis are now a core demand of the healthcare industry to process patient medical data in various ways and provide real-time medication to improve healthcare delivery. In summary, the standardization of healthcare data plays a crucial role in clinical and translational data analysis systems, especially when large-scale data are involved. Moreover, healthcare applications for clinical statistics and analysis can significantly enhance healthcare by connecting clinical data with analytic tools, thereby engaging practitioners or clinicians in the process of medical data analysis.<sup id=\"rdp-ebb-cite_ref-:6_25-0\" class=\"reference\"><a href=\"#cite_note-:6-25\">[25]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-26\" class=\"reference\"><a href=\"#cite_note-26\">[26]<\/a><\/sup>\n<\/p><p>In response to the pressing need to address the complex and multifaceted challenges of data analytics in the healthcare industry, this research study puts forth a cutting-edge and innovative FHIR standard-based data analytics framework. This platform is designed to tackle the healthcare industry\u2019s data analytics issues and provide them with a scalable, standards-based data model. At present, this pioneering framework is tailored to work with <a href=\"https:\/\/www.limswiki.org\/index.php\/Workflow\" title=\"Workflow\" class=\"wiki-link\" data-key=\"92bd8748272e20d891008dcb8243e8a8\">workflows<\/a> specifically designed for patient clinical data originating from two distinct <a href=\"https:\/\/www.limswiki.org\/index.php\/Hospital_information_system\" title=\"Hospital information system\" class=\"wiki-link\" data-key=\"d8385de7b1f39a39d793f8ce349b448d\">hospital information systems<\/a>: patient registration systems (PRSs) and <a href=\"https:\/\/www.limswiki.org\/index.php\/Laboratory_information_system\" title=\"Laboratory information system\" class=\"wiki-link\" data-key=\"37add65b4d1c678b382a7d4817a9cf64\">laboratory information systems<\/a> (LISs). Other possible data analysis workflows and customized research scenarios on the patient data from other HISs could be performed on FHIR-based data but are not currently directly supported by our framework without any modification.\n<\/p><p>The developed framework utilizes a FHIR database as its dataset, with FHIR RESTful APIs that query different types of FHIR resources from the database algorithmically. The mapping algorithm and analytic engine then process the retrieved data and generate various data analytics from patient clinical data, presenting the results to end-users via a user-friendly interface.\n<\/p><p>In short, this research study provides a state-of-the-art solution for healthcare data analytics, offering healthcare professionals an innovative platform to conduct data analysis on clinical data using FHIR. With the FHIR Data Analytics Framework, healthcare professionals can now extract meaningful insights from patient data and leverage these insights to enhance patient care delivery, promote better health outcomes, and drive healthcare industry advancements forward.\n<\/p><p>This research work has three main contributions: First, the entire framework and workflow design follow the FHIR data standard, which could be reused for any other clinical data domains and could provide support for any clinical data that follow the FHIR standard. Second, the data analysis workflow and tools incorporate the experience of clinical researchers and statisticians, which could provide a starting point for FHIR researchers in this cutting-edge standard. Third, the intelligent mapping algorithm is artfully designed to facilitate the sublime process of data analytics or data analysis within the realm of FHIR-based data. The mapping algorithm could be reused for any other clinical data that follow the FHIR specification and need to process the FHIR-based data for other purposes, such as research, developing an <a href=\"https:\/\/www.limswiki.org\/index.php\/Artificial_intelligence\" title=\"Artificial intelligence\" class=\"wiki-link\" data-key=\"0c45a597361ca47e1cd8112af676276e\">artificial intelligence<\/a> (AI) model or <a href=\"https:\/\/www.limswiki.org\/index.php\/Machine_learning\" title=\"Machine learning\" class=\"wiki-link\" data-key=\"79aab39cfa124c958cd1dbcab3dde122\">machine learning<\/a> (ML) model, etc.\n<\/p><p>The FHIR Data Analytics Framework comprises six layers: the FHIR database, the FHIR query engine layer, the mapping algorithm\/agent layer, the FHIR-compliant database layer, the analytics engine layer, and the user interface. The rest of this manuscript is structured accordingly. The next section provides a comprehensive literature review, followed by a discussion of the five major materials used in this study. Then, the framework\u2019s architecture is described in detail, followed by the implementation details, an explanation of the experiment setup, and the results. We close by describing the limitations of this approach, as well as a discussion, future plans, and finally a conclusion.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Literature_review\">Literature review<\/span><\/h2>\n<p>Throughout the years, financial and administrative data were deemed essential attributes for planning purposes. However, in recent times, comprehensive healthcare data have become crucial to institutional strategic planning and self-analysis.<sup id=\"rdp-ebb-cite_ref-27\" class=\"reference\"><a href=\"#cite_note-27\">[27]<\/a><\/sup> The healthcare industry heavily relies on various data sources, such as EHR analysis (EHRA), biomedical image analysis (BIA), sensor data analysis (SDA), biomedical signal analysis (BSA), genomic data analysis (GDA), clinical text mining (CTM), and other analytics methods to process and analyze clinical data.<sup id=\"rdp-ebb-cite_ref-28\" class=\"reference\"><a href=\"#cite_note-28\">[28]<\/a><\/sup> Analyzing and performing data analytics on clinical data in the healthcare settings is a fundamental requirement in the healthcare industry. Despite this, the literature scarcely acknowledges the use of data analytics in the healthcare industry.\n<\/p><p>In our thorough literature review, we noticed some efforts that utilized various clinical data sources in the data analytics domain. For example, the Observational Health Data Sciences and Informatics (OHDSI) program has generated an enormous volume of work in the field of health data analytics, including the creation of the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM).<sup id=\"rdp-ebb-cite_ref-29\" class=\"reference\"><a href=\"#cite_note-29\">[29]<\/a><\/sup> The OMOP provides a target data model for health data analytics, along with analytic routines and common vocabularies that could be run over the common data model.<sup id=\"rdp-ebb-cite_ref-30\" class=\"reference\"><a href=\"#cite_note-30\">[30]<\/a><\/sup>\n<\/p><p>Furthermore, the OMOP has a rich ecosystem of applications that have been developed to assist in its implementation and use, such as the ATLAS user interface designed by the OHDSI community<sup id=\"rdp-ebb-cite_ref-31\" class=\"reference\"><a href=\"#cite_note-31\">[31]<\/a><\/sup> to facilitate analytic queries over the OMOP data model. Moreover, researchers have also explored the use of the <a href=\"https:\/\/www.limswiki.org\/index.php\/OpenEHR\" title=\"OpenEHR\" class=\"wiki-link\" data-key=\"4c3f67f4e3102639ce8c6bff842b8982\">openEHR<\/a> model within health data analytics, as exemplified by the work of Chunlan <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-32\" class=\"reference\"><a href=\"#cite_note-32\">[32]<\/a><\/sup> in developing the Archetype Query Language (AQL), which is a standard way of querying data from openEHR-based systems.<sup id=\"rdp-ebb-cite_ref-33\" class=\"reference\"><a href=\"#cite_note-33\">[33]<\/a><\/sup> The AQL has been implemented in many EHRs and analytics software tools and provides important design features for this type of capability.<sup id=\"rdp-ebb-cite_ref-:3_14-2\" class=\"reference\"><a href=\"#cite_note-:3-14\">[14]<\/a><\/sup>\n<\/p><p>However, while these attempts have been applied to EHR datasets, the application of such techniques to data represented in the FHIR standard is a relatively new and challenging concept. Therefore, the researchers are looking for new techniques with which they can apply data analytics to the clinical data represented in the FHIR-based standard. However, as aforementioned, FHIR is a young data standard<sup id=\"rdp-ebb-cite_ref-:3_14-3\" class=\"reference\"><a href=\"#cite_note-:3-14\">[14]<\/a><\/sup>, and limited research related to FHIR analytics has been reported.<sup id=\"rdp-ebb-cite_ref-34\" class=\"reference\"><a href=\"#cite_note-34\">[34]<\/a><\/sup> A recent scientific literature review study reveals that only a few studies have been reported in the literature that discussed FHIR analytics.<sup id=\"rdp-ebb-cite_ref-:4_15-1\" class=\"reference\"><a href=\"#cite_note-:4-15\">[15]<\/a><\/sup> Thus, the concept of FHIR data analytics is extremely new, and so far, the state of FHIR analytics is at an early stage. Therefore, applying data analytics or data analysis is challenging and an extremely new concept in this domain. However, some researchers have made some initial efforts in FHIR-based analytical circumstances, such as the prediction of sepsis based on the FHIR standard by Lakshman <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-35\" class=\"reference\"><a href=\"#cite_note-35\">[35]<\/a><\/sup> and the deployment of clinical predictive models via FHIR in Web Services explained by Khalilia <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-36\" class=\"reference\"><a href=\"#cite_note-36\">[36]<\/a><\/sup>\n<\/p><p>Furthermore, the use of FHIR to store and analyze medical data on a large scale has also been implemented and integrated into the Google Cloud and Microsoft Azure <a href=\"https:\/\/www.limswiki.org\/index.php\/Cloud_computing\" title=\"Cloud computing\" class=\"wiki-link\" data-key=\"fcfe5882eaa018d920cedb88398b604f\">cloud<\/a> platforms.<sup id=\"rdp-ebb-cite_ref-:5_20-1\" class=\"reference\"><a href=\"#cite_note-:5-20\">[20]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:6_25-1\" class=\"reference\"><a href=\"#cite_note-:6-25\">[25]<\/a><\/sup> In addition, the tech company Startups has recognized the analytical capabilities of FHIR and utilized the doc.ai application to provide personalized medicine, automate the process of controlling audit files, and store data in a structured way.<sup id=\"rdp-ebb-cite_ref-37\" class=\"reference\"><a href=\"#cite_note-37\">[37]<\/a><\/sup>\n<\/p><p>Moreover, FHIR was used to support clinical decisions and to build a distributed phenotyping analytics platform.<sup id=\"rdp-ebb-cite_ref-38\" class=\"reference\"><a href=\"#cite_note-38\">[38]<\/a><\/sup> Kreuzthaler <i>et al.<\/i> discussed the use and benefits of standardized data in analytical approaches.<sup id=\"rdp-ebb-cite_ref-39\" class=\"reference\"><a href=\"#cite_note-39\">[39]<\/a><\/sup> In addition, Franz <i>et al.<\/i> developed a monitoring system with the FHIR data standard.<sup id=\"rdp-ebb-cite_ref-40\" class=\"reference\"><a href=\"#cite_note-40\">[40]<\/a><\/sup> Liu <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-41\" class=\"reference\"><a href=\"#cite_note-41\">[41]<\/a><\/sup> explained many ways to make bulk FHIR data available for analytic queries. The authors concluded that Apache Parquet<sup id=\"rdp-ebb-cite_ref-42\" class=\"reference\"><a href=\"#cite_note-42\">[42]<\/a><\/sup> is the ideal tool for storing and querying FHIR data in the context of large-scale analytics using Apache Spark.\n<\/p><p>Grimes <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-:3_14-4\" class=\"reference\"><a href=\"#cite_note-:3-14\">[14]<\/a><\/sup> discussed the use of FHIR data analytics using the pathling concept. However, it works in a limited domain because some operations are not easily or even currently possible to achieve via the FHIR REST API specification, such as data aggregation, searching the data, etc. Therefore, it is extremely challenging to implement. Furthermore, Dunn <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-43\" class=\"reference\"><a href=\"#cite_note-43\">[43]<\/a><\/sup> explained genomic data analysis using FHIR in a cloud framework. However, it only applies to the analysis of genomic data using a cloud framework and would be challenging to apply to clinical data represented in FHIR and implement in traditional healthcare settings. Similarly, Gruendner <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-44\" class=\"reference\"><a href=\"#cite_note-44\">[44]<\/a><\/sup> described the FHIR data formatting for statistical analysis. However, this technique only generated the FHIR data but failed to provide any platform for clinical data analysis or data analytics using REST APIs. Therefore, it is extremely challenging to generalize the concept and provide a platform for medical software developers and researchers to perform any data analytics on the clinical data or use the resulting data for research purposes.\n<\/p><p>Moreover, the notable <a href=\"https:\/\/www.limswiki.org\/index.php\/Health_information_technology\" title=\"Health information technology\" class=\"wiki-link\" data-key=\"9c8ef822470559f757db89f3fa234cc0\">health information technology<\/a> (HIT) services provider Cerner Corporation produces the Bunsen<sup id=\"rdp-ebb-cite_ref-45\" class=\"reference\"><a href=\"#cite_note-45\">[45]<\/a><\/sup> library that encodes FHIR resources within Apache Spark<sup id=\"rdp-ebb-cite_ref-46\" class=\"reference\"><a href=\"#cite_note-46\">[46]<\/a><\/sup> datasets. This work facilitates loading, transforming, and analyzing FHIR data. Cerner Corporation has also been involved with implementation of Structured Query Language (SQL) on the FHIR proposal<sup id=\"rdp-ebb-cite_ref-47\" class=\"reference\"><a href=\"#cite_note-47\">[47]<\/a><\/sup>, which is a projection of the FHIR data model onto the relational query model and SQL language. Additionally, Google also discussed and implemented a method for encoding FHIR data using the Buffers Protocol.<sup id=\"rdp-ebb-cite_ref-48\" class=\"reference\"><a href=\"#cite_note-48\">[48]<\/a><\/sup> Furthermore, Google also developed many tools and techniques<sup id=\"rdp-ebb-cite_ref-49\" class=\"reference\"><a href=\"#cite_note-49\">[49]<\/a><\/sup> for using FHIR with the BigQuery analytics platform, integrating with the FHIR Bulk Data API, and using FHIR data within cloud-based data processing and machine learning pipelines.\n<\/p><p>Despite the various initial attempts at data analytics on clinical data represented in the FHIR data standard, there has been no user-friendly data analytics framework or visualized tool to help healthcare users such as practitioners, providers, and patients perform various data analytics on patient clinical data. To address this gap, our research study developed a framework with a user-friendly interface that enables healthcare practitioners, providers, and patients to perform data analytics on the clinical data used in two HISs and represented in the FHIR-based standard.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Materials\">Materials<\/span><\/h2>\n<p>In this section, we are discussing various materials that will help us develop our framework. This <a href=\"https:\/\/www.limswiki.org\/index.php\/Information\" title=\"Information\" class=\"wiki-link\" data-key=\"6300a14d9c2776dcca0999b5ed940e7d\">information<\/a> is helpful for the readers to know about the challenges and framework pre-development procedures involved in this undertaking.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Required_outcomes\">Required outcomes<\/span><\/h3>\n<p>Our ambition was to develop a data analytics framework that could perform various types of analytics on clinical data typically used in healthcare facilities and represented in the FHIR standard. Our esteemed endeavor has borne fruit, and we have proudly fashioned a data analytics framework for healthcare environments, performing an array of analytical procedures on clinical data and elegantly visualizing the resulting insights.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"User_research_and_inputs\">User research and inputs<\/span><\/h3>\n<p>As previously discussed, support for FHIR and related data analytics is still in its infancy. Moreover, the clinical data flow in healthcare settings and the data analytics concept on clinical data represented in the FHIR format remain unclear at this stage. Particularly for individuals outside of the medical field, comprehending this concept can prove to be a challenging task. Therefore, to better grasp the FHIR data analytics concept and its workflow in the healthcare environment, we decided to take input from various professionals working in healthcare settings. We conducted numerous interviews with doctors, practitioners, patients, pharmacists, and others in the healthcare industry to obtain a more comprehensive understanding of workflow and user requirements. These interviews consisted of both open-ended and closed questions related to the current challenges within the healthcare data analytics domain. Furthermore, we sought to understand the data analytics needs of various stakeholders, including patients, practitioners, and healthcare providers, regarding the healthcare industry.\n<\/p><p>This process helped us validate our assumptions about adopting FHIR data analytics in the healthcare industry and provided insight into the views of users (practitioners, patients, providers, etc.) regarding the adoption of FHIR data analytics, as well as their opinions on workflow with this new technique in this domain. It also identified a range of use case scenarios for various analytics that we could implement in this prototype. Based on what we learned from this process, we selected the following two parameters (use cases) to serve as the focal point of our work:\n<\/p>\n<ul><li><b>Patient cohort selection<\/b>: This involves the selection and retrieval of patient information\/records based on complex inclusion and exclusion criteria.<\/li>\n<li><b>Data preparation<\/b>: This includes processing and reshaping data in preparation for use with statistical models or tools.<\/li><\/ul>\n<h3><span class=\"mw-headline\" id=\"Challenges\">Challenges<\/span><\/h3>\n<p>FHIR has a highly nested, complex, and graph-like data format that represents clinical data in a resource structure in JSON\/XML format. With a hierarchical tree structure, the data elements are nested, making it difficult to represent within traditional relational data models, especially when simplifying query logic is a primary goal. Representing the data in a traditional relational data model is essential for data analytics and analysis. However, the graphical FHIR resource structure poses a significant challenge, and optimizing the data structure for analytics and analysis queries across a wide range of use cases also raises performance issues.\n<\/p><p>To handle these challenges, we have developed a cutting-edge mapping algorithm\/agent to convert FHIR resource data into a sample EMR format and store it in a relational data model before conducting any data analytics. Our mapping algorithm was used to transform the clinical data stored in FHIR resources into a relational data model.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"The_clinical_data_analysis_workflow_design\">The clinical data analysis workflow design<\/span><\/h3>\n<p>Performing data analytics on the data present in the dataset is challenging, as it relies totally on workflows (business use case scenarios). The design of such workflows is quite difficult, particularly for non-medical experts, because they have issues identifying various parameters for the clinical data used in the healthcare settings, which include user requirements, data constraints, and more. Therefore, we had discussions with medical experts and, on the basis of their inputs and our common clinical data analysis requirements, we designed two general analysis workflows: patient-centered data analysis and cohort-based data analysis. Furthermore, we elaborated on the workflows and designed five primary workflows that are used in healthcare settings on the patient data and are suitable for performing data analytics on our dataset (see Table 1).\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"2\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 1.<\/b> List of data analysis workflows (business use cases) for patient data in the healthcare setting.\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Number\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Description\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Investigate registered patients in healthcare settings\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Investigate registered patients in healthcare settings within a specified timeframe\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Investigate patients having various types of allergies\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">4\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Investigate various types of tests ordered by a physician, organization, etc.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">5\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Investigate various types of tests ordered by a physician, organization, etc., within a specified timeframe\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>The patient-centered data analysis workflow facilitates the browsing of various pieces of information focused on the individual. Patient-specific data derived from multiple sources are integrated into a single identifier. In the FHIR data model, the patient is an independent resource, while other resources such as observation and practitioner have a property \u201csubject\u201d that links them to a specific patient object, representing patient-centered relationships.\n<\/p><p>The cohort-based analysis workflow refers to more common data analysis needs in clinical statistics and studies. In this workflow, the Condition\/AllergyIntolerance\/Observation\/Practitioner of a cohort is largely measured by the distribution of patient characteristics in different dimensions. The workflow is designed to support a wide range of clinical data analysis tasks, including patient registration analysis, patient allergy timeline analysis, patient laboratory test analysis, cohort gender\/age distribution statistics, and more. Overall, our workflows provide a robust framework for performing data analytics on healthcare datasets.\n<\/p>\n<h3><span id=\"rdp-ebb-FHIR_REST_API's_working_mechanism\"><\/span><span class=\"mw-headline\" id=\"FHIR_REST_API.27s_working_mechanism\">FHIR REST API's working mechanism<\/span><\/h3>\n<p>The FHIR specification defines standard REST APIs to exchange a variety of healthcare data and perform a range of operations on the clinical data represented in the FHIR resource structure. These APIs are also known as the core FHIR REST APIs. The power of these APIs lies in their use of the widely accepted HTTP (GET, POST, PUT, DELETE) protocol to perform pre-defined operations such as CRUD (Create, Read, Update, Delete) on any FHIR resource. For example, with just a few clicks, one can access and retrieve the update history, view information, delete, create, or update any instance of a FHIR resource. Figure 1 illustrates the view of these operations.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig1_Ayaz_Healthcare23_11-12.png\" class=\"image wiki-link\" data-key=\"015bfdc53ca0f083ca2c5b95974dc1b0\"><img alt=\"Fig1 Ayaz Healthcare23 11-12.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/0\/04\/Fig1_Ayaz_Healthcare23_11-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 1.<\/b> CRUD operations of the patient resource in the FHIR server.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>Furthermore, every FHIR API conforms to a common signature and format, ensuring that FHIR-compliant systems can retrieve specific healthcare data using the same API signature and format. For example, to retrieve patient demographic information based on the patient\u2019s name and date of birth, one can use the following API:\n<\/p><p><tt>GET <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/baseURL\/Patient?given=\" target=\"_blank\">http:\/\/baseURL\/Patient?given=<\/a>[patient given name]&birthDate=[date of birth]<\/tt>\n<\/p><p>This API will retrieve the patient\u2019s name and date of birth. In this API, \u201cPatient\u201d is the FHIR patient resource, while \u201cgiven\u201d and \u201cbirthDate\u201d are the given parameters. The output of this API will be in standard JSON\/XML format, with tags and elements following strict standards. Figure 2 provides an illustrative view of a sample API operation mechanism in which the APIs access the clinical data, and we performed data analytics on that data.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig2_Ayaz_Healthcare23_11-12.png\" class=\"image wiki-link\" data-key=\"eee759b276ed55e2e3447c0f536aac93\"><img alt=\"Fig2 Ayaz Healthcare23 11-12.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/1\/1f\/Fig2_Ayaz_Healthcare23_11-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 2.<\/b> REST APIs Operations: The operations performed on clinical data represented in FHIR resources.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h2><span class=\"mw-headline\" id=\"FHIR_data_analytics_framework\">FHIR data analytics framework<\/span><\/h2>\n<p>We developed a FHIR data analytics framework used to perform various data analytics on the clinical data represented in the FHIR resource structure. In our use case scenario, the FHIR resources are stored in the Mango database that we developed in our previous porotype. We developed various APIs on top of this database to retrieve the data stored in the FHIR resources format and perform data analytics. Figure 3 explains various sections of this framework and their connections. This framework has the following six major parts:\n<\/p>\n<ol><li>FHIR database<\/li>\n<li>FHIR query engine<\/li>\n<li>Mapping algorithm<\/li>\n<li>FHIR-compliant database (relational database model)<\/li>\n<li>Analytic engine<\/li>\n<li>User interface<\/li><\/ol>\n<p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig3_Ayaz_Healthcare23_11-12.png\" class=\"image wiki-link\" data-key=\"9614f02a60dc38dda620411e9a4f8fd4\"><img alt=\"Fig3 Ayaz Healthcare23 11-12.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/a\/ab\/Fig3_Ayaz_Healthcare23_11-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 3.<\/b> Block diagram of the proposed data analytic framework: Explains various sections of the framework and their connections.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h3><span class=\"mw-headline\" id=\"FHIR_database\">FHIR database<\/span><\/h3>\n<p>The FHIR database is the collection of FHIR resources that we already developed in our previous prototype and would be used as a dataset. Therefore, we are not discussing the creation of this database in this study. Within our database, we have different types of resources, each comprising a grand total of 100 individual resources, but we utilized only those resources that are used for our data analysis.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"FHIR_query_engine_layer\">FHIR query engine layer<\/span><\/h3>\n<p>The FHIR query engine is a collection of FHIR queries, executing only FHIR queries based on core FHIR CRUD operations. Our query engine is responsible for accessing a list of available FHIR resources from the FHIR databases and preparing them for further processing. For this purpose, it uses the core FHIR RESTful APIs. Therefore, our query engine adeptly employs these RESTful APIs to extract and gather all FHIR resources in bulk out of the FHIR database and do some processing, filtering, and transformation within client-side code (in our case, the query engine). We leveraged the core FHIR GET and search APIs to access all resources from the FHIR database. The resulting data (FHIR resources) are assumed to be available in JSON format, the standard format for bulk FHIR data interchange. Table 2 shows the resulting data that have been retrieved from the FHIR database using REST APIs. For this purpose, we used an algorithm (see Algorithm 1) to access all FHIR resources housed within the database. Each type of resource has its own unique title and access parameters; therefore, for different FHIR resources, we used different resource names and search and access parameters within the resource URL to access each resource type. Figure 4 shows the block diagram of the query engine.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"3\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 2.<\/b> FHIR resources retrieved from FHIR database using APIs.\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Number\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Resource type\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Total resources\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Patient\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">AllergyIntolerance\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Practitioner\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">4\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">ServiceRequest\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">5\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">DiagnosticReport\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">6\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Condition\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">7\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Appointment\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"1\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Algorithm 1.<\/b> Algorithm to retrieve resources from FHIR database.\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">\n<p><code>1. <b>Function<\/b> Retrive_Resources()<br \/>\n2. define resource type, e.g., patient<br \/>\n3. define search parameters, e.g., resource id or any other attribute(s)<br \/>\n4. value = Read resource id<br \/>\n5.\u2003\u2003 <b>while<\/b> (resources are available) <b>do<\/b><br \/>\n6. \u2003\u2003 <b>GET<\/b> [base-url]\/RsourceName?id = value<br \/>\n7.\u2003\u2003 <b>end while<\/b><br \/>\n8. <b>end function<\/b> ** Retrive_Resources function **<\/code>\n<\/p>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig4_Ayaz_Healthcare23_11-12.png\" class=\"image wiki-link\" data-key=\"e8a5a67e04052c146901da74a805921e\"><img alt=\"Fig4 Ayaz Healthcare23 11-12.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/f\/f9\/Fig4_Ayaz_Healthcare23_11-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 4.<\/b> Query engine working mechanism: Query engine read FHIR resources in bulk.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>The first feature provided by REST API is the <tt>read (GET)<\/tt> operation. This operation provided a way to access data and prepared it for further operations via various sub-operations. This standard FHIR API reads the FHIR resources from the databases or servers and transfers to the clients in the form of JSON.\n<\/p><p>The <tt>GET<\/tt> operation is designed to accept data extracted from the database or server via FHIR APIs operations. One of the primary functions of the <tt>GET<\/tt> request as a data request is a method to provide the data to the client. During the <tt>GET<\/tt> request operation, the clients (we) must provide the server or database with URLs indicating which data (data from resources) we seek to retrieve.\n<\/p><p>These URLs also enable us to receive updates on the operation\u2019s progress and valuable information about retrieving the final results.\n<\/p><p>The retrieved data are made available to the client in a JSON format (in our case, the query engine). Figure 5 demonstrates that the query engine part reads the FHIR resources from the database using these APIs. \n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig5_Ayaz_Healthcare23_11-12.png\" class=\"image wiki-link\" data-key=\"94fe9f3e1643605a56620daf8533cf3b\"><img alt=\"Fig5 Ayaz Healthcare23 11-12.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/1\/1f\/Fig5_Ayaz_Healthcare23_11-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 5.<\/b> The <tt>GET<\/tt> APIs to extract resources from FHIR database.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>Here is an example of the API query:\n<\/p><p><tt><b>GET<\/b> [base-url]\/resource-type? parameters<\/tt>\n<\/p><p>For example, we could obtain data from a patient resource with identifier 23 using this query:\n<\/p><p><tt><b>GET<\/b> [base-url]\/patient? identifier = 23<\/tt>\n<\/p>\n<h3><span id=\"rdp-ebb-Mapping_agent\/algorithm\"><\/span><span class=\"mw-headline\" id=\"Mapping_agent.2Falgorithm\">Mapping agent\/algorithm<\/span><\/h3>\n<h4><span class=\"mw-headline\" id=\"Need_of_mapping_algorithm\">Need of mapping algorithm<\/span><\/h4>\n<p>The FHIR REST APIs are currently in their nascent stage, offering limited functionalities and operations that can be leveraged for healthcare data analytics applications. The FHIR REST APIs can only perform the core CRUD (Create, Read, Update, and Delete) operations, alongside a handful of other basic functionalities, on data stored in the FHIR resources. These operations are executed using standard mechanisms provided by FHIR, and the REST APIs are happy to execute these operations on various FHIR resources while exchanging data between the FHIR server and the client.\n<\/p><p>However, the healthcare landscape is rapidly evolving, and there is an increasing demand for more advanced and complex operations on patient data in the healthcare environment, particularly in the FHIR data analytics domain. Additionally, healthcare analytics applications need to be improved to reduce the data processing burden and enhance the quality of data analyses.<sup id=\"rdp-ebb-cite_ref-:3_14-5\" class=\"reference\"><a href=\"#cite_note-:3-14\">[14]<\/a><\/sup> Consequently, the REST APIs must evolve to prepare themselves for these challenges by incorporating more complex functionalities and executing more complicated queries. For this purpose, FHIR offers standard mechanisms for extending API functionality, such as extension operations and search profiles.\n<\/p><p>Certain types of operations, including data transformation, aggregations, search operations, and many more, are unfortunately unachievable or impossible using the core FHIR APIs specification.<sup id=\"rdp-ebb-cite_ref-:3_14-6\" class=\"reference\"><a href=\"#cite_note-:3-14\">[14]<\/a><\/sup> This limitation implies that executing more complex queries to perform advanced operations, such as any data analytics or analysis operations on clinical data stored in FHIR resources, is quite challenging and limited at this stage of REST. In other words, the core FHIR APIs encounter difficulties while performing data analytics directly on the patient data stored in the FHIR resources structure in the FHIR server or database. However, the use of data analytics in healthcare information systems is essential in the modern healthcare environment. As a result, we leveraged the FHIR core API functionalities and implemented a specialized intermediate layer known as a mapping algorithm\/agent to simplify data analytics operations on the data stored in the FHIR resources.\n<\/p>\n<h4><span class=\"mw-headline\" id=\"Role_of_mapping_algorithm\">Role of mapping algorithm<\/span><\/h4>\n<p>The FHIR APIs present us with a wealth of resources, returned in the JSON format, which is a complex, hierarchical structure that nests data elements within tags. However, this structure is unsuitable for data analytics operations, which typically require structured or unstructured data, not data in a hierarchal structure.<sup id=\"rdp-ebb-cite_ref-50\" class=\"reference\"><a href=\"#cite_note-50\">[50]<\/a><\/sup> Therefore, we must preprocess the JSON data by converting it into a tabular format and storing it in a FHIR-compliant relational database before applying any analytics.\n<\/p><p>For this purpose, we used a special agent that mapped the FHIR resource data into a format more suitable for data analytics. This mapping agent was responsible for converting the retrieved FHIR resource data via core FHIR APIs into a flat data format. The resulting data elements were then stored in a FHIR-compliant database, ready for analytics. The mapping algorithm is presented in Algorithm 2. Our mapping algorithm worked as a mapping agent between the FHIR API and FHIR-compliant database for data conversion. This mapping algorithm worked seamlessly for all types of resources in our dataset, for example, Patient, AllergyIntolerance, Practitioner, Condition, DiagnosticReport, ServiceRequest, Appointment, etc. Whenever we retrieved FHIR resource data from the FHIR centralized database, we applied the mapping algorithm on the way during FHIR API operations to retrieve and transform the data into the FHIR-compliant database; we named this data-mapping mechanism \u201cData Retrieval on Fly (DRF).\u201d The working mechanisms of this algorithm are illustrated in Figure 6, which depicts how it acted as a mediator between the FHIR API and a FHIR-compliant database, thus enabling the efficient conversion of hierarchical data into tabular data for analytics purposes.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"1\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Algorithm 2.<\/b> Mapping Algorithm (Transform JSON data to EMR format).\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">\n<p><code>1. <b>Function<\/b> void main ()<br \/>\n2.\u2003\u2003 Create Tables in MySQL database, once table for each resources type data and link these tables<br \/>\n3.\u2003\u2003 Resource = Read (FHIR API resource)<br \/>\n4.\u2003\u2003 Templet = Resource-Templet (Resource)<br \/>\n5.\u2003\u2003 counter = Count(Temple)<br \/>\n6.\u2003\u2003 <b>while<\/b> (counter > 0) <b>do<\/b><br \/>\n7. \u2003\u2003 <b>If<\/b> (Templet.Tag == Resource.Tag) <b>then<\/b><br \/>\n8.\u2003\u2003 \u2003\u2003 Table. attribute = Resource.Tag.Value<br \/>\n9. \u2003\u2003 <b>end if<\/b><br \/>\n10.\u2003\u2003 counter = counter \u2212 1<br \/>\n11.\u2003\u2003 <b>end while<\/b><br \/>\n12. <b>end function<\/b> ** main function **<br \/>\n13. ** This function used to compare Resource type **<br \/>\n14. <b>Function<\/b> string Resource-Templet (Resource type)<br \/>\n15. ** Create one dimension array for all resources and stored their tags. This is pre-defined templet for all resources **<br \/>\n16. define string Result<br \/>\n17. String Array List = [Patient, Condition, AllergyIntolerance, Practitioner, ServiceRequest, DiagnosticReport, Appointment, \u2026\u2026\u2026]<br \/>\n18. String Patient [] = [\u201cidentifier\u201d, \u201cname\u201d, \u201ctelecom\u201d, \u201caddress\u201c, \u201cgender\u201d \u2026\u2026\u2026\u2026]<br \/>\n19. String Condition [] = [\u201cidentifier\u201d, \u201cclinical status\u201d, \u201ccategory\u201d, \u201ccode\u201d \u2026\u2026\u2026\u2026]<br \/>\n20. String AllergyIntolerance [] = [\u201cidentifier\u201d, \u201cclinical status\u201d, \u201ccode\u201d, \u2026\u2026\u2026\u2026]<br \/>\n21. String Practitioner [] = [\u201cidentifier\u201d, \u201cname\u201d, \u201caddress\u201d, \u201cqualification\u201d, \u2026\u2026\u2026\u2026]<br \/>\n22. String DiagnosticReport [] = [\u201cidentifier\u201d, \u201cbaseOn\u201d status\u201d, \u201ccategory\u201d, \u201ccode\u201d,\u2026\u2026.\u2026]<br \/>\n23. String ServiceRequest [] = [\u201cidentifier\u201d, \u201cbaseOn\u201d status\u201d, \u201ccategory\u201d, \u201crequester\u201d,\u2026\u2026]<br \/>\n24. String Appointment [] = [\u201cidentifier\u201d, \u201cstatus\u201d, \u201cappointmentType\u201d, \u201cpriority\u201d, \u2026\u2026\u2026\u2026]<br \/>\n25.\u2003\u2003 <b>If<\/b> (type == Patient) <b>then<\/b><br \/>\n26. \u2003\u2003 Result = \u201cPatient\u201d<br \/>\n27.\u2003\u2003 \u2003\u2003 <b>else if<\/b> (type == Condition) <b>then<\/b><br \/>\n28.\u2003\u2003\u2003\u2003 \u2003\u2003 Result = \u201cCondition\u201d<br \/>\n29.\u2003 \u2003\u2003\u2003\u2003 \u2003\u2003 <b>else if<\/b> (type == AllergyIntolerance) <b>then<\/b><br \/>\n30.\u2003 \u2003 \u2003\u2003\u2003\u2003 \u2003\u2003 Result = \u201cAllergyIntolerance\u201d<br \/>\n31.\u2003 \u2003 \u2003 \u2003\u2003\u2003\u2003 \u2003\u2003 <b>else if<\/b> (type == Practitioner) <b>then<\/b><br \/>\n32.\u2003 \u2003 \u2003 \u2003 \u2003\u2003\u2003\u2003 \u2003\u2003 Result = \u201cPractitioner\u201d<br \/>\n33.\u2003 \u2003 \u2003 \u2003 \u2003 \u2003\u2003\u2003\u2003 \u2003\u2003 <b>else if<\/b> (type == DiagnosticReport) <b>then<\/b><br \/>\n34.\u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003\u2003\u2003\u2003 \u2003\u2003 Result = \u201d DiagnosticReport\u201d<br \/>\n35.\u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003\u2003\u2003\u2003 \u2003\u2003 <b>else if<\/b> (type == ServiceRequest) <b>then<\/b><br \/>\n36.\u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003\u2003\u2003\u2003 \u2003\u2003 Result = \u201cServiceRequest\u201d<br \/>\n37.\u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003\u2003\u2003\u2003 \u2003\u2003 <b>else<\/b><br \/>\n38.\u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003 \u2003\u2003\u2003\u2003 \u2003\u2003 Result = \u201cAppointment\u201d<br \/>\n39.\u2003\u2003 <b>end if<\/b><br \/>\n40. return (Result)<br \/>\n41. <b>end function<\/b> ** Resource-Templet function **<br \/>\n42. ** This function used to count the total number of tags in the resource **<br \/>\n43. <b>Function<\/b> int Count(String Templet)<br \/>\n44.\u2003\u2003 int counter = Templet.length<br \/>\n45. return (counter)<br \/>\n46. <b>end function<\/b> ** Count function **<\/code>\n<\/p>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig6_Ayaz_Healthcare23_11-12.png\" class=\"image wiki-link\" data-key=\"ac2d9c489c0bb1071a84e04837f5c5d7\"><img alt=\"Fig6 Ayaz Healthcare23 11-12.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/5\/5e\/Fig6_Ayaz_Healthcare23_11-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 6.<\/b> Mapping algorithm working mechanism.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h3><span class=\"mw-headline\" id=\"FHIR-compliant_database\">FHIR-compliant database<\/span><\/h3>\n<p>We created a special database called the FHIR-compliant database. This is a relational database schema with a collection of tables that have been designed to store the data represented in FHIR resources. The tables are connected with each other, and each table stores clinical data represented in the FHIR resources.\n<\/p><p>We have multiple resources, and each resource represents different types of clinical data. Therefore, first we created a table schema according to the data represented in the FHIR resources and logically connected these tables to facilitate the analytic query engine to query the data from multiple tables according to the workflows in the result generation process. Each resource was stored in a single table or spread across multiple tables, for example, the patient resource data spread across multiple tables, etc. The table\u2019s creation and connection were specifically designed to cater to the needs of the proposed workflows and required result generation.\n<\/p><p>Second, we applied a mapping algorithm that enabled us to retrieve the data elements from the FHIR resources and store them accurately in the corresponding tables in the FHIR-compliant database. The algorithm retrieved the data from the FHIR resources and then pushed it to the corresponding table. When querying the data from the FHIR database, the FHIR query engine utilizes RESTful APIs to read the resources in JSON format. On the way, the mapping algorithm seamlessly pre-processed this JSON data and transformed it to the relational database schema. This process is completed automatically, and all data from all FHIR resources are transformed into the sample EMR data format and stored in the relational tables. Figure 7 presents the FHIR-compliant database.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig7_Ayaz_Healthcare23_11-12.png\" class=\"image wiki-link\" data-key=\"b78e6a769a9fb074cbc9258ba22e2f69\"><img alt=\"Fig7 Ayaz Healthcare23 11-12.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/d\/df\/Fig7_Ayaz_Healthcare23_11-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 7.<\/b> FHIR-compliant database: A sample schema of compliant database.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h3><span class=\"mw-headline\" id=\"Data_analytics_engine_layer\">Data analytics engine layer<\/span><\/h3>\n<p>The data analytics layer plays a key role in this prototype. Once the FHIR resource data are seamlessly mapped to the relational database tables, they become ready for any data analytics operations. The data analytics is based on workflows (use cases scenarios) that we have already designed for optimal results.\n<\/p><p>Our data analytics engine (DAE) is a collection of selective SQL queries proficient in merging data from multiple tables, thereby providing unparalleled data analysis. We have created a series of distinct SQL queries, catering to our business use cases, which are then executed on the data stored in the SQL database to generate exceptional results. The queries have been designed in alignment with our workflows and expected outcomes.\n<\/p><p>The resulting data are unequivocally valuable and accessible to the end-users via an intuitive and efficient user interface. The detail-oriented results generated by the data analytics engine are undoubtedly the backbone of our prototype, providing insights into the data.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"User_interface\">User interface<\/span><\/h3>\n<p>The user interface of our framework is an elegant and sophisticated section, where the end-users access their desired data and obtain results catered specifically to their unique requirements. We developed a user-friendly graphical user interface (GUI) to efficiently process data and generate results.\n<\/p><p>As a demonstration of the utility of our prototype, we developed an experimental data analysis GUI that shows the use of the search operations within the generic tool for exploring FHIR data sets. We created a number of FHIR data sets and a graphic visualization of these data sets that allowed for the demonstration of the data analytics on the clinical data used in healthcare settings and represented in the FHIR-based standard. The user interface of our prototype is presented in Figure 8.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig8_Ayaz_Healthcare23_11-12.png\" class=\"image wiki-link\" data-key=\"558e990c2f1e3206e4fc9df9aa816091\"><img alt=\"Fig8 Ayaz Healthcare23 11-12.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/9\/98\/Fig8_Ayaz_Healthcare23_11-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 8.<\/b> Experimental user interface for data analytics.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h2><span id=\"rdp-ebb-Methods\/Implementation\"><\/span><span class=\"mw-headline\" id=\"Methods.2FImplementation\">Methods\/Implementation<\/span><\/h2>\n<p>In this prototype or research work, we have stored our FHIR resource datasets in our NoSQL database (Mongo DB), which we had developed in our previous prototype. Therefore, we leverage the core FHIR APIs to perform data analytics on the data stored in these FHIR resources. We have employed the technique to download FHIR data into the FHIR-compliant database (SQL DB) and then applied data analytics to this FHIR data. For this purpose, we have utilized our developed mapping algorithm to transfer the FHIR resource data into the relational database tables. This has made it effortless to query data using standard SQL queries or tools and perform data analytics tasks on the data stored in these FHIR resources. It is essential to note that all the retrieval data from the FHIR resources require merging and formatting to support data analysis. As the patient\u2019s unique clinical identifier is the key to connecting these objects, we have utilized this number to merge the data into a group of tables in a relational database to support further querying and analysis.\n<\/p><p>We have an extensive array of FHIR resources stored in our database; therefore, we have utilized the core FHIR GET and Search APIs to retrieve all the resources from the database. These APIs have seamlessly accessed the FHIR resources from the FHIR database, and we have performed various data analytics tasks depending on the defined use cases. To provide our esteemed readers with a clear understanding of these APIs\u2019 working mechanisms, we have discussed how FHIR APIs work for data analytics. The prior Figure 2 illustrates a sample API operation mechanism in which the APIs access the clinical data and then perform the data analytics. This refers to the specialization of the FHIR API that focuses on providing the API\u2019s functionality that is useful for healthcare data analytics applications.\n<\/p><p>This implementation has been executed in two phases:\n<\/p><p><b>Phase 1<\/b>: We have developed various FHIR APIs to retrieve the FHIR resources from the FHIR database and then pre-process these resources using our developed algorithm to map the clinical data elements stored in the FHIR resource tags to a relational data model or schema and store the resulting data into the MySQL database. We have magnificently processed the FHIR resources via our algorithm, retrieved all data elements from these resources, and stored the result in database tables; we called it the FHIR-compliant database. For this purpose, we have crafted a database schema (tables) in the MySQL database (see the prior Figure 7). Each resource type requires different parameters in the REST API URL to retrieve the FHIR resource from the database. Therefore, for each resource, we have provided a resource name and parameters depending on the resource type and data retrieval. For this purpose, we have executed an algorithm to perform this job for us. When the FHIR APIs retrieve resources from the FHIR database, on the way, the mapping agent\/algorithm pre-processes the JSON format of FHIR resources and maps the data stored in various tags of JSON structure into the various MySQL database tables.\n<\/p><p><b>Phase 2<\/b>: When the data were converted from a graph structure to a relational data model format, we applied various data analytics techniques to the data stored in the MySQL database. For this purpose, we have developed various types of SQL queries to generate our results. These SQL queries have impeccably matched the requirements of our use cases, defined for our required data analytics. The output of these data analytics use cases is shown in the Section 7. Figure 9 shows the implementation process, while Figure 10 describes the complete framework process, including the techniques and computational tools applied in each step.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig9_Ayaz_Healthcare23_11-12.png\" class=\"image wiki-link\" data-key=\"668959b69cac0ded497afea24739172e\"><img alt=\"Fig9 Ayaz Healthcare23 11-12.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/1\/1d\/Fig9_Ayaz_Healthcare23_11-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 9.<\/b> Described the implementation components and process.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p><a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig10_Ayaz_Healthcare23_11-12.png\" class=\"image wiki-link\" data-key=\"ba45f540fbb4f8b1e57c9d797da71c05\"><img alt=\"Fig10 Ayaz Healthcare23 11-12.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/a\/ac\/Fig10_Ayaz_Healthcare23_11-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 10.<\/b> Framework working process: Describes each step working and implementation processing.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h2><span class=\"mw-headline\" id=\"Experiments\">Experiments<\/span><\/h2>\n<p>We implemented our data analytics prototype\/concept leveraging the FHIR database (Mongo DB), <a href=\"https:\/\/www.limswiki.org\/index.php\/Python_(programming_language)\" title=\"Python (programming language)\" class=\"wiki-link\" data-key=\"ef6905a29cbb75d3c71e6bdf6e2915dd\">Python 3.9.8 programming language<\/a>, and <a href=\"https:\/\/www.limswiki.org\/index.php\/MySQL\" title=\"MySQL\" class=\"wiki-link\" data-key=\"35005451bfcd508bce47c58e72260128\">MySQL<\/a> 5.6 database. We developed FHIR APIs, which enabled us to seamlessly retrieve various resources stored in the Mongo DB, consisting of a dataset size of 700 resources, inclusive of 100 resources of each resource type. Furthermore, before applying data analytics, we implemented our mapping algorithm\/agent, enabling the smooth transformation of FHIR resource tags to FHIR-compliant database (MYSQL) tables. We used the following:\n<\/p>\n<ul><li><b>Dataset size<\/b>: 700 resources (including 100 resources of each resource type)<\/li>\n<li><b>Hardware<\/b>: 4 Cores, 32 GB of RAM<\/li>\n<li><b>Software<\/b>: Windows 10 OS, Python 3.9.8 programming language, Mongo DB 4.4, MySQL 5.6 DB<\/li><\/ul>\n<p>Our experiment consisted of two phases:\n<\/p><p><b>Phase 1<\/b>: In this step, we implemented our FHIR APIs and executed algorithms to retrieve the FHIR resources from the Mongo DB. Furthermore, we also executed a mapping algorithm to transform the FHIR resource data into the relational database tables.\n<\/p><p><b>Phase 2<\/b>: In this step, we executed various SQL queries to perform highly precise data analytics based on the defined use cases and generate the required results.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Results\">Results<\/span><\/h2>\n<p>To provide the underlying data for our esteemed results, we used the data stored in the relational data model, generated from the FHIR dataset stored in Table 2. The results are based on the use cases we defined in our previous step. We have a number of use cases, each based on a dataset different from others. Therefore, we executed various queries based on the use cases. We generated various results from the FHIR dataset. We are discussing these use cases and their results in detail here.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Use_case_1\">Use case 1<\/span><\/h3>\n<p>In this scenario, the queries used within the patient\u2019s scalability count the number of patients that have been dutifully registered in the healthcare unit. These unparalleled queries seamlessly retrieve data from the patient table, which are associated with the esteemed patient resource in the FHIR dataset. Table 3 shows the retrieval data associated with patients gender-wise, and Figure 11 shows the graphical representation of this data.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"2\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 3.<\/b> Registered patients gender-wise (patient-centered-based analysis).\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Male\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Female\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">55\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">45\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig11_Ayaz_Healthcare23_11-12.png\" class=\"image wiki-link\" data-key=\"d0631224587986b7cb34d124621980ea\"><img alt=\"Fig11 Ayaz Healthcare23 11-12.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/9\/91\/Fig11_Ayaz_Healthcare23_11-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 11.<\/b> Registered patients Gender-wise (patient-centered-based analysis).<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h3><span class=\"mw-headline\" id=\"Use_case_2\">Use case 2<\/span><\/h3>\n<p>In this scenario, the queries used within the patient\u2019s scalability count the number of patients registered within the healthcare unit across a variety of years. These queries are specifically designed to retrieve relevant data from the patient table, which are closely associated with the patient resource within the FHIR dataset. The queries retrieved the data related to patients who have been registered within the healthcare system over a span of several years, ranging from the year 1950 to the year 2021. Table 4 presents the retrieval of the registered patients\u2019 data in various years, while Figure 12 describes a graphical representation of this data.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"10\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 4.<\/b> Registered patients within a specified timeframe (patient-centered-based analysis).\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Year\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">1950\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">1951\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">1952\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">1953\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">1955\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">-----\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">2013\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">2018\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">2021\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Patient's number<\/b>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">-----\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig12_Ayaz_Healthcare23_11-12.png\" class=\"image wiki-link\" data-key=\"616444d1410c91b7c702cce3b239737d\"><img alt=\"Fig12 Ayaz Healthcare23 11-12.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/2\/25\/Fig12_Ayaz_Healthcare23_11-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 12.<\/b> Registered patients within a specified timeframe (patient-centered-based analysis).<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h3><span class=\"mw-headline\" id=\"Use_case_3\">Use case 3<\/span><\/h3>\n<p>In this particular scenario, the patient\u2019s scalability has been measured by employing sophisticated queries aimed at counting the multitude of patients afflicted with diverse types of allergies. These queries were designed to extract relevant data from both the allergy and patient tables, which are associated with the \"Patient\" and \"AllergyIntolerance\" resources within the FHIR dataset. It joined data from these two tables because they belong to \"Patient\" and \"AllergyIntolerance\" resources and are spread across multiple tables and FHIR resources. Via these queries, relevant information relating to patients suffering from various allergies has been successfully retrieved. Table 5 presents the success of these queries, providing a comprehensive breakdown of the number of patients affected by different types of allergies. Furthermore, Figure 13 shows the graphical representation of this result.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"3\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 5.<\/b> Number of patients having various types of allergies (cohort-based interactive analyses).\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Number\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Allergy\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Number of patients\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Shellfish\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">9\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Glyburide\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">8\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Latex\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">5\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">4\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Coal tar\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">6\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">5\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Neomycin\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">12\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">6\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Codeine\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">8\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">7\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">IVP dye\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">10\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">8\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Caffeine\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">5\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">9\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Levaquin\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">5\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">10\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Seafood\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">6\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">11\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Rifampin\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">12\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Norco\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">6\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">13\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Penicillium\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">5\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">14\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Benztropine\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">6\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">15\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Watermelon\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">16\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Metoprolol\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">17\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">IV dye\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig13_Ayaz_Healthcare23_11-12.png\" class=\"image wiki-link\" data-key=\"e392279c3a96cf6061d728b045eca861\"><img alt=\"Fig13 Ayaz Healthcare23 11-12.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/7\/74\/Fig13_Ayaz_Healthcare23_11-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 13.<\/b> Patients and various types of allergies association (cohort-based interactive analyses).<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h3><span class=\"mw-headline\" id=\"Use_case_4\">Use case 4<\/span><\/h3>\n<p>In this particular scenario, the queries employed in the patient\u2019s scalability quantify the number of distinct medical tests that have been requested by either a healthcare organization or a practitioner. These queries procure data from various tables, including patient, order, provider, practitioner, etc., which are associated with the \"Patient,\" \"Practitioner,\" \"DiagnosticReport,\" and \"ServiceRequest\" resources in the FHIR dataset. The queries retrieved information related to various types of test orders that are present within the healthcare system. Table 6 represents the various types of medical tests undertaken by the patient. Additionally, the graphical representation of this data is illustrated in Figure 14.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"9\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 6.<\/b> Patient various types of medical test orders (cohort-based interactive analyses).\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Test name\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">HIV\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">CBC\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">CT scan\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">X-ray, ankle\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">MRI\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Blood culture\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">COVID\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">SGPT\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Test order percentage<\/b>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">16\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">15\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">15\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">14\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">12\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">10\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">9\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">9\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig14_Ayaz_Healthcare23_11-12.png\" class=\"image wiki-link\" data-key=\"a8b8c4ef1e5b5c978eed4b860161c822\"><img alt=\"Fig14 Ayaz Healthcare23 11-12.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/f\/fa\/Fig14_Ayaz_Healthcare23_11-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 14.<\/b> Patient various types of medical test orders (cohort-based interactive analyses).<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p><br \/>\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Use_case_5\">Use case 5<\/span><\/h3>\n<p>In this scenario, the queries employed within the patient\u2019s scalability counts the number of sundry categories of medical tests ordered by a healthcare organization or practitioner in different years. These queries procure data from various tables, including patient, order, provider, practitioner, etc., which are associated with the \"Patient,\" \"Practitioner,\" \"DiagnosticReport,\" and \"ServiceRequest\" resources in the FHIR dataset. These queries have retrieved relevant information regarding assorted test orders in the healthcare system spanning a timeline from 1950 to 2021. The resulting outcome of these queries has been presented in Table 7, summarizing the diverse medical tests undertaken by the patient. Additionally, Figure 15 describes the graphical representation of this information.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"10\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 7.<\/b> Patient various types of medical test orders within a specified timeframe (cohort-based interactive analyses).\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Year\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">1951\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">1952\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">1953\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">1955\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">-----\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">2010\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">2015\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">2019\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">2020\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Number of tests ordered<\/b>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">4\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">4\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">-----\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">5\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">6\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">5\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p><br \/>\n<a href=\"https:\/\/www.limswiki.org\/index.php\/File:Fig15_Ayaz_Healthcare23_11-12.png\" class=\"image wiki-link\" data-key=\"9c09359f7b3c99aaa695804adfcf8126\"><img alt=\"Fig15 Ayaz Healthcare23 11-12.png\" src=\"https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/e\/ef\/Fig15_Ayaz_Healthcare23_11-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Figure 15.<\/b> Patient various types of medical test orders within a specified timeframe (cohort-based interactive analyses).<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h2><span class=\"mw-headline\" id=\"Limitations\">Limitations<\/span><\/h2>\n<p>Our developed framework is capable of performing various types of descriptive data analytics on clinical data used in healthcare settings and represented in the FHIR-based standard. However, it is important to note that our study is limited in that it focuses solely on business use cases for patient clinical data belonging to two HID: PRSs and LISs. Other possible data analysis workflows and customized research scenarios based on patient data from other HISs could be performed on FHIR-based data, but our current framework or tool does not directly support them without modification. In addition, there are some technical challenges in this research work:\n<\/p>\n<ol><li>Our framework is currently developed under the FHIR R4 version and needs to be upgraded to the official FHIR R5 version when it gets finalized and released by HL7.<\/li>\n<li>Our framework might face issues in the coming FHIR version. HL7 FHIR specification requirements are changing over time, and the current resources might be replaced with any other new resources in the coming FHIR version. Additionally, the resource nature (from non-normative to normative) is changing over time. In this case, our framework might face challenges. Therefore, it needs to be updated in the coming FHIR versions if any of the mentioned cases happen. However, if none of these changes happen in the FHIR R5 version, it will work perfectly.<\/li>\n<li>Our framework executed multiple algorithms, such as the algorithm for accessing the FHIR resources via the RESTful APIs and the algorithm to map data from the FHIR resources to the EMR data format, and executed queries to perform data analytics for the end users. Therefore, the performance might not be ideal for every dataset. It worked excellently for our dataset (which is small), but the performance might be affected when dealing with large datasets, for example, when the number of resources and data elements in the dataset is in the billions or trillions.<\/li>\n<li>The interface of our framework works for our dataset (patient data used in PRSs and LISs); therefore, it would update if the workflow changed and included the data from other HISs.<\/li><\/ol>\n<h2><span class=\"mw-headline\" id=\"Discussion\">Discussion<\/span><\/h2>\n<p>In this study, we have developed an integrated framework or visual tool leveraging the cutting-edge FHIR standard, with prototype implementation and evaluation, aiming to empower standardized clinical statistics and analysis applications. This research work has three main contributions: First, the entire framework and workflow design follow the FHIR data standards, which could be reused for any other clinical data domain and could provide support for any clinical data that follows the FHIR standard. Second, the data analysis workflow and tools incorporate the experience of clinical researchers and statisticians and leverage powerful Python analytics, which could provide a starting point for FHIR researchers in this cutting-edge standard. Third, the intelligent mapping algorithm, artfully designed to facilitate the sublime process of data analytics or data analysis within the realm of FHIR-based data. The mapping algorithm could be reused for any other clinical data that follow the FHIR specification and need to process the FHIR-based data for other purposes, such as research or developing an AI or ML model, etc.\n<\/p><p>Our research effectively used the data-mapping algorithm for FHIR-based data to facilitate the data analytics process. Furthermore, mapped data could be utilized for other purposes, such as research, etc. Although recently, another technique, namely pathling<sup id=\"rdp-ebb-cite_ref-:3_14-7\" class=\"reference\"><a href=\"#cite_note-:3-14\">[14]<\/a><\/sup>, has been used for data analytics on FHIR-based data. However, it works in a limited domain because some operations are not easily or even currently possible to achieve via the FHIR REST API specification, such as data aggregation, searching the data, etc. Therefore, it is extremely challenging to implement. Furthermore, this technique is language-specific. Therefore, it needs to redesign the entire framework for a new language. Our technique is easy to implement and generally could be used for all FHIR-based data types and FHIR resources with minor modifications. Furthermore, the implementation process would work for every language.\n<\/p><p>Our developed framework or tool provides a user-friendly GUI to the end-users, such as healthcare professionals and researchers. The developed interface is used for FHIR data mapping and analytics purposes. Therefore, we developed two sub-menus, one for data mapping and a second for data analytic purposes (see the prior Figure 8). However, we only discussed the data analytics sub-menu in this research work. The data mapping sub-menu is out of the scope of this study. Our data analytics sub-menu provided all options for our required results based on the defined use-cases. For example, the \u201cRegistered Patients\u201d option provided results for all registered patients in the patient information systems. Similarly, \u201cTest Order\u201d generates the results of various types of patient laboratory tests ordered by any practitioner, healthcare organization, laboratory, etc. All the remaining options work accordingly. In short, it could greatly facilitate interactive, user-friendly data analysis.\n<\/p><p>In the future, we have a plan to extend our framework by adding data from other HISs and updating the framework, including the data workflows and user interface, to make it more generic for users and researchers. Furthermore, we also intend to adopt the FHIR R5 version with particular <a href=\"https:\/\/www.limswiki.org\/index.php\/COVID-19\" class=\"mw-redirect wiki-link\" title=\"COVID-19\" data-key=\"da9bd20c492b2a17074ad66c2fe25652\">COVID-19<\/a> and <a href=\"https:\/\/www.limswiki.org\/index.php\/Cancer\" title=\"Cancer\" class=\"wiki-link\" data-key=\"fcd6751ee9aef5d96b8448c082d5e582\">cancer<\/a>-related resource definitions to represent COVID-19 and cancer data in our framework. This will help people working in the healthcare industry to enhance the consistency and quality of data analysis for COVID-19 and cancer data. Moreover, it will open more research dimensions for healthcare data analytic researchers in these areas.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Conclusions\">Conclusions<\/span><\/h2>\n<p>In this study, we discussed the need for a data analytics tool to improve data analysis and reduce the skill burden in the healthcare industry. We have designed a comprehensive framework that empowers healthcare users (patients, practitioners, healthcare providers, etc.) to perform advanced data analysis on patient data used in healthcare settings and represented in the FHIR-based standard. The framework incorporates different data workflows based on patient data derived from two HISs, namely PRSs and LISs, represented in the FHIR-based standard. Our use cases facilitate both patient-centered and cohort-based analysis and address common clinical user and researcher requirements. Although currently limited to two HISs, the framework is flexible and can be extended to include data from other systems represented in the FHIR-based standard. With ongoing improvements, our framework will be valuable for healthcare applications in statistics and analytics. Overall, the goal of developing a state-of-the-art data analytics framework for clinical data in healthcare settings has been achieved.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Acknowledgements\">Acknowledgements<\/span><\/h2>\n<h3><span class=\"mw-headline\" id=\"Author_contributions\">Author contributions<\/span><\/h3>\n<p>Conceptualization, M.A.; Methodology, M.A. and H.K.A.; software, M.A.; validation, M.A.; Formal analysis, T.J.A. and H.K.A.; Investigation, N.N.B.A., M.A., and M.F.P.; Data curation, N.N.B.A. and T.J.A.; Writing\u2014original draft, M.A.; Writing\u2014review & editing, M.A.; visualization, M.A.; Supervision, M.F.P. All authors have read and agreed to the published version of the manuscript.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Funding\">Funding<\/span><\/h3>\n<p>This research supported by Princess Nourah bint Abdulrahman University Researchers Supporting Project number (PNURSP2023R384), Princess Nourah bint Abdulrahman University, and P.O. Box 84428, Riyadh 11671, Saudi Arabia.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Informed_consent\">Informed consent<\/span><\/h3>\n<p>Informed consent was obtained from all subjects involved in the study. In this manuscript, we used data from the MIMIC-III database. The establishment of this database was approved by the Massachusetts Institute of Technology (Cambridge, MA, USA) and Beth Israel Deaconess Medical Center (Boston, MA, USA), and consent was obtained for the original data collection. Therefore, the ethical approval statement and the need for informed consent were waived for this manuscript.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Data_availability_statement\">Data availability statement<\/span><\/h3>\n<p>The datasets used or analyzed in this study are available from the corresponding author on reasonable request.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Conflict_of_interest\">Conflict of interest<\/span><\/h3>\n<p>All authors declare that they have no conflict of interest.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"References\">References<\/span><\/h2>\n<div class=\"reflist references-column-width\" style=\"-moz-column-width: 30em; -webkit-column-width: 30em; column-width: 30em; list-style-type: decimal;\">\n<div class=\"mw-references-wrap mw-references-columns\"><ol class=\"references\">\n<li id=\"cite_note-:0-1\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:0_1-0\">1.0<\/a><\/sup> <sup><a href=\"#cite_ref-:0_1-1\">1.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Safran, C.; Bloomrosen, M.; Hammond, W. E.; Labkoff, S.; Markel-Fox, S.; Tang, P. C.; Detmer, D. E. (1 January 2007). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/academic.oup.com\/jamia\/article-lookup\/doi\/10.1197\/jamia.M2273\" target=\"_blank\">\"Toward a National Framework for the Secondary Use of Health Data: An American Medical Informatics Association White Paper\"<\/a> (in en). <i>Journal of the American Medical Informatics Association<\/i> <b>14<\/b> (1): 1\u20139. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1197%2Fjamia.M2273\" target=\"_blank\">10.1197\/jamia.M2273<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1067-5027\" target=\"_blank\">1067-5027<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC2329823\/\" target=\"_blank\">PMC2329823<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/17077452\" target=\"_blank\">17077452<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/academic.oup.com\/jamia\/article-lookup\/doi\/10.1197\/jamia.M2273\" target=\"_blank\">https:\/\/academic.oup.com\/jamia\/article-lookup\/doi\/10.1197\/jamia.M2273<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Toward+a+National+Framework+for+the+Secondary+Use+of+Health+Data%3A+An+American+Medical+Informatics+Association+White+Paper&rft.jtitle=Journal+of+the+American+Medical+Informatics+Association&rft.aulast=Safran&rft.aufirst=C.&rft.au=Safran%2C%26%2332%3BC.&rft.au=Bloomrosen%2C%26%2332%3BM.&rft.au=Hammond%2C%26%2332%3BW.+E.&rft.au=Labkoff%2C%26%2332%3BS.&rft.au=Markel-Fox%2C%26%2332%3BS.&rft.au=Tang%2C%26%2332%3BP.+C.&rft.au=Detmer%2C%26%2332%3BD.+E.&rft.date=1+January+2007&rft.volume=14&rft.issue=1&rft.pages=1%E2%80%939&rft_id=info:doi\/10.1197%2Fjamia.M2273&rft.issn=1067-5027&rft_id=info:pmc\/PMC2329823&rft_id=info:pmid\/17077452&rft_id=https%3A%2F%2Facademic.oup.com%2Fjamia%2Farticle-lookup%2Fdoi%2F10.1197%2Fjamia.M2273&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-2\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-2\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Trotter, F. (20 August 2012). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/thehealthcareblog.com\/blog\/2012\/08\/20\/who-owns-patient-data\/\" target=\"_blank\">\"Who Owns Patient Data?\"<\/a>. <i>The Health Care Blog<\/i>. Holt, M.<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/thehealthcareblog.com\/blog\/2012\/08\/20\/who-owns-patient-data\/\" target=\"_blank\">https:\/\/thehealthcareblog.com\/blog\/2012\/08\/20\/who-owns-patient-data\/<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 04 December 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Who+Owns+Patient+Data%3F&rft.atitle=The+Health+Care+Blog&rft.aulast=Trotter%2C+F.&rft.au=Trotter%2C+F.&rft.date=20+August+2012&rft.pub=Holt%2C+M.&rft_id=https%3A%2F%2Fthehealthcareblog.com%2Fblog%2F2012%2F08%2F20%2Fwho-owns-patient-data%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:1-3\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:1_3-0\">3.0<\/a><\/sup> <sup><a href=\"#cite_ref-:1_3-1\">3.1<\/a><\/sup> <sup><a href=\"#cite_ref-:1_3-2\">3.2<\/a><\/sup> <sup><a href=\"#cite_ref-:1_3-3\">3.3<\/a><\/sup> <sup><a href=\"#cite_ref-:1_3-4\">3.4<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation book\">Hersh, W.R. (2014). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/dmice.ohsu.edu\/hersh\/hoyt-14-analytics.pdf\" target=\"_blank\">\"Chapter 3: Healthcare Data Analytics\"<\/a>. In Hoyt, Robert E.; Yoshihashi, Ann. <i>Health informatics: practical guide for healthcare and information technology professionals<\/i> (Fifth ed.). Informatics Education. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-304-79110-8<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/dmice.ohsu.edu\/hersh\/hoyt-14-analytics.pdf\" target=\"_blank\">https:\/\/dmice.ohsu.edu\/hersh\/hoyt-14-analytics.pdf<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Chapter+3%3A+Healthcare+Data+Analytics&rft.atitle=Health+informatics%3A+practical+guide+for+healthcare+and+information+technology+professionals&rft.aulast=Hersh%2C+W.R.&rft.au=Hersh%2C+W.R.&rft.date=2014&rft.edition=Fifth&rft.pub=Informatics+Education&rft.isbn=978-1-304-79110-8&rft_id=https%3A%2F%2Fdmice.ohsu.edu%2Fhersh%2Fhoyt-14-analytics.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-4\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-4\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation book\">Davenport, Thomas H.; Harris, Jeanne G. (2007). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/ocm74649067\" target=\"_blank\"><i>Competing on analytics: the new science of winning<\/i><\/a>. Boston, Mass: Harvard Business School Press. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-4221-0332-6. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Online_Computer_Library_Center\" data-key=\"b53206e2204c7e657858a88b56c8ac4a\">OCLC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/oclc\/ocm74649067\" target=\"_blank\">ocm74649067<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/ocm74649067\" target=\"_blank\">https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/ocm74649067<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=Competing+on+analytics%3A+the+new+science+of+winning&rft.aulast=Davenport&rft.aufirst=Thomas+H.&rft.au=Davenport%2C%26%2332%3BThomas+H.&rft.au=Harris%2C%26%2332%3BJeanne+G.&rft.date=2007&rft.place=Boston%2C+Mass&rft.pub=Harvard+Business+School+Press&rft.isbn=978-1-4221-0332-6&rft_id=info:oclcnum\/ocm74649067&rft_id=https%3A%2F%2Fwww.worldcat.org%2Ftitle%2Fmediawiki%2Foclc%2Focm74649067&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-5\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-5\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Cortada, J.W.; Gordon, D.; Leniban, B. (January 2012). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/web.archive.org\/web\/20230811171408\/https:\/\/dl200.dokumen.tips\/file_download\/file1\/55cf864e550346484b9653d8?ext=pdf&vaild=0dd1e37e9f43ee23f6c4498f7ea6068f889867446631bb6d00f103b2be23026403af450c3f0533792a3d70fefb74d4a3224e33087229ddc130762dfe529d5d3eGwX9zBiXxNLqkR1dRP8JhSIvoVq8eA7QwvpZf4WRyR8XqCxum0FIGy%2FtXjig%2BNXcyJEVYfvEgLNvMpc7ieUN%2BiIwx2Xor%2F0hLDjO1tU%2FGAk%3D\" target=\"_blank\">\"The value of analytics in healthcare: From insights to outcomes\"<\/a> (PDF). IBM Institute for Business Value. Archived from <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www-935.ibm.com\/services\/us\/gbs\/thoughtleadership\/ibvhealthcare-analytics.html\" target=\"_blank\">the original<\/a> on 11 August 2023<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/web.archive.org\/web\/20230811171408\/https:\/\/dl200.dokumen.tips\/file_download\/file1\/55cf864e550346484b9653d8?ext=pdf&vaild=0dd1e37e9f43ee23f6c4498f7ea6068f889867446631bb6d00f103b2be23026403af450c3f0533792a3d70fefb74d4a3224e33087229ddc130762dfe529d5d3eGwX9zBiXxNLqkR1dRP8JhSIvoVq8eA7QwvpZf4WRyR8XqCxum0FIGy%2FtXjig%2BNXcyJEVYfvEgLNvMpc7ieUN%2BiIwx2Xor%2F0hLDjO1tU%2FGAk%3D\" target=\"_blank\">https:\/\/web.archive.org\/web\/20230811171408\/https:\/\/dl200.dokumen.tips\/file_download\/file1\/55cf864e550346484b9653d8?ext=pdf&vaild=0dd1e37e9f43ee23f6c4498f7ea6068f889867446631bb6d00f103b2be23026403af450c3f0533792a3d70fefb74d4a3224e33087229ddc130762dfe529d5d3eGwX9zBiXxNLqkR1dRP8JhSIvoVq8eA7QwvpZf4WRyR8XqCxum0FIGy%2FtXjig%2BNXcyJEVYfvEgLNvMpc7ieUN%2BiIwx2Xor%2F0hLDjO1tU%2FGAk%3D<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 04 December 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=The+value+of+analytics+in+healthcare%3A+From+insights+to+outcomes&rft.atitle=&rft.aulast=Cortada%2C+J.W.%3B+Gordon%2C+D.%3B+Leniban%2C+B.&rft.au=Cortada%2C+J.W.%3B+Gordon%2C+D.%3B+Leniban%2C+B.&rft.date=January+2012&rft.pub=IBM+Institute+for+Business+Value&rft_id=https%3A%2F%2Fweb.archive.org%2Fweb%2F20230811171408%2Fhttps%3A%2F%2Fdl200.dokumen.tips%2Ffile_download%2Ffile1%2F55cf864e550346484b9653d8%3Fext%3Dpdf%26vaild%3D0dd1e37e9f43ee23f6c4498f7ea6068f889867446631bb6d00f103b2be23026403af450c3f0533792a3d70fefb74d4a3224e33087229ddc130762dfe529d5d3eGwX9zBiXxNLqkR1dRP8JhSIvoVq8eA7QwvpZf4WRyR8XqCxum0FIGy%252FtXjig%252BNXcyJEVYfvEgLNvMpc7ieUN%252BiIwx2Xor%252F0hLDjO1tU%252FGAk%253D&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-6\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-6\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Adams, J.; Klein, J. (2011). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.advisory.com\/Research\/IT-Strategy-Council\/Research-Notes\/2011\/Business-Intelligence-and-Analytics-in-Health-C\" target=\"_blank\">\"Business Intelligence and Analytics in Health Care\u2014A Primer\"<\/a>. The Advisory Board Company<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.advisory.com\/Research\/IT-Strategy-Council\/Research-Notes\/2011\/Business-Intelligence-and-Analytics-in-Health-C\" target=\"_blank\">https:\/\/www.advisory.com\/Research\/IT-Strategy-Council\/Research-Notes\/2011\/Business-Intelligence-and-Analytics-in-Health-C<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 06 December 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Business+Intelligence+and+Analytics+in+Health+Care%E2%80%94A+Primer&rft.atitle=&rft.aulast=Adams%2C+J.%3B+Klein%2C+J.&rft.au=Adams%2C+J.%3B+Klein%2C+J.&rft.date=2011&rft.pub=The+Advisory+Board+Company&rft_id=https%3A%2F%2Fwww.advisory.com%2FResearch%2FIT-Strategy-Council%2FResearch-Notes%2F2011%2FBusiness-Intelligence-and-Analytics-in-Health-C&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-7\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-7\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Gardner, E. (1 March 2023). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/web.archive.org\/web\/20130302053700\/http:\/\/www.healthdatamanagement.com\/issues\/21_3\/The-HIT-Approach-to-Big-Data-Anayltics-45735-1.html\" target=\"_blank\">\"The HIT Approach to Big Data\"<\/a>. <i>HealthData Management<\/i>. SourceMedia. Archived from <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.healthdatamanagement.com\/issues\/21_3\/The-HIT-Approach-to-Big-Data-Anayltics-45735-1.html\" target=\"_blank\">the original<\/a> on 02 March 2013<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/web.archive.org\/web\/20130302053700\/http:\/\/www.healthdatamanagement.com\/issues\/21_3\/The-HIT-Approach-to-Big-Data-Anayltics-45735-1.html\" target=\"_blank\">https:\/\/web.archive.org\/web\/20130302053700\/http:\/\/www.healthdatamanagement.com\/issues\/21_3\/The-HIT-Approach-to-Big-Data-Anayltics-45735-1.html<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 30 November 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=The+HIT+Approach+to+Big+Data&rft.atitle=HealthData+Management&rft.aulast=Gardner%2C+E.&rft.au=Gardner%2C+E.&rft.date=1+March+2023&rft.pub=SourceMedia&rft_id=https%3A%2F%2Fweb.archive.org%2Fweb%2F20130302053700%2Fhttp%3A%2F%2Fwww.healthdatamanagement.com%2Fissues%2F21_3%2FThe-HIT-Approach-to-Big-Data-Anayltics-45735-1.html&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-8\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-8\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Sledge, G.W.; Miller, R.S.; Hauser. R. (3 June 2013). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/web.archive.org\/web\/20180602212656\/https:\/\/meetinglibrary.asco.org\/record\/78971\/edbook\" target=\"_blank\">\"CancerLinQ and the Future of Cancer Care\"<\/a>. <i>ASCO Meeting Library<\/i>. ASCO University. Archived from <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/meetinglibrary.asco.org\/content\/58-132\" target=\"_blank\">the original<\/a> on 02 June 2018<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/web.archive.org\/web\/20180602212656\/https:\/\/meetinglibrary.asco.org\/record\/78971\/edbook\" target=\"_blank\">https:\/\/web.archive.org\/web\/20180602212656\/https:\/\/meetinglibrary.asco.org\/record\/78971\/edbook<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 02 December 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=CancerLinQ+and+the+Future+of+Cancer+Care&rft.atitle=ASCO+Meeting+Library&rft.aulast=Sledge%2C+G.W.%3B+Miller%2C+R.S.%3B+Hauser.+R.&rft.au=Sledge%2C+G.W.%3B+Miller%2C+R.S.%3B+Hauser.+R.&rft.date=3+June+2013&rft.pub=ASCO+University&rft_id=https%3A%2F%2Fweb.archive.org%2Fweb%2F20180602212656%2Fhttps%3A%2F%2Fmeetinglibrary.asco.org%2Frecord%2F78971%2Fedbook&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-9\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-9\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ayaz, Muhammad; Pasha, Muhammad F.; Alzahrani, Mohammed Y.; Budiarto, Rahmat; Stiawan, Deris (30 July 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/medinform.jmir.org\/2021\/7\/e21929\" target=\"_blank\">\"The Fast Health Interoperability Resources (FHIR) Standard: Systematic Literature Review of Implementations, Applications, Challenges and Opportunities\"<\/a> (in EN). <i>JMIR Medical Informatics<\/i> <b>9<\/b> (7): e21929. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.2196%2F21929\" target=\"_blank\">10.2196\/21929<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/medinform.jmir.org\/2021\/7\/e21929\" target=\"_blank\">https:\/\/medinform.jmir.org\/2021\/7\/e21929<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+Fast+Health+Interoperability+Resources+%28FHIR%29+Standard%3A+Systematic+Literature+Review+of+Implementations%2C+Applications%2C+Challenges+and+Opportunities&rft.jtitle=JMIR+Medical+Informatics&rft.aulast=Ayaz&rft.aufirst=Muhammad&rft.au=Ayaz%2C%26%2332%3BMuhammad&rft.au=Pasha%2C%26%2332%3BMuhammad+F.&rft.au=Alzahrani%2C%26%2332%3BMohammed+Y.&rft.au=Budiarto%2C%26%2332%3BRahmat&rft.au=Stiawan%2C%26%2332%3BDeris&rft.date=30+July+2021&rft.volume=9&rft.issue=7&rft.pages=e21929&rft_id=info:doi\/10.2196%2F21929&rft_id=https%3A%2F%2Fmedinform.jmir.org%2F2021%2F7%2Fe21929&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-10\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-10\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Centers for Medicare & Medicaid Services (2023). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ecqi.healthit.gov\/fhir\" target=\"_blank\">\"FHIR - Fast Healthcare Interoperability Resources\"<\/a>. <i>eCQI Resource Center<\/i>. Centers for Medicare & Medicaid Services<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ecqi.healthit.gov\/fhir\" target=\"_blank\">https:\/\/ecqi.healthit.gov\/fhir<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 10 February 2023<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=FHIR+-+Fast+Healthcare+Interoperability+Resources&rft.atitle=eCQI+Resource+Center&rft.aulast=Centers+for+Medicare+%26+Medicaid+Services&rft.au=Centers+for+Medicare+%26+Medicaid+Services&rft.date=2023&rft.pub=Centers+for+Medicare+%26+Medicaid+Services&rft_id=https%3A%2F%2Fecqi.healthit.gov%2Ffhir&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-11\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-11\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation\" id=\"rdp-ebb-CITEREFBraunstein2018\">Braunstein, Mark L. (2018), <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/978-3-319-93414-3_10\" target=\"_blank\">\"SMART on FHIR\"<\/a> (in en), <i>Health Informatics on FHIR: How HL7's New API is Transforming Healthcare<\/i> (Cham: Springer International Publishing): 205\u2013225, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2F978-3-319-93414-3_10\" target=\"_blank\">10.1007\/978-3-319-93414-3_10<\/a>, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-3-319-93413-6<span class=\"printonly\">, <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/978-3-319-93414-3_10\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/978-3-319-93414-3_10<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 2023-08-11<\/span><\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=SMART+on+FHIR&rft.jtitle=Health+Informatics+on+FHIR%3A+How+HL7%27s+New+API+is+Transforming+Healthcare&rft.aulast=Braunstein&rft.aufirst=Mark+L.&rft.au=Braunstein%2C%26%2332%3BMark+L.&rft.date=2018&rft.pages=205%E2%80%93225&rft.place=Cham&rft.pub=Springer+International+Publishing&rft_id=info:doi\/10.1007%2F978-3-319-93414-3_10&rft.isbn=978-3-319-93413-6&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2F978-3-319-93414-3_10&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:2-12\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:2_12-0\">12.0<\/a><\/sup> <sup><a href=\"#cite_ref-:2_12-1\">12.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation web\">Staff Reporter, APAC (28 November 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/healthcareasiamagazine.com\/co-written-partner\/analytics-and-data-driven-healthcare-be-fuelled-fhir-interoperability-boost-intersystems-anz-study\" target=\"_blank\">\"Analytics and Data-Driven Healthcare to Be Fuelled by FHIR Interoperability Boost: InterSystems ANZ Study\"<\/a>. <i>HealthcareAsia<\/i><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/healthcareasiamagazine.com\/co-written-partner\/analytics-and-data-driven-healthcare-be-fuelled-fhir-interoperability-boost-intersystems-anz-study\" target=\"_blank\">https:\/\/healthcareasiamagazine.com\/co-written-partner\/analytics-and-data-driven-healthcare-be-fuelled-fhir-interoperability-boost-intersystems-anz-study<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 04 December 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Analytics+and+Data-Driven+Healthcare+to+Be+Fuelled+by+FHIR+Interoperability+Boost%3A+InterSystems+ANZ+Study&rft.atitle=HealthcareAsia&rft.aulast=Staff+Reporter%2C+APAC&rft.au=Staff+Reporter%2C+APAC&rft.date=28+November+2022&rft_id=https%3A%2F%2Fhealthcareasiamagazine.com%2Fco-written-partner%2Fanalytics-and-data-driven-healthcare-be-fuelled-fhir-interoperability-boost-intersystems-anz-study&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-13\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-13\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Ostrovskiy, S. (10 November 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/edenlab.io\/blog\/what-is-fhir-a-brief-overview-of-its-role-in-interoperability\" target=\"_blank\">\"What Is FHIR: A Brief Overview of Its Role in Interoperability\"<\/a>. <i>Edenlab<\/i><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/edenlab.io\/blog\/what-is-fhir-a-brief-overview-of-its-role-in-interoperability\" target=\"_blank\">https:\/\/edenlab.io\/blog\/what-is-fhir-a-brief-overview-of-its-role-in-interoperability<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 04 December 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=What+Is+FHIR%3A+A+Brief+Overview+of+Its+Role+in+Interoperability&rft.atitle=Edenlab&rft.aulast=Ostrovskiy%2C+S.&rft.au=Ostrovskiy%2C+S.&rft.date=10+November+2021&rft_id=https%3A%2F%2Fedenlab.io%2Fblog%2Fwhat-is-fhir-a-brief-overview-of-its-role-in-interoperability&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:3-14\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:3_14-0\">14.0<\/a><\/sup> <sup><a href=\"#cite_ref-:3_14-1\">14.1<\/a><\/sup> <sup><a href=\"#cite_ref-:3_14-2\">14.2<\/a><\/sup> <sup><a href=\"#cite_ref-:3_14-3\">14.3<\/a><\/sup> <sup><a href=\"#cite_ref-:3_14-4\">14.4<\/a><\/sup> <sup><a href=\"#cite_ref-:3_14-5\">14.5<\/a><\/sup> <sup><a href=\"#cite_ref-:3_14-6\">14.6<\/a><\/sup> <sup><a href=\"#cite_ref-:3_14-7\">14.7<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Grimes, John; Szul, Piotr; Metke-Jimenez, Alejandro; Lawley, Michael; Loi, Kylynn (8 September 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/jbiomedsem.biomedcentral.com\/articles\/10.1186\/s13326-022-00277-1\" target=\"_blank\">\"Pathling: analytics on FHIR\"<\/a> (in en). <i>Journal of Biomedical Semantics<\/i> <b>13<\/b> (1): 23. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1186%2Fs13326-022-00277-1\" target=\"_blank\">10.1186\/s13326-022-00277-1<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2041-1480\" target=\"_blank\">2041-1480<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC9455941\/\" target=\"_blank\">PMC9455941<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/36076268\" target=\"_blank\">36076268<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/jbiomedsem.biomedcentral.com\/articles\/10.1186\/s13326-022-00277-1\" target=\"_blank\">https:\/\/jbiomedsem.biomedcentral.com\/articles\/10.1186\/s13326-022-00277-1<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Pathling%3A+analytics+on+FHIR&rft.jtitle=Journal+of+Biomedical+Semantics&rft.aulast=Grimes&rft.aufirst=John&rft.au=Grimes%2C%26%2332%3BJohn&rft.au=Szul%2C%26%2332%3BPiotr&rft.au=Metke-Jimenez%2C%26%2332%3BAlejandro&rft.au=Lawley%2C%26%2332%3BMichael&rft.au=Loi%2C%26%2332%3BKylynn&rft.date=8+September+2022&rft.volume=13&rft.issue=1&rft.pages=23&rft_id=info:doi\/10.1186%2Fs13326-022-00277-1&rft.issn=2041-1480&rft_id=info:pmc\/PMC9455941&rft_id=info:pmid\/36076268&rft_id=https%3A%2F%2Fjbiomedsem.biomedcentral.com%2Farticles%2F10.1186%2Fs13326-022-00277-1&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:4-15\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:4_15-0\">15.0<\/a><\/sup> <sup><a href=\"#cite_ref-:4_15-1\">15.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Lehne, Moritz; Luijten, Sandra; Vom Felde Genannt Imbusch, Paulina; Thun, Sylvia (2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/SHTI190805\" target=\"_blank\">\"The Use of FHIR in Digital Health \u2013 A Review of the Scientific Literature\"<\/a>. <i>German Medical Data Sciences: Shaping Change \u2013 Creative Solutions for Innovative Medicine<\/i>: 52\u201358. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3233%2FSHTI190805\" target=\"_blank\">10.3233\/SHTI190805<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/SHTI190805\" target=\"_blank\">https:\/\/ebooks.iospress.nl\/doi\/10.3233\/SHTI190805<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+Use+of+FHIR+in+Digital+Health+%E2%80%93+A+Review+of+the+Scientific+Literature&rft.jtitle=German+Medical+Data+Sciences%3A+Shaping+Change+%E2%80%93+Creative+Solutions+for+Innovative+Medicine&rft.aulast=Lehne&rft.aufirst=Moritz&rft.au=Lehne%2C%26%2332%3BMoritz&rft.au=Luijten%2C%26%2332%3BSandra&rft.au=Vom+Felde+Genannt+Imbusch%2C%26%2332%3BPaulina&rft.au=Thun%2C%26%2332%3BSylvia&rft.date=2019&rft.pages=52%E2%80%9358&rft_id=info:doi\/10.3233%2FSHTI190805&rft_id=https%3A%2F%2Febooks.iospress.nl%2Fdoi%2F10.3233%2FSHTI190805&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-16\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-16\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/qrvey.com\/fhir-healthcare-analytics\/\" target=\"_blank\">\"FHIR Analytics in Healthcare\"<\/a>. Qrvey, Inc<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/qrvey.com\/fhir-healthcare-analytics\/\" target=\"_blank\">https:\/\/qrvey.com\/fhir-healthcare-analytics\/<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 05 December 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=FHIR+Analytics+in+Healthcare&rft.atitle=&rft.pub=Qrvey%2C+Inc&rft_id=https%3A%2F%2Fqrvey.com%2Ffhir-healthcare-analytics%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-17\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-17\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ajibade, Samuel-Soma M.; Ayaz, Muhammad; Ngo-Hoang, Dai-Long; Tabuena, Almighty C.; Rabbi, Fazle; Tilaye, Getahun; Bassey, Mbiatke Anthony (25 June 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9815272\/\" target=\"_blank\">\"Analysis of Improved Evolutionary Algorithms Using Students\u2019 Datasets\"<\/a>. <i>2022 IEEE International Conference on Automatic Control and Intelligent Systems (I2CACIS)<\/i> (Shah Alam, Malaysia: IEEE): 180\u2013185. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FI2CACIS54679.2022.9815272\" target=\"_blank\">10.1109\/I2CACIS54679.2022.9815272<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-6654-9581-3<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9815272\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/9815272\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Analysis+of+Improved+Evolutionary+Algorithms+Using+Students%E2%80%99+Datasets&rft.jtitle=2022+IEEE+International+Conference+on+Automatic+Control+and+Intelligent+Systems+%28I2CACIS%29&rft.aulast=Ajibade&rft.aufirst=Samuel-Soma+M.&rft.au=Ajibade%2C%26%2332%3BSamuel-Soma+M.&rft.au=Ayaz%2C%26%2332%3BMuhammad&rft.au=Ngo-Hoang%2C%26%2332%3BDai-Long&rft.au=Tabuena%2C%26%2332%3BAlmighty+C.&rft.au=Rabbi%2C%26%2332%3BFazle&rft.au=Tilaye%2C%26%2332%3BGetahun&rft.au=Bassey%2C%26%2332%3BMbiatke+Anthony&rft.date=25+June+2022&rft.pages=180%E2%80%93185&rft.place=Shah+Alam%2C+Malaysia&rft.pub=IEEE&rft_id=info:doi\/10.1109%2FI2CACIS54679.2022.9815272&rft.isbn=978-1-6654-9581-3&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F9815272%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-18\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-18\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Rabbi, Fazle; Ayaz, Muhammad; Dayupay, Johnry P.; Oyebode, Oluwadare Joshua; Gido, Nathaniel G.; Adhikari, Nirmal; Tabuena, Almighty C.; Ajibade, Samuel-Soma M. <i>et al.<\/i> (23 July 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9845171\/\" target=\"_blank\">\"Gaussian Map to Improve Firefly Algorithm Performance\"<\/a>. <i>2022 IEEE 13th Control and System Graduate Research Colloquium (ICSGRC)<\/i> (Shah Alam, Malaysia: IEEE): 88\u201392. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FICSGRC55096.2022.9845171\" target=\"_blank\">10.1109\/ICSGRC55096.2022.9845171<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-6654-6806-0<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/9845171\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/9845171\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Gaussian+Map+to+Improve+Firefly+Algorithm+Performance&rft.jtitle=2022+IEEE+13th+Control+and+System+Graduate+Research+Colloquium+%28ICSGRC%29&rft.aulast=Rabbi&rft.aufirst=Fazle&rft.au=Rabbi%2C%26%2332%3BFazle&rft.au=Ayaz%2C%26%2332%3BMuhammad&rft.au=Dayupay%2C%26%2332%3BJohnry+P.&rft.au=Oyebode%2C%26%2332%3BOluwadare+Joshua&rft.au=Gido%2C%26%2332%3BNathaniel+G.&rft.au=Adhikari%2C%26%2332%3BNirmal&rft.au=Tabuena%2C%26%2332%3BAlmighty+C.&rft.au=Ajibade%2C%26%2332%3BSamuel-Soma+M.&rft.au=Bassey%2C%26%2332%3BMbiatke+Anthony&rft.date=23+July+2022&rft.pages=88%E2%80%9392&rft.place=Shah+Alam%2C+Malaysia&rft.pub=IEEE&rft_id=info:doi\/10.1109%2FICSGRC55096.2022.9845171&rft.isbn=978-1-6654-6806-0&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F9845171%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-19\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-19\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ajibade, Samuel-Soma M.; Zaidi, Abdelhamid; Tapales, Catherine P.; Ngo-Hoang, Dai-Long; Ayaz, Muhammad; Dayupay, Johnry P.; Aminu Dodo, Yakubu; Chaudhury, Sushovan <i>et al.<\/i> (17 December 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ieeexplore.ieee.org\/document\/10001810\/\" target=\"_blank\">\"Data Mining Analysis of Online Drug Reviews\"<\/a>. <i>2022 IEEE 10th Conference on Systems, Process & Control (ICSPC)<\/i> (Malacca, Malaysia: IEEE): 247\u2013251. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FICSPC55597.2022.10001810\" target=\"_blank\">10.1109\/ICSPC55597.2022.10001810<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-6654-7098-8<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ieeexplore.ieee.org\/document\/10001810\/\" target=\"_blank\">https:\/\/ieeexplore.ieee.org\/document\/10001810\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Data+Mining+Analysis+of+Online+Drug+Reviews&rft.jtitle=2022+IEEE+10th+Conference+on+Systems%2C+Process+%26+Control+%28ICSPC%29&rft.aulast=Ajibade&rft.aufirst=Samuel-Soma+M.&rft.au=Ajibade%2C%26%2332%3BSamuel-Soma+M.&rft.au=Zaidi%2C%26%2332%3BAbdelhamid&rft.au=Tapales%2C%26%2332%3BCatherine+P.&rft.au=Ngo-Hoang%2C%26%2332%3BDai-Long&rft.au=Ayaz%2C%26%2332%3BMuhammad&rft.au=Dayupay%2C%26%2332%3BJohnry+P.&rft.au=Aminu+Dodo%2C%26%2332%3BYakubu&rft.au=Chaudhury%2C%26%2332%3BSushovan&rft.au=Adediran%2C%26%2332%3BAnthonia+Oluwatosin&rft.date=17+December+2022&rft.pages=247%E2%80%93251&rft.place=Malacca%2C+Malaysia&rft.pub=IEEE&rft_id=info:doi\/10.1109%2FICSPC55597.2022.10001810&rft.isbn=978-1-6654-7098-8&rft_id=https%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F10001810%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:5-20\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:5_20-0\">20.0<\/a><\/sup> <sup><a href=\"#cite_ref-:5_20-1\">20.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Giannangelo, Kathy; Fenton, Susan H. (20 May 2008). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC2396499\/\" target=\"_blank\">\"SNOMED CT Survey: An Assessment of Implementation in EMR\/EHR Applications\"<\/a>. <i>Perspectives in Health Information Management \/ AHIMA, American Health Information Management Association<\/i> <b>5<\/b>: 7. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1559-4122\" target=\"_blank\">1559-4122<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/2396499\/\" target=\"_blank\">2396499<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/18509501\" target=\"_blank\">18509501<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC2396499\/\" target=\"_blank\">https:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC2396499\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=SNOMED+CT+Survey%3A+An+Assessment+of+Implementation+in+EMR%2FEHR+Applications&rft.jtitle=Perspectives+in+Health+Information+Management+%2F+AHIMA%2C+American+Health+Information+Management+Association&rft.aulast=Giannangelo&rft.aufirst=Kathy&rft.au=Giannangelo%2C%26%2332%3BKathy&rft.au=Fenton%2C%26%2332%3BSusan+H.&rft.date=20+May+2008&rft.volume=5&rft.pages=7&rft.issn=1559-4122&rft_id=info:pmc\/2396499&rft_id=info:pmid\/18509501&rft_id=https%3A%2F%2Fwww.ncbi.nlm.nih.gov%2Fpmc%2Farticles%2FPMC2396499%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-21\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-21\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ayaz, M. (2017). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ijmse.org\/Volume8\/Issue2.html\" target=\"_blank\">\"Cloud Computing Base Electronic Health Record System Architecture for Disabled Children\"<\/a>. <i>International Journal of Multidisciplinary Sciences and Engineering<\/i> <b>8<\/b> (2): 24\u201328<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.ijmse.org\/Volume8\/Issue2.html\" target=\"_blank\">http:\/\/www.ijmse.org\/Volume8\/Issue2.html<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Cloud+Computing+Base+Electronic+Health+Record+System+Architecture+for+Disabled+Children&rft.jtitle=International+Journal+of+Multidisciplinary+Sciences+and+Engineering&rft.aulast=Ayaz%2C+M.&rft.au=Ayaz%2C+M.&rft.date=2017&rft.volume=8&rft.issue=2&rft.pages=24%E2%80%9328&rft_id=http%3A%2F%2Fwww.ijmse.org%2FVolume8%2FIssue2.html&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-22\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-22\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Bresnick, J. (8 May 2017). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/healthitanalytics.com\/news\/48-of-businesses-including-healthcare-face-big-data-skills-gap\" target=\"_blank\">\"48% of Businesses, Including Healthcare, Face Big Data Skills Gap\"<\/a>. <i>Health IT Analytics<\/i>. TechTarget<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/healthitanalytics.com\/news\/48-of-businesses-including-healthcare-face-big-data-skills-gap\" target=\"_blank\">https:\/\/healthitanalytics.com\/news\/48-of-businesses-including-healthcare-face-big-data-skills-gap<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 06 December 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=48%25+of+Businesses%2C+Including+Healthcare%2C+Face+Big+Data+Skills+Gap&rft.atitle=Health+IT+Analytics&rft.aulast=Bresnick%2C+J.&rft.au=Bresnick%2C+J.&rft.date=8+May+2017&rft.pub=TechTarget&rft_id=https%3A%2F%2Fhealthitanalytics.com%2Fnews%2F48-of-businesses-including-healthcare-face-big-data-skills-gap&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-23\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-23\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ayaz, M. (2017). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ijmse.org\/Volume8\/Issue3.html\" target=\"_blank\">\"A Novel Model of Software Process Improvements for Small and Medium Scale Enterprises by using the Big Data Analytics Approach\"<\/a>. <i>International Journal of Multidisciplinary Sciences and Engineering<\/i> <b>8<\/b> (3): 1\u201310<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.ijmse.org\/Volume8\/Issue3.html\" target=\"_blank\">http:\/\/www.ijmse.org\/Volume8\/Issue3.html<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Novel+Model+of+Software+Process+Improvements+for+Small+and+Medium+Scale+Enterprises+by+using+the+Big+Data+Analytics+Approach&rft.jtitle=International+Journal+of+Multidisciplinary+Sciences+and+Engineering&rft.aulast=Ayaz%2C+M.&rft.au=Ayaz%2C+M.&rft.date=2017&rft.volume=8&rft.issue=3&rft.pages=1%E2%80%9310&rft_id=http%3A%2F%2Fwww.ijmse.org%2FVolume8%2FIssue3.html&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-24\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-24\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ayaz. M. (2017). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ijmse.org\/Volume8\/Issue2.html\" target=\"_blank\">\"A Seminal Hybrid Business Process Management Model\"<\/a>. <i>International Journal of Multidisciplinary Sciences and Engineering<\/i> <b>8<\/b> (2): 38\u201342<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.ijmse.org\/Volume8\/Issue2.html\" target=\"_blank\">http:\/\/www.ijmse.org\/Volume8\/Issue2.html<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Seminal+Hybrid+Business+Process+Management+Model&rft.jtitle=International+Journal+of+Multidisciplinary+Sciences+and+Engineering&rft.aulast=Ayaz.+M.&rft.au=Ayaz.+M.&rft.date=2017&rft.volume=8&rft.issue=2&rft.pages=38%E2%80%9342&rft_id=http%3A%2F%2Fwww.ijmse.org%2FVolume8%2FIssue2.html&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:6-25\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:6_25-0\">25.0<\/a><\/sup> <sup><a href=\"#cite_ref-:6_25-1\">25.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Hong, Na; Prodduturi, Naresh; Wang, Chen; Jiang, Guoqian (2017). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/978-1-61499-830-3-868\" target=\"_blank\">\"Shiny FHIR: An Integrated Framework Leveraging Shiny R and HL7 FHIR to Empower Standards-Based Clinical Data Applications\"<\/a>. <i>MEDINFO 2017: Precision Healthcare through Informatics<\/i>: 868\u2013872. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3233%2F978-1-61499-830-3-868\" target=\"_blank\">10.3233\/978-1-61499-830-3-868<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/978-1-61499-830-3-868\" target=\"_blank\">https:\/\/ebooks.iospress.nl\/doi\/10.3233\/978-1-61499-830-3-868<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Shiny+FHIR%3A+An+Integrated+Framework+Leveraging+Shiny+R+and+HL7+FHIR+to+Empower+Standards-Based+Clinical+Data+Applications&rft.jtitle=MEDINFO+2017%3A+Precision+Healthcare+through+Informatics&rft.aulast=Hong&rft.aufirst=Na&rft.au=Hong%2C%26%2332%3BNa&rft.au=Prodduturi%2C%26%2332%3BNaresh&rft.au=Wang%2C%26%2332%3BChen&rft.au=Jiang%2C%26%2332%3BGuoqian&rft.date=2017&rft.pages=868%E2%80%93872&rft_id=info:doi\/10.3233%2F978-1-61499-830-3-868&rft_id=https%3A%2F%2Febooks.iospress.nl%2Fdoi%2F10.3233%2F978-1-61499-830-3-868&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-26\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-26\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ayaz, Muhammad; Pasha, Muhammad Fermi; Le, Tham Yu; Alahmadi, Tahani Jaser; Abdullah, Nik Nailah Binti; Alhababi, Zaid Ali (30 January 2023). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.mdpi.com\/2227-9032\/11\/3\/390\" target=\"_blank\">\"A Framework for Automatic Clustering of EHR Messages Using a Spatial Clustering Approach\"<\/a> (in en). <i>Healthcare<\/i> <b>11<\/b> (3): 390. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3390%2Fhealthcare11030390\" target=\"_blank\">10.3390\/healthcare11030390<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2227-9032\" target=\"_blank\">2227-9032<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC9914110\/\" target=\"_blank\">PMC9914110<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/36766965\" target=\"_blank\">36766965<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.mdpi.com\/2227-9032\/11\/3\/390\" target=\"_blank\">https:\/\/www.mdpi.com\/2227-9032\/11\/3\/390<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Framework+for+Automatic+Clustering+of+EHR+Messages+Using+a+Spatial+Clustering+Approach&rft.jtitle=Healthcare&rft.aulast=Ayaz&rft.aufirst=Muhammad&rft.au=Ayaz%2C%26%2332%3BMuhammad&rft.au=Pasha%2C%26%2332%3BMuhammad+Fermi&rft.au=Le%2C%26%2332%3BTham+Yu&rft.au=Alahmadi%2C%26%2332%3BTahani+Jaser&rft.au=Abdullah%2C%26%2332%3BNik+Nailah+Binti&rft.au=Alhababi%2C%26%2332%3BZaid+Ali&rft.date=30+January+2023&rft.volume=11&rft.issue=3&rft.pages=390&rft_id=info:doi\/10.3390%2Fhealthcare11030390&rft.issn=2227-9032&rft_id=info:pmc\/PMC9914110&rft_id=info:pmid\/36766965&rft_id=https%3A%2F%2Fwww.mdpi.com%2F2227-9032%2F11%2F3%2F390&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-27\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-27\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation book\">Shortliffe, Edward Hance; Cimino, James J.; Chiang, Michael F., eds. (2021). <i>Biomedical Informatics: Computer applications in health care and biomedicine<\/i> (5th edition ed.). Cham, Switzerland: Springer. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-3-030-58720-8.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=Biomedical+Informatics%3A+Computer+applications+in+health+care+and+biomedicine&rft.date=2021&rft.edition=5th+edition&rft.place=Cham%2C+Switzerland&rft.pub=Springer&rft.isbn=978-3-030-58720-8&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-28\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-28\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation book\">Reddy, Chandan K.; Aggarwal, Charu C., eds. (23 June 2015) (in en). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.taylorfrancis.com\/books\/9781482232127\" target=\"_blank\"><i>Healthcare Data Analytics<\/i><\/a> (0 ed.). Chapman and Hall\/CRC. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1201%2Fb18588\" target=\"_blank\">10.1201\/b18588<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-4822-3212-7<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.taylorfrancis.com\/books\/9781482232127\" target=\"_blank\">https:\/\/www.taylorfrancis.com\/books\/9781482232127<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=Healthcare+Data+Analytics&rft.date=23+June+2015&rft.edition=0&rft.pub=Chapman+and+Hall%2FCRC&rft_id=info:doi\/10.1201%2Fb18588&rft.isbn=978-1-4822-3212-7&rft_id=https%3A%2F%2Fwww.taylorfrancis.com%2Fbooks%2F9781482232127&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-29\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-29\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Hripcsak, George; Duke, Jon D.; Shah, Nigam H.; Reich, Christian G.; Huser, Vojtech; Schuemie, Martijn J.; Suchard, Marc A.; Park, Rae Woong <i>et al.<\/i> (2015). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/978-1-61499-564-7-574\" target=\"_blank\">\"Observational Health Data Sciences and Informatics (OHDSI): Opportunities for Observational Researchers\"<\/a>. <i>MEDINFO 2015: eHealth-enabled Health<\/i>: 574\u2013578. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3233%2F978-1-61499-564-7-574\" target=\"_blank\">10.3233\/978-1-61499-564-7-574<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/978-1-61499-564-7-574\" target=\"_blank\">https:\/\/ebooks.iospress.nl\/doi\/10.3233\/978-1-61499-564-7-574<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Observational+Health+Data+Sciences+and+Informatics+%28OHDSI%29%3A+Opportunities+for+Observational+Researchers&rft.jtitle=MEDINFO+2015%3A+eHealth-enabled+Health&rft.aulast=Hripcsak&rft.aufirst=George&rft.au=Hripcsak%2C%26%2332%3BGeorge&rft.au=Duke%2C%26%2332%3BJon+D.&rft.au=Shah%2C%26%2332%3BNigam+H.&rft.au=Reich%2C%26%2332%3BChristian+G.&rft.au=Huser%2C%26%2332%3BVojtech&rft.au=Schuemie%2C%26%2332%3BMartijn+J.&rft.au=Suchard%2C%26%2332%3BMarc+A.&rft.au=Park%2C%26%2332%3BRae+Woong&rft.au=Wong%2C%26%2332%3BIan+Chi+Kei&rft.date=2015&rft.pages=574%E2%80%93578&rft_id=info:doi\/10.3233%2F978-1-61499-564-7-574&rft_id=https%3A%2F%2Febooks.iospress.nl%2Fdoi%2F10.3233%2F978-1-61499-564-7-574&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-30\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-30\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Hripcsak, George; Duke, Jon D.; Shah, Nigam H.; Reich, Christian G.; Huser, Vojtech; Schuemie, Martijn J.; Suchard, Marc A.; Park, Rae Woong <i>et al.<\/i> (2015). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/978-1-61499-564-7-574\" target=\"_blank\">\"Observational Health Data Sciences and Informatics (OHDSI): Opportunities for Observational Researchers\"<\/a>. <i>MEDINFO 2015: eHealth-enabled Health<\/i>: 574\u2013578. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3233%2F978-1-61499-564-7-574\" target=\"_blank\">10.3233\/978-1-61499-564-7-574<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/978-1-61499-564-7-574\" target=\"_blank\">https:\/\/ebooks.iospress.nl\/doi\/10.3233\/978-1-61499-564-7-574<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Observational+Health+Data+Sciences+and+Informatics+%28OHDSI%29%3A+Opportunities+for+Observational+Researchers&rft.jtitle=MEDINFO+2015%3A+eHealth-enabled+Health&rft.aulast=Hripcsak&rft.aufirst=George&rft.au=Hripcsak%2C%26%2332%3BGeorge&rft.au=Duke%2C%26%2332%3BJon+D.&rft.au=Shah%2C%26%2332%3BNigam+H.&rft.au=Reich%2C%26%2332%3BChristian+G.&rft.au=Huser%2C%26%2332%3BVojtech&rft.au=Schuemie%2C%26%2332%3BMartijn+J.&rft.au=Suchard%2C%26%2332%3BMarc+A.&rft.au=Park%2C%26%2332%3BRae+Woong&rft.au=Wong%2C%26%2332%3BIan+Chi+Kei&rft.date=2015&rft.pages=574%E2%80%93578&rft_id=info:doi\/10.3233%2F978-1-61499-564-7-574&rft_id=https%3A%2F%2Febooks.iospress.nl%2Fdoi%2F10.3233%2F978-1-61499-564-7-574&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-31\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-31\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/github.com\/OHDSI\/Atlas\/wiki\" target=\"_blank\">\"ATLAS - A unified interface for the OHDSI tools\"<\/a>. <i>GitHub<\/i>. 30 May 2019<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/github.com\/OHDSI\/Atlas\/wiki\" target=\"_blank\">https:\/\/github.com\/OHDSI\/Atlas\/wiki<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=ATLAS+-+A+unified+interface+for+the+OHDSI+tools&rft.atitle=GitHub&rft.date=30+May+2019&rft_id=https%3A%2F%2Fgithub.com%2FOHDSI%2FAtlas%2Fwiki&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-32\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-32\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ma, Chunlan; Frankel, Heath; Beale, Thomas; Heard, Sam (2007). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/17911747\" target=\"_blank\">\"EHR query language (EQL)--a query language for archetype-based health records\"<\/a>. <i>Studies in Health Technology and Informatics<\/i> <b>129<\/b> (Pt 1): 397\u2013401. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0926-9630\" target=\"_blank\">0926-9630<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/17911747\" target=\"_blank\">17911747<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/17911747\" target=\"_blank\">https:\/\/pubmed.ncbi.nlm.nih.gov\/17911747<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=EHR+query+language+%28EQL%29--a+query+language+for+archetype-based+health+records&rft.jtitle=Studies+in+Health+Technology+and+Informatics&rft.aulast=Ma&rft.aufirst=Chunlan&rft.au=Ma%2C%26%2332%3BChunlan&rft.au=Frankel%2C%26%2332%3BHeath&rft.au=Beale%2C%26%2332%3BThomas&rft.au=Heard%2C%26%2332%3BSam&rft.date=2007&rft.volume=129&rft.issue=Pt+1&rft.pages=397%E2%80%93401&rft.issn=0926-9630&rft_id=info:pmid\/17911747&rft_id=https%3A%2F%2Fpubmed.ncbi.nlm.nih.gov%2F17911747&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-33\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-33\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">The openEHR Foundation (4 February 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/specifications.openehr.org\/releases\/QUERY\/latest\/AQL.html\" target=\"_blank\">\"openEHR - Archetype Query Language (AQL)\"<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/specifications.openehr.org\/releases\/QUERY\/latest\/AQL.html\" target=\"_blank\">https:\/\/specifications.openehr.org\/releases\/QUERY\/latest\/AQL.html<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 10 August 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=openEHR+-+Archetype+Query+Language+%28AQL%29&rft.atitle=&rft.aulast=The+openEHR+Foundation&rft.au=The+openEHR+Foundation&rft.date=4+February+2021&rft_id=https%3A%2F%2Fspecifications.openehr.org%2Freleases%2FQUERY%2Flatest%2FAQL.html&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-34\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-34\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Karim, Md Rezaul; Nguyen, Binh-Phi; Zimmermann, Lukas; Kirsten, Toralf; L\u00f6be, Matthias; Meineke, Frank; Stenzhorn, Holger; Kohlbacher, Oliver <i>et al.<\/i> (2018) (in en). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/publikationen.uni-tuebingen.de\/xmlui\/handle\/10900\/86681\" target=\"_blank\"><i>A Distributed Analytics Platform to Execute FHIR based Phenotyping Algorithms<\/i><\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.15496%2Fpublikation-28068\" target=\"_blank\">10.15496\/publikation-28068<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/publikationen.uni-tuebingen.de\/xmlui\/handle\/10900\/86681\" target=\"_blank\">https:\/\/publikationen.uni-tuebingen.de\/xmlui\/handle\/10900\/86681<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=A+Distributed+Analytics+Platform+to+Execute+FHIR+based+Phenotyping+Algorithms&rft.aulast=Karim&rft.aufirst=Md+Rezaul&rft.au=Karim%2C%26%2332%3BMd+Rezaul&rft.au=Nguyen%2C%26%2332%3BBinh-Phi&rft.au=Zimmermann%2C%26%2332%3BLukas&rft.au=Kirsten%2C%26%2332%3BToralf&rft.au=L%C3%B6be%2C%26%2332%3BMatthias&rft.au=Meineke%2C%26%2332%3BFrank&rft.au=Stenzhorn%2C%26%2332%3BHolger&rft.au=Kohlbacher%2C%26%2332%3BOliver&rft.au=Decker%2C%26%2332%3BStefan&rft.date=2018&rft_id=info:doi\/10.15496%2Fpublikation-28068&rft_id=https%3A%2F%2Fpublikationen.uni-tuebingen.de%2Fxmlui%2Fhandle%2F10900%2F86681&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-35\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-35\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Lakshman, V.; Amrollahi, F.; Koppisetty, V.S. et al. (2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/par.nsf.gov\/servlets\/purl\/10084140\" target=\"_blank\">\"DeepAISE on FHIR\u2014An Interoperable Real-Time Predictive Analytic Platform for Early Prediction of Sepsis\"<\/a>. <i>Proceedings of the AMIA Annual Symposium<\/i><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/par.nsf.gov\/servlets\/purl\/10084140\" target=\"_blank\">https:\/\/par.nsf.gov\/servlets\/purl\/10084140<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 12 December 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=DeepAISE+on+FHIR%E2%80%94An+Interoperable+Real-Time+Predictive+Analytic+Platform+for+Early+Prediction+of+Sepsis&rft.atitle=Proceedings+of+the+AMIA+Annual+Symposium&rft.aulast=Lakshman%2C+V.%3B+Amrollahi%2C+F.%3B+Koppisetty%2C+V.S.+et+al.&rft.au=Lakshman%2C+V.%3B+Amrollahi%2C+F.%3B+Koppisetty%2C+V.S.+et+al.&rft.date=2018&rft_id=https%3A%2F%2Fpar.nsf.gov%2Fservlets%2Fpurl%2F10084140&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-36\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-36\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Khalilia, Mohammed; Choi, Myung; Henderson, Amelia; Iyengar, Sneha; Braunstein, Mark; Sun, Jimeng (2015). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/26958207\" target=\"_blank\">\"Clinical Predictive Modeling Development and Deployment through FHIR Web Services\"<\/a>. <i>AMIA ... Annual Symposium proceedings. AMIA Symposium<\/i> <b>2015<\/b>: 717\u2013726. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1942-597X\" target=\"_blank\">1942-597X<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/4765683\/\" target=\"_blank\">4765683<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/26958207\" target=\"_blank\">26958207<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/26958207\" target=\"_blank\">https:\/\/pubmed.ncbi.nlm.nih.gov\/26958207<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Clinical+Predictive+Modeling+Development+and+Deployment+through+FHIR+Web+Services&rft.jtitle=AMIA+...+Annual+Symposium+proceedings.+AMIA+Symposium&rft.aulast=Khalilia&rft.aufirst=Mohammed&rft.au=Khalilia%2C%26%2332%3BMohammed&rft.au=Choi%2C%26%2332%3BMyung&rft.au=Henderson%2C%26%2332%3BAmelia&rft.au=Iyengar%2C%26%2332%3BSneha&rft.au=Braunstein%2C%26%2332%3BMark&rft.au=Sun%2C%26%2332%3BJimeng&rft.date=2015&rft.volume=2015&rft.pages=717%E2%80%93726&rft.issn=1942-597X&rft_id=info:pmc\/4765683&rft_id=info:pmid\/26958207&rft_id=https%3A%2F%2Fpubmed.ncbi.nlm.nih.gov%2F26958207&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-37\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-37\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">doc.ai (10 May 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/medium.com\/@_doc_ai\/doc-ai-is-on-fire-oops-we-mean-fhir-ea2912b2864b\" target=\"_blank\">\"doc.ai is on fire. Oops we mean FHIR:)\"<\/a>. <i>Medium<\/i><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/medium.com\/@_doc_ai\/doc-ai-is-on-fire-oops-we-mean-fhir-ea2912b2864b\" target=\"_blank\">https:\/\/medium.com\/@_doc_ai\/doc-ai-is-on-fire-oops-we-mean-fhir-ea2912b2864b<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 30 January 2023<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=doc.ai+is+on+fire.+Oops+we+mean+FHIR%3A%29&rft.atitle=Medium&rft.aulast=doc.ai&rft.au=doc.ai&rft.date=10+May+2018&rft_id=https%3A%2F%2Fmedium.com%2F%40_doc_ai%2Fdoc-ai-is-on-fire-oops-we-mean-fhir-ea2912b2864b&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-38\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-38\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Semler, Sebastian; Wissing, Frank; Heyder, Ralf (1 July 2018). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.3414\/ME18-03-0003\" target=\"_blank\">\"German Medical Informatics Initiative: A National Approach to Integrating Health Data from Patient Care and Medical Research\"<\/a> (in en). <i>Methods of Information in Medicine<\/i> <b>57<\/b> (S 01): e50\u2013e56. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3414%2FME18-03-0003\" target=\"_blank\">10.3414\/ME18-03-0003<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0026-1270\" target=\"_blank\">0026-1270<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC6178199\/\" target=\"_blank\">PMC6178199<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/30016818\" target=\"_blank\">30016818<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.3414\/ME18-03-0003\" target=\"_blank\">http:\/\/www.thieme-connect.de\/DOI\/DOI?10.3414\/ME18-03-0003<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=German+Medical+Informatics+Initiative%3A+A+National+Approach+to+Integrating+Health+Data+from+Patient+Care+and+Medical+Research&rft.jtitle=Methods+of+Information+in+Medicine&rft.aulast=Semler&rft.aufirst=Sebastian&rft.au=Semler%2C%26%2332%3BSebastian&rft.au=Wissing%2C%26%2332%3BFrank&rft.au=Heyder%2C%26%2332%3BRalf&rft.date=1+July+2018&rft.volume=57&rft.issue=S+01&rft.pages=e50%E2%80%93e56&rft_id=info:doi\/10.3414%2FME18-03-0003&rft.issn=0026-1270&rft_id=info:pmc\/PMC6178199&rft_id=info:pmid\/30016818&rft_id=http%3A%2F%2Fwww.thieme-connect.de%2FDOI%2FDOI%3F10.3414%2FME18-03-0003&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-39\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-39\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Kreuzthaler, Markus; Mart&#237; nez-Costa, Catalina; Kaiser, Peter; Schulz, Stefan (2017). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/978-1-61499-759-7-24\" target=\"_blank\">\"Semantic Technologies for Re-Use of Clinical Routine Data\"<\/a>. <i>Health Informatics Meets eHealth<\/i>: 24\u201331. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3233%2F978-1-61499-759-7-24\" target=\"_blank\">10.3233\/978-1-61499-759-7-24<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/978-1-61499-759-7-24\" target=\"_blank\">https:\/\/ebooks.iospress.nl\/doi\/10.3233\/978-1-61499-759-7-24<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Semantic+Technologies+for+Re-Use+of+Clinical+Routine+Data&rft.jtitle=Health+Informatics+Meets+eHealth&rft.aulast=Kreuzthaler&rft.aufirst=Markus&rft.au=Kreuzthaler%2C%26%2332%3BMarkus&rft.au=Mart%26%23237&rft.au=nez-Costa%2C%26%2332%3BCatalina&rft.au=Kaiser%2C%26%2332%3BPeter&rft.au=Schulz%2C%26%2332%3BStefan&rft.date=2017&rft.pages=24%E2%80%9331&rft_id=info:doi\/10.3233%2F978-1-61499-759-7-24&rft_id=https%3A%2F%2Febooks.iospress.nl%2Fdoi%2F10.3233%2F978-1-61499-759-7-24&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-40\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-40\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Franz, Barbara (2015). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.ejbi.org\/scholarly-articles\/applying-fhir-in-an-integrated-health-monitoring-system.pdf\" target=\"_blank\">\"Applying FHIR in an Integrated Health Monitoring System\"<\/a>. <i>European Journal for Biomedical Informatics<\/i> <b>11<\/b> (02). <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.24105%2Fejbi.2015.11.2.8\" target=\"_blank\">10.24105\/ejbi.2015.11.2.8<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.ejbi.org\/scholarly-articles\/applying-fhir-in-an-integrated-health-monitoring-system.pdf\" target=\"_blank\">https:\/\/www.ejbi.org\/scholarly-articles\/applying-fhir-in-an-integrated-health-monitoring-system.pdf<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Applying+FHIR+in+an+Integrated+Health+Monitoring+System&rft.jtitle=European+Journal+for+Biomedical+Informatics&rft.aulast=Franz&rft.aufirst=Barbara&rft.au=Franz%2C%26%2332%3BBarbara&rft.date=2015&rft.volume=11&rft.issue=02&rft_id=info:doi\/10.24105%2Fejbi.2015.11.2.8&rft_id=https%3A%2F%2Fwww.ejbi.org%2Fscholarly-articles%2Fapplying-fhir-in-an-integrated-health-monitoring-system.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-41\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-41\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Liu, Dianbo; Sahu, Ricky; Ignatov, Vlad; Gottlieb, Dan; Mandl, Kenneth D. (4 March 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC7153160\/\" target=\"_blank\">\"High Performance Computing on Flat FHIR Files Created with the New SMART\/HL7 Bulk Data Access Standard\"<\/a>. <i>AMIA Annual Symposium Proceedings<\/i> <b>2019<\/b>: 592\u2013596. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1942-597X\" target=\"_blank\">1942-597X<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/7153160\/\" target=\"_blank\">7153160<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/32308853\" target=\"_blank\">32308853<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC7153160\/\" target=\"_blank\">https:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC7153160\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=High+Performance+Computing+on+Flat+FHIR+Files+Created+with+the+New+SMART%2FHL7+Bulk+Data+Access+Standard&rft.jtitle=AMIA+Annual+Symposium+Proceedings&rft.aulast=Liu&rft.aufirst=Dianbo&rft.au=Liu%2C%26%2332%3BDianbo&rft.au=Sahu%2C%26%2332%3BRicky&rft.au=Ignatov%2C%26%2332%3BVlad&rft.au=Gottlieb%2C%26%2332%3BDan&rft.au=Mandl%2C%26%2332%3BKenneth+D.&rft.date=4+March+2020&rft.volume=2019&rft.pages=592%E2%80%93596&rft.issn=1942-597X&rft_id=info:pmc\/7153160&rft_id=info:pmid\/32308853&rft_id=https%3A%2F%2Fwww.ncbi.nlm.nih.gov%2Fpmc%2Farticles%2FPMC7153160%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-42\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-42\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/parquet.apache.org\/\" target=\"_blank\">\"Apache Parquet\"<\/a>. Google<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/parquet.apache.org\/\" target=\"_blank\">https:\/\/parquet.apache.org\/<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 10 August 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Apache+Parquet&rft.atitle=&rft.pub=Google&rft_id=https%3A%2F%2Fparquet.apache.org%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-43\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-43\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Dunn, Tim; Cosgun, Erdal (5 January 2023). Arighi, Cecilia. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/academic.oup.com\/bioinformaticsadvances\/article\/doi\/10.1093\/bioadv\/vbac095\/6994207\" target=\"_blank\">\"A cloud-based pipeline for analysis of FHIR and long-read data\"<\/a> (in en). <i>Bioinformatics Advances<\/i> <b>3<\/b> (1): vbac095. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1093%2Fbioadv%2Fvbac095\" target=\"_blank\">10.1093\/bioadv\/vbac095<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2635-0041\" target=\"_blank\">2635-0041<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC9872570\/\" target=\"_blank\">PMC9872570<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/36726729\" target=\"_blank\">36726729<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/academic.oup.com\/bioinformaticsadvances\/article\/doi\/10.1093\/bioadv\/vbac095\/6994207\" target=\"_blank\">https:\/\/academic.oup.com\/bioinformaticsadvances\/article\/doi\/10.1093\/bioadv\/vbac095\/6994207<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+cloud-based+pipeline+for+analysis+of+FHIR+and+long-read+data&rft.jtitle=Bioinformatics+Advances&rft.aulast=Dunn&rft.aufirst=Tim&rft.au=Dunn%2C%26%2332%3BTim&rft.au=Cosgun%2C%26%2332%3BErdal&rft.date=5+January+2023&rft.volume=3&rft.issue=1&rft.pages=vbac095&rft_id=info:doi\/10.1093%2Fbioadv%2Fvbac095&rft.issn=2635-0041&rft_id=info:pmc\/PMC9872570&rft_id=info:pmid\/36726729&rft_id=https%3A%2F%2Facademic.oup.com%2Fbioinformaticsadvances%2Farticle%2Fdoi%2F10.1093%2Fbioadv%2Fvbac095%2F6994207&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-44\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-44\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Gruendner, Julian; Gulden, Christian; Kampf, Marvin; Mate, Sebastian; Prokosch, Hans-Ulrich; Zierk, Jakob (1 April 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/medinform.jmir.org\/2021\/4\/e25645\" target=\"_blank\">\"A Framework for Criteria-Based Selection and Processing of Fast Healthcare Interoperability Resources (FHIR) Data for Statistical Analysis: Design and Implementation Study\"<\/a> (in en). <i>JMIR Medical Informatics<\/i> <b>9<\/b> (4): e25645. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.2196%2F25645\" target=\"_blank\">10.2196\/25645<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2291-9694\" target=\"_blank\">2291-9694<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC8050750\/\" target=\"_blank\">PMC8050750<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/33792554\" target=\"_blank\">33792554<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/medinform.jmir.org\/2021\/4\/e25645\" target=\"_blank\">https:\/\/medinform.jmir.org\/2021\/4\/e25645<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Framework+for+Criteria-Based+Selection+and+Processing+of+Fast+Healthcare+Interoperability+Resources+%28FHIR%29+Data+for+Statistical+Analysis%3A+Design+and+Implementation+Study&rft.jtitle=JMIR+Medical+Informatics&rft.aulast=Gruendner&rft.aufirst=Julian&rft.au=Gruendner%2C%26%2332%3BJulian&rft.au=Gulden%2C%26%2332%3BChristian&rft.au=Kampf%2C%26%2332%3BMarvin&rft.au=Mate%2C%26%2332%3BSebastian&rft.au=Prokosch%2C%26%2332%3BHans-Ulrich&rft.au=Zierk%2C%26%2332%3BJakob&rft.date=1+April+2021&rft.volume=9&rft.issue=4&rft.pages=e25645&rft_id=info:doi\/10.2196%2F25645&rft.issn=2291-9694&rft_id=info:pmc\/PMC8050750&rft_id=info:pmid\/33792554&rft_id=https%3A%2F%2Fmedinform.jmir.org%2F2021%2F4%2Fe25645&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-45\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-45\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/github.com\/cerner\/bunsen\" target=\"_blank\">\"cerner \/ bunsen\"<\/a>. <i>GitHub<\/i>. 20 November 2020<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/github.com\/cerner\/bunsen\" target=\"_blank\">https:\/\/github.com\/cerner\/bunsen<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 10 August 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=cerner+%2F+bunsen&rft.atitle=GitHub&rft.date=20+November+2020&rft_id=https%3A%2F%2Fgithub.com%2Fcerner%2Fbunsen&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-46\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-46\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Zaharia, M.; Chowdhury, M.; Frankling, M.J. et al. (2010). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www1.icsi.berkeley.edu\/pubs\/networking\/ICSI_sparkclustercomputing10.pdf\" target=\"_blank\">\"Spark: Cluster Computing with Working Sets\"<\/a> (PDF). pp. 1\u20137<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www1.icsi.berkeley.edu\/pubs\/networking\/ICSI_sparkclustercomputing10.pdf\" target=\"_blank\">https:\/\/www1.icsi.berkeley.edu\/pubs\/networking\/ICSI_sparkclustercomputing10.pdf<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 21 December 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Spark%3A+Cluster+Computing+with+Working+Sets&rft.atitle=&rft.aulast=Zaharia%2C+M.%3B+Chowdhury%2C+M.%3B+Frankling%2C+M.J.+et+al.&rft.au=Zaharia%2C+M.%3B+Chowdhury%2C+M.%3B+Frankling%2C+M.J.+et+al.&rft.date=2010&rft.pages=pp.+1%E2%80%937&rft_id=https%3A%2F%2Fwww1.icsi.berkeley.edu%2Fpubs%2Fnetworking%2FICSI_sparkclustercomputing10.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-47\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-47\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Brush, R.; Mandel, J. (2023). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/github.com\/FHIR\/sql-on-fhir\" target=\"_blank\">\"SQL on FHIR\"<\/a>. <i>GitHub<\/i><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/github.com\/FHIR\/sql-on-fhir\" target=\"_blank\">https:\/\/github.com\/FHIR\/sql-on-fhir<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 10 August 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=SQL+on+FHIR&rft.atitle=GitHub&rft.aulast=Brush%2C+R.%3B+Mandel%2C+J.&rft.au=Brush%2C+R.%3B+Mandel%2C+J.&rft.date=2023&rft_id=https%3A%2F%2Fgithub.com%2FFHIR%2Fsql-on-fhir&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-48\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-48\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/protobuf.dev\/\" target=\"_blank\">\"Protocol Buffers\"<\/a>. <i>Protocol Buffers Documentation<\/i>. Google, LLC. 2022<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/protobuf.dev\/\" target=\"_blank\">https:\/\/protobuf.dev\/<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 10 August 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Protocol+Buffers&rft.atitle=Protocol+Buffers+Documentation&rft.date=2022&rft.pub=Google%2C+LLC&rft_id=https%3A%2F%2Fprotobuf.dev%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-49\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-49\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/github.com\/google\/fhir\" target=\"_blank\">\"google \/ fhir\"<\/a>. <i>GitHub<\/i>. 2022<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/github.com\/google\/fhir\" target=\"_blank\">https:\/\/github.com\/google\/fhir<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 10 August 2022<\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=google+%2F+fhir&rft.atitle=GitHub&rft.date=2022&rft_id=https%3A%2F%2Fgithub.com%2Fgoogle%2Ffhir&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-50\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-50\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Chong, Dazhi; Shi, Hui (3 July 2015). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.tandfonline.com\/doi\/full\/10.1080\/23270012.2015.1082449\" target=\"_blank\">\"Big data analytics: a literature review\"<\/a> (in en). <i>Journal of Management Analytics<\/i> <b>2<\/b> (3): 175\u2013201. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1080%2F23270012.2015.1082449\" target=\"_blank\">10.1080\/23270012.2015.1082449<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2327-0012\" target=\"_blank\">2327-0012<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.tandfonline.com\/doi\/full\/10.1080\/23270012.2015.1082449\" target=\"_blank\">http:\/\/www.tandfonline.com\/doi\/full\/10.1080\/23270012.2015.1082449<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Big+data+analytics%3A+a+literature+review&rft.jtitle=Journal+of+Management+Analytics&rft.aulast=Chong&rft.aufirst=Dazhi&rft.au=Chong%2C%26%2332%3BDazhi&rft.au=Shi%2C%26%2332%3BHui&rft.date=3+July+2015&rft.volume=2&rft.issue=3&rft.pages=175%E2%80%93201&rft_id=info:doi\/10.1080%2F23270012.2015.1082449&rft.issn=2327-0012&rft_id=http%3A%2F%2Fwww.tandfonline.com%2Fdoi%2Ffull%2F10.1080%2F23270012.2015.1082449&rfr_id=info:sid\/en.wikipedia.org:Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<\/ol><\/div><\/div>\n<h2><span class=\"mw-headline\" id=\"Notes\">Notes<\/span><\/h2>\n<p>This presentation is faithful to the original, with only a few minor changes to presentation, though grammar and word usage was substantially updated for improved readability. In some cases important information was missing from the references, and that information was added. In the original, citations three and four are identical; for this version, those citations were combined, making the total citation count one less than the original 50. Numerous cited URLs from the original were broken; suitable archived versions were found for this version. In some cases, a suitable archived version could not be found.\n<\/p>\n<!-- \nNewPP limit report\nCached time: 20231215112220\nCache expiry: 86400\nDynamic content: false\nComplications: []\nCPU time usage: 1.567 seconds\nReal time usage: 2.443 seconds\nPreprocessor visited node count: 45611\/1000000\nPost\u2010expand include size: 369540\/2097152 bytes\nTemplate argument size: 126827\/2097152 bytes\nHighest expansion depth: 25\/40\nExpensive parser function count: 0\/100\nUnstrip recursion depth: 0\/20\nUnstrip post\u2010expand size: 105815\/5000000 bytes\nLua time usage: 0.070\/7 seconds\nLua virtual size: 5332992\/52428800 bytes\nLua estimated memory usage: 0 bytes\n-->\n<!--\nTransclusion expansion time report (%,ms,calls,template)\n100.00% 1201.456 1 -total\n 71.57% 859.861 1 Template:Reflist\n 55.48% 666.513 50 Template:Citation\/core\n 37.06% 445.256 25 Template:Cite_journal\n 17.83% 214.268 1 Template:Dead_link\n 17.61% 211.600 20 Template:Cite_web\n 17.25% 207.268 1 Template:Fix\n 15.44% 185.536 1 Template:Category_handler\n 9.00% 108.161 44 Template:Date\n 5.61% 67.461 59 Template:Citation\/identifier\n-->\n\n<!-- Saved in parser cache with key limswiki:pcache:idhash:14349-0!canonical and timestamp 20231215112218 and revision id 52794. Serialized with JSON.\n -->\n<\/div><\/div><div class=\"printfooter\">Source: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.limswiki.org\/index.php\/Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data\">https:\/\/www.limswiki.org\/index.php\/Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data<\/a><\/div>\n<!-- end content --><div class=\"visualClear\"><\/div><\/div><\/div><div class=\"visualClear\"><\/div><\/div><!-- end of the left (by default at least) column --><div class=\"visualClear\"><\/div><\/div>\n\n\n\n<\/body>","96ca1abdbcc7bf60389fe942b678382d_images":["https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/0\/04\/Fig1_Ayaz_Healthcare23_11-12.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/1\/1f\/Fig2_Ayaz_Healthcare23_11-12.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/a\/ab\/Fig3_Ayaz_Healthcare23_11-12.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/f\/f9\/Fig4_Ayaz_Healthcare23_11-12.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/1\/1f\/Fig5_Ayaz_Healthcare23_11-12.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/5\/5e\/Fig6_Ayaz_Healthcare23_11-12.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/d\/df\/Fig7_Ayaz_Healthcare23_11-12.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/9\/98\/Fig8_Ayaz_Healthcare23_11-12.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/1\/1d\/Fig9_Ayaz_Healthcare23_11-12.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/a\/ac\/Fig10_Ayaz_Healthcare23_11-12.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/9\/91\/Fig11_Ayaz_Healthcare23_11-12.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/2\/25\/Fig12_Ayaz_Healthcare23_11-12.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/7\/74\/Fig13_Ayaz_Healthcare23_11-12.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/f\/fa\/Fig14_Ayaz_Healthcare23_11-12.png","https:\/\/s3.limswiki.org\/www.limswiki.org\/images\/e\/ef\/Fig15_Ayaz_Healthcare23_11-12.png"],"96ca1abdbcc7bf60389fe942b678382d_timestamp":1702682171,"34fb9a0e0648cc8855dc5ea15c3c3a92_type":"article","34fb9a0e0648cc8855dc5ea15c3c3a92_title":"Cadmium bioconcentration and translocation potential in day-neutral and photoperiod-sensitive hemp grown hydroponically for the medicinal market (Marebesi et al. 2023)","34fb9a0e0648cc8855dc5ea15c3c3a92_url":"https:\/\/www.cannaqa.wiki\/index.php?title=Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market","34fb9a0e0648cc8855dc5ea15c3c3a92_plaintext":"\n\nJournal:Cadmium bioconcentration and translocation potential in day-neutral and photoperiod-sensitive hemp grown hydroponically for the medicinal marketFrom CannaQAWikiJump to navigationJump to searchFull article title\n \nCadmium bioconcentration and translocation potential in day-neutral and photoperiod-sensitive hemp grown hydroponically for the medicinal marketJournal\n \nWaterAuthor(s)\n \nMarebesi, Amando O.; Lessl, Jason T.; Coolong, Timothy W.Author affiliation(s)\n \nUniversity of GeorgiaPrimary contact\n \nEmail: aom at uga dot eduYear published\n \n2023Volume and issue\n \n15(12)Article #\n \n2176DOI\n \n10.3390\/w15122176ISSN\n \n2073-4441Distribution license\n \nCreative Commons Attribution 4.0 InternationalWebsite\n \nhttps:\/\/www.mdpi.com\/2073-4441\/15\/12\/2176Download\n \nhttps:\/\/www.mdpi.com\/2073-4441\/15\/12\/2176\/pdf?version=1686282553 (PDF)\n\nContents \n\n1 Abstract \n2 Introduction \n3 Materials and methods \n\n3.1 Experimental settings \n3.2 Mineral analysis \n3.3 Plant growth and biomass yield \n3.4 Cannabinoid analysis \n3.5 Statistical analysis \n\n\n4 Results and discussion \n\n4.1 Plant height and biomass yield \n4.2 Cd concentration in hemp tissues \n4.3 Nutrient partitioning \n4.4 Total THC and CBD in hemp flower \n\n\n5 Conclusions \n6 Supplementary materials \n7 Abbreviations, acronyms, and initialisms \n8 Acknowledgements \n\n8.1 Author contributions \n8.2 Funding \n8.3 Data availability \n8.4 Conflicts of interest \n\n\n9 References \n10 Notes \n\n\n\nAbstract \nHeavy metal contamination of agricultural soils is potentially concerning when growing crops for human consumption. Industrial hemp (Cannabis sativa L.) has been reported to tolerate the presence of heavy metals such as cadmium (Cd) in the soil. Therefore, the objectives of this study were to evaluate Cd uptake and translocation in two day-length-sensitive (DLS) and two day-neutral (DN) hemp varieties grown for the medicinal market and to determine the impact of Cd exposure on cannabinoid concentrations in flowers. A hydroponic experiment was conducted by exposing plants to 0 mg\u00b7L\u22121 Cd and 2.5 mg\u00b7L\u22121 Cd in the nutrient solution. Cadmium concentrations ranged from 16.1 to 2274.2 mg\u00b7kg\u22121 in roots, though all four varieties accumulated significant concentrations of Cd in aboveground tissues, with translocation factors ranging from 6.5 to 193. Whole-plant bioconcentration factors ranged from 20 to 1051 mg\u00b7kg\u22121. Cannabinoid concentrations were negatively impacted by Cd exposure in DN varieties but were unaffected in DLS varieties. Biomass was reduced by Cd exposure demonstrating that these varieties might not be suitable for growth on contaminated soil or for phytoremediation. There is potential for Cd accumulation in flowers, showing the need for heavy metal testing of C. sativa consumer products.\nKeywords: auto-flower, bioconcentration factor, cannabinoid, heavy metal, hemp, metal stress, translocation factor\n\nIntroduction \nFor centuries, humans have cultivated hemp (Cannabis sativa L.) for its fiber, seed, therapeutic, and psychoactive properties. During the domestication process, wild Cannabis populations have been subject to selection, giving rise to the multiple varieties that exist today.[1] The term \"industrial hemp\" is commonly used to refer to C. sativa plants[2] with total tetrahydrocannabinol (THC) concentrations below 0.3%. Plants with total THC concentrations above 0.3% are classified as marijuana and subject to federal prohibition in the United States (U.S.)[3] Current industrial hemp breeding efforts target fiber, seed, or cannabinoid production, based on the end user. Hemp varieties with high cannabidiol (CBD) concentrations are often selected for medicinal and therapeutic uses. There is also a focus on developing varieties that are day-neutral (DN) or minimally sensitive to photoperiod in order to expand production opportunities.[4]\nHemp is generally considered a qualitative short-day plant that flowers in response to decreasing photoperiods. Hemp selections that flower in response to photoperiod are known as day-length-sensitive (DLS). After emergence, hemp undergoes a photoperiod-dependent vegetative phase maintained by exposure to approximately 14\u201318 hours or more of light daily.[5][6] When hemp is planted during periods of short days (<13 hours of light), it may flower prematurely. Premature flowering, prior to complete vegetative development, can result in yield reductions.[7][8] In contrast, some hemp varieties exhibit DN flowering tendencies known colloquially as \u201cauto-flower\u201d hemp. These DN varieties are relatively insensitive to photoperiod for flower induction. The DN trait is speculated to arise from Cannabis ruderalis (C. sativa ssp. ruderalis) and may have originated from hemp located in high latitudes where photoperiods can be long and growing seasons are typically short or regions with relatively short and constant photoperiods.[5][9][10] Advantages of DN hemp varieties include the ability to flower in regions that have little variation in photoperiod throughout the year (tropics) or during times of the year when photoperiods may be inadequate to grow DLS varieties. However, many DN types of hemp have been reported to be particularly sensitive to environmental stressors such as high temperatures and may have lower yields than comparable DLS varieties.[11]\nIn addition to uses for fiber, seed, and medicinal purposes, hemp has also been proposed as a candidate for use in phytoremediation, which utilizes plants to remove contaminants, such as heavy metals or other chemicals from soils.[12][13] Accumulator plant species can uptake heavy metals from soils, even at low external concentrations, and concentrate them in plant tissues.[14] By growing accumulator plants in contaminated soil, it is possible to realize in situ decontamination, an economically viable approach that preserves physicochemical soil characteristics, while removing contaminants.[15] The morphophysiological characteristics of hemp, such as high biomass production, deep roots, and short life cycle, make it a potential candidate for phytoremediation.[16][17][18][19][20]\nHeavy metal contamination of agricultural soils is a concern when growing crops for food or medicinal purposes, due to potential harm to human and animal health.[21][22][23] Cadmium (Cd) contamination in the environment has been linked to anthropogenic activities, such as mining and smelting. Further, Cd can be introduced to soils via contaminated manure, sewage sludge, and phosphate fertilizers.[24] Cadmium is known to cause health issues when ingested in amounts greater than the provisional tolerable monthly intake (PTMI) of 25 \u03bcg\u00b7kg\u22121 of body weight.[25] In previous studies utilizing naturally and artificially contaminated soil and substrate containing from 0 to up to 200 mg\u00b7kg\u22121 Cd, hemp varieties grown for fiber production accumulated Cd in aboveground tissues at levels that could be harmful to human health.[26][27][28][29] For instance, the hemp fiber variety Silistrinski grown in naturally contaminated soil containing 12.2 mg\u00b7kg\u22121 Cd accumulated 1.22 mg\u00b7kg\u22121 Cd in its flowers.[27]\nThere are multiple indicators that can be used to determine the accumulation potential of a plant species. Bioconcentration factor (BCF) is the ratio between the metal concentration in plant tissues and the initial metal concentration in the soil or growing solution.[20][30][31][32][33] This indicator has also been used interchangeably with terms such as accumulation factor (AF)[28], biological absorption coefficient (BAC) or index of bioaccumulation (IBA).[23] A separate indicator of accumulation potential is the translocation factor (TF), which is the ratio between the metal concentration in the above ground biomass and the metal concentration in the roots.[31][32] Additionally, plant growth parameters can be assessed to determine the tolerance index (TI), calculated as the ratio between growth in contaminated and non-contaminated soils.[20][30] There is significant variability in BCF among plant species and chemical elements. It has been proposed that plants with BCF >100 mg\u00b7kg\u22121 Cd on a dry weight (DW) basis in its leaves could be referred to as hyperaccumulators.[14] Conversely, Chaney and Baklanov[12] suggested that true hyperaccumulators are able to accumulate higher concentrations of metals in leaves than in roots (TF > 1).\nFew studies have evaluated heavy metal accumulation in hemp flowers, with most research utilizing fiber hemp varieties to determine heavy metal uptake for phytoremediation purposes.[16][17][19][34] Due to the harmful effects of Cd and other heavy metals on human health, the U.S. hemp industry has attempted to implement standards regarding maximum allowable levels of metals in C. sativa consumer products, which vary by state.[35] As hemp flowers are increasingly grown for the medicinal market, determining Cd distribution among plant organs, as well as bioconcentration and root-to-shoot translocation factors, are of importance. We hypothesize that there are distinctions in Cd accumulation and distribution among plant tissues in hemp varieties with different growth and flowering habits. Therefore, the objectives of this study were to evaluate nutrient partitioning and Cd uptake, translocation, and accumulation in DLS and DN hemp varieties, and to determine the impact of Cd exposure to cannabinoids in plant flowers.\n\nMaterials and methods \nExperimental settings \nThe experiment was conducted in a greenhouse in Watkinsville, GA, USA (lat. 33\u00b05\u2032 N, long. 83\u00b03\u2032 W) from January 2022 to April 2022. Feminized seed from two DLS hemp varieties, T1 and Von (Sunbelt Hemp Source, Moultrie, GA, USA), and two DN varieties Apricot Auto (Blue Forest Farms, New York, NY, USA) and Auto CBD Alpha Explorer (Alpha Explorer) (Phylos Bioscience, Portland, OR, USA) were sown into engineered foam cubes (3.33 cm L \u00d7 2.54 cm W \u00d7 3.81 cm D; Oasis Grower Solutions, Kent, OH, USA) for germination. Foam cubes were placed in plastic trays over a germination mat set at 24 \u00b0C exposed to a mist irrigation system, which applied water twice daily for one minute each. Supplemental lighting (approximately 100 \u00b5mol\u00b7m\u22122\u00b7s\u22121) was used during germination. Seedlings were maintained under these conditions for four weeks, after which they were placed into plastic netted containers (4.7 cm W \u00d7 5.1 cm D) and transferred to 37.9 L plastic containers (Rubbermaid Inc. Wooster, OH, USA) filled with 28 L of well water. The well water was analyzed for nutrient concentrations periodically throughout the experiment (Table 1). Four seedlings per replicate were placed equidistantly (24.3 cm apart) in holes drilled in the container lid. Welded wire mesh frames were attached to each lid to support plants. A 15.2 cm aquarium air stone attached to an air pump (Active Aqua; Hydrofarm, Petaluma, CA, USA) was placed inside the container to aerate the nutrient solution throughout the experiment. Container volume was maintained by adding well water every two to three days. At transplant, a nutrient solution was added to the plastic containers using a half-strength Hoagland\u2019s solution[36] (Table 1).\n\n\n\n\n\n\n\nTable 1. Mineral nutrient concentrations in the nutrient solution used in this study. Notes: Values shown in mg\u00b7L\u22121. ND = not detected. i Nutrient solution comprised from the following compounds: Ca(NO3)2\u22194H2O, KNO3, KH2PO4, MgSO4\u22197H2O, H3BO3, MnCl2\u22194H2O, ZnSO4\u22197H2O, CuSO4\u22195H2O, H2MoO4\u2219H2O, and Sequestrene 330.\n\n\n\n\nN\n\nP\n\nK\n\nCa\n\nMg\n\nB\n\nCu\n\nMo\n\nFe\n\nMn\n\nZn\n\n\nWell water\n\nND\n\n<0.01\n\n2.7\n\n12.1\n\n2.1\n\n<0.01\n\n<0.05\n\n<0.01\n\n<0.1\n\n<0.1\n\n<0.1\n\n\nNutrient solutioni\n\n105\n\n15.5\n\n117\n\n100\n\n24.3\n\n0.3\n\n0.01\n\n0.005\n\n0.5\n\n0.25\n\n0.025\n\n\nTotal concentration\n\n105\n\n15.5\n\n119.7\n\n112.1\n\n26.4\n\n0.3\n\n<0.05\n\n<0.01\n\n0.5\n\n0.25\n\n<0.1\n\n\n\nPlants were grown for three weeks in the base nutrient solution after which nutrient solutions were replaced completely and Cd treatments were added using CdSO4\u00b78H2O, to achieve 0 (control) and 2.5 mg\u00b7L\u22121 Cd. Cadmium concentrations were chosen based on the results of previous studies[34][35], which evaluated hemp exposure to Cd in hydroponic systems. The experimental treatments were arranged in a randomized complete block design, with four hemp varieties exposed to two levels of Cd with four replicates each. Nutrient solutions were maintained to a constant volume by adding water every two to three days, and nutrients were replaced every three weeks for the remainder of the experiment. The electrical conductivity (EC) and pH of the solutions were measured weekly. Solution pH was adjusted to 5.5 when necessary (pH down; General Hydroponics, Santa Rosa, CA, USA). Supplemental light (approximately 100 \u00b5mol\u00b7m\u22122\u00b7s\u22121) was used to provide 18\/6 light\/dark hours for four weeks of vegetative growth after transplanting seedlings into containers. Supplemental lights were turned off to allow for flower induction in the DLS varieties for the remaining seven weeks of production (average day length 12 hours and 38 minutes). The DN varieties (Apricot Auto and Auto CBD Alpha Explorer) exhibited visually detectable flower development one week prior to the induction of flowering in the DLS varieties (Von and T1). Therefore, the DN varieties were harvested one week prior to the DLS varieties to ensure that plants flowered for the same length of time.\nNutrient solutions were sampled at the beginning and end of each three-week cycle using 20 mL scintillation vials (HDPE; Thermo-Fisher Scientific\u2122, Waltham, MS, USA), and stored at \u22124 \u00b0C until the analysis of mineral nutrient concentrations (Table S1). Temperature and relative humidity (RH) of the greenhouse were monitored at canopy height hourly (VP4; Meter Group Inc., Pullman WA, USA) and averaged 19.1 \u00b1 3.1 \u00b0C and 74 \u00b1 0.1% RH for the experiment. Photosynthetic active radiation was also monitored hourly throughout the experiment (QSO-S; Meter Group Inc.) and plants were exposed to an average daily light integral (DLI) average of 21.6 \u00b1 9.0 mol\u00b7m\u22122\u00b7d\u22121.\n\nMineral analysis \nSamples of fresh root, stem, leaf, and flower tissues were collected for Cd analysis at harvest. Composite samples (50 g fresh material) were taken from each of the four plants in a replicate (container). Roots were triple-washed with deionized water after removal. Ten of the youngest fully expanded leaves were collected from the top one-third of each plant (main stem and lateral branches) and rinsed with deionized water. Stem samples were collected from the bottom two-thirds of the main stem. Flower material was sampled from the top of the main stem and the top one-third of plants. Samples were placed in a forced air oven set at 55 \u00b0C for 72 hours until a constant weight was achieved. Dried plant material was then ground in a Wiley mill (Thomas Scientific, Swedesboro, NJ, USA) and passed through a 20-mesh screen. Samples were digested using EPA Method 3052.[37] In brief, 0.5 gram samples were placed in fluorocarbon polymer microwave vessels, 10 mL of concentrated nitric acid were added to each vessel which was then sealed. The microwave digester (Mars 6 Microwave; CEM Corp., Matthews, NC, USA) was heated to 200 \u00b0C for 30 minutes and digested (solutions) were then transferred quantitatively into volumetric flasks and brought to 100 mL volume with deionized water prior to analysis.\nSamples of the hydroponic solutions were filtered using a 0.45 \u00b5M PTFE membrane (Thermo-Fisher Scientific\u2122 Choice\u2122 Polypropylene Syringe Filters) and acidified using 2% (v\/v) high purity nitric acid (HNO3) (Certified ACS Plus, Fisher Scientific, Pittsburgh, PA, USA) prior to analysis. Hydroponic solutions and plant tissues were analyzed for multiple elements\u2014phosphorous (P), potassium (K), sulfur (S), calcium (Ca), magnesium (Mg), iron (Fe), manganese (Mn), aluminum (Al), boron (B), copper (Cu), zinc (Zn), nickel (Ni), and cadmium (Cd)\u2014following EPA Method 200.8[38] by inductively coupled plasma\u2013optical emission spectroscopy (ICP-OES) (Spectro Arcos FHS16; Spectro Ametek USA, Wilmington, MA, USA). The instrument parameter settings and wavelengths used are displayed in the supplementary material (Tables S3 and S4, and Figure S1). The instrument reporting limit for Cd was <0.005 mg\u00b7L\u22121. Results were expressed as mg\u00b7L\u22121. Calibration standards utilized in this analysis were from a certified source (Inorganic Ventures, Christiansburg, VA, lot number: N2-MEB667614). Independent laboratory performance checks were also run with acceptable deviations for recoveries set at 100 \u00b1 5.0%.\nThe BCF was calculated by dividing the Cd concentration in plant tissues by the initial Cd concentration in the nutrient solution..[20][30][31][32][33] The TF (%) was calculated by dividing the sum of Cd concentration in leaves, flowers, and stems by the Cd concentration in roots, and multiplying by 100.[31][32] Cadmium uptake rates (\u00b5mol\u00b7plant\u22121\u00b7d\u22121) were calculated by the following equation (adapted from Ali et al.[30]): ((([Cdinitial \u2212 Cdfinal])\/number of plants)\/treatment days)\/root biomass)\n\nPlant growth and biomass yield \nPlant height was determined by measuring the distance from the base of the stem to the tip of the apically dominant flower at harvest.[11]\nTo quantify leaf, flower, stem, and root biomass, four whole plants per replicate were air-dried at ambient temperatures inside the greenhouse for two weeks and then separated into roots, stems, and leaf and flower biomass. Dry leaf and flower materials were manually pulled from plants following industry standards used for hemp biomass intended in cannabinoid extraction. Subsamples were taken from the air-dried materials and further dried in a forced air oven set at 55 \u00b0C for 48 hours until a constant weight was achieved. The dry weights of the whole plant samples were then normalized based on subsample moisture content.[11]\n\nCannabinoid analysis \nApproximately 25 g of fresh flower tissue sampled from inflorescences located on the top one-third of the plants were sampled during weeks six (DN varieties) and seven (DLS varieties) of flowering (49 and 56 days after treatment [DAT], respectively) and dried separately from other samples as follows. Flower material was placed on a perforated aluminum baking sheet and dried to approximately 15% moisture content in a walk-in cooler with a temperature set point of 13 \u00b0C and 55% relative humidity for 14 days. The appropriate relative humidity was maintained using a dehumidifier. The dried material was hand trimmed to remove leaves, sealed in a metalized resealable food bag (Uline, Braselton, GA, USA) and stored at \u22124 \u00b0C for cannabinoid analysis. The acidic and neutral forms of the cannabinoids, THC and CBD, were determined in dried flower material by a commercial laboratory using high-performance liquid chromatography and a diode array detector set to 230 nm (SJ Labs and Analytics, Macon, GA, USA). The limit of detection for THC and CBD was 0.02%. Total cannabinoid concentrations were calculated by the following formula: total cannabinoid = neutral + (acidic form \u00d7 0.877). The percentage of dry matter for all samples was recorded and the results were reported on a dry weight basis.\n\nStatistical analysis \nStatistical analysis was conducted using JMP Pro 15 (SAS, Cary, NC, USA). Data were subjected to a one-way ANOVA procedure with Student\u2019s t-test (p < 0.05) or Tukey\u2019s Honest Significant Difference test (p < 0.05) conducted for mean separation when appropriate. Tissue Cd and cannabinoid concentrations were log-transformed to ensure equal variance prior to statistical analysis. Non-transformed data are presented.\n\nResults and discussion \nPlant height and biomass yield \nPlant heights were significantly reduced by Cd exposure in DN varieties, Apricot Auto, and Alpha Explorer. Plants exposed to 2.5 mg\u00b7L\u22121 Cd were shorter than plants in the control (0 mg\u00b7L\u22121 Cd) treatment (Figure 1). Plant heights of the two DLS varieties, Von and T1, were not significantly affected by Cd treatment. Nevertheless, the average decline in plant height of plants exposed to Cd was 20.4% in the DLS varieties and 35.6% in the DN varieties. Exposure to Cd also significantly reduced flower and leaf, stem, and root biomass in the DN varieties, Apricot Auto, and Alpha Explorer (Table 2). In the DLS variety Von, the dry weight of flower and leaf tissues significantly decreased in the 2.5 mg\u00b7L\u22121 Cd treatment compared to the control, while dry weights of stem and root tissues were unaffected by Cd. The dry weights of the flower and leaf, and root tissue were significantly reduced by exposure to Cd in the DLS variety T1. Whole plant biomass in all four varieties was significantly reduced by Cd treatments, with an average reduction of 74.4% in the DN varieties and 50.2% in the DLS varieties.\n\r\n\n\n\n\n\n\n\n\n\n\nFig. 1. Average plant height at harvest \u00b1 SE of four hemp (Cannabis sativa L.) varieties. Values correspond to averages of four replicates with four plants each. Bars associated with the same uppercase letter(s) indicate no significant differences among hemp varieties at p \u2264 0.05 according to Tukey\u2019s HSD test. Bars associated with the same lowercase letter indicate no significant differences between control and treated plants for a given hemp variety at p \u2264 0.05 according to Student\u2019s t-test.\n\n\n\n\n\n\n\n\n\nTable 2. Dry weight of flowers + leaves, stems, roots, and whole plant at harvest \u00b1 SE of four hemp (Cannabis sativa L.) varieties exposed to 0 (control) and 2.5 (treated) mg\u00b7L\u22121 Cd in a nutrient solution. Notes: Biomass values measured as g\u00b7Plant\u22121. Values followed by the same uppercase letter(s) indicate no significant differences among hemp varieties for each plant tissue according to Tukey\u2019s Honest Significant Difference test (p < 0.05). Values followed by the same lowercase letter within each individual hemp variety row indicate no significant differences between control and Cd treated plants for each plant tissue according to Student\u2019s t-test (p < 0.05).\n\n\n\n\nFlower + leaf\n\nStems\n\nRoots\n\nWhole plant\n\n\nVariety\n\nControl\n\nTreated\n\nControl\n\nTreated\n\nControl\n\nTreated\n\nControl\n\nTreated\n\n\nApricot Auto\n\n40.0 \u00b1 2.3 AB a\n\n9.9 \u00b1 2.0 B b\n\n17.7 \u00b1 1.2 A a\n\n6.5 \u00b1 1.0 A b\n\n8.3 \u00b1 0.3 B a\n\n3.2 \u00b1 0.4 B b\n\n66.0 \u00b1 3.2 B a\n\n19.5 \u00b1 2.7 AB b\n\n\nAlpha Explorer\n\n37.8 \u00b1 2.2 B a\n\n8.2 \u00b1 1.4 B b\n\n23.6 \u00b1 1.5 A a\n\n4.5 \u00b1 0.8 A b\n\n7.1 \u00b1 0.7 B a\n\n2.1 \u00b1 0.4 B b\n\n68.4 \u00b1 3.3 B a\n\n14.8 \u00b1 2.4 B b\n\n\nVon\n\n46.2 \u00b1 1.3 AB a\n\n22.2 \u00b1 7.2 AB b\n\n29.3 \u00b1 1.7 A a\n\n15.2 \u00b1 5.6 A a\n\n20.1 \u00b1 1.0 A a\n\n13.3 \u00b1 2.9 A a\n\n95.6 \u00b1 1.5 A a\n\n50.7 \u00b1 15.3 A b\n\n\nT1\n\n48.6 \u00b1 2.4 A a\n\n28.5 \u00b1 5.6 A b\n\n27.7 \u00b1 6.5 A a\n\n13.9 \u00b1 2.7 A a\n\n22.9 \u00b1 2.0 A a\n\n9.7 \u00b1 2.7 AB b\n\n99.2 \u00b1 6.4 A a\n\n52.2 \u00b1 9.6 A b\n\n\n\nThe DLS varieties, Von and T1, generated significantly greater root biomass relative to the DN varieties when not exposed to Cd. In Cd-treated plants, root biomass was significantly greater in Von compared to the two DN varieties. Root biomass in T1 was not significantly different from any other variety when exposed to Cd. Flower and leaf tissue yields were highest in DLS variety T1 in both control and Cd-exposed treatments. Exposure to Cd resulted in a significant decrease in flower and leaf biomass of 75% and 78%, in Apricot Auto and Alpha Explorer DN varieties, respectively. Exposure to Cd significantly reduced flower and leaf biomass in the DLS varieties Von and T1 by 52% and 41%, respectively. Whole-plant, stem, and root biomass was also significantly reduced to a greater extent in DN varieties compared to the DLS varieties. This suggests that the DLS varieties used in the present study may be more tolerant to Cd exposure at 2.5 mg\u00b7L\u22121 than the DN varieties evaluated. These results agree with previous studies that have reported a decrease in shoot biomass of hemp plants exposed to Cd.[26][34][35][39]\n\nCd concentration in hemp tissues \nCd concentrations in hemp tissues were affected by plant variety and Cd treatment (Table 3). In DN varieties, Apricot Auto and Alpha Explorer, and the DLS variety Von, Cd concentrations were highest in roots. In contrast, the DLS variety, T1 had similar Cd concentrations in roots, leaf, and stem tissue upon exposure to Cd. These results are consistent with previous literature, which reported that roots were the preferred tissue for Cd accumulation in hemp[27][34][35][40][41] and that Cd accumulation increased with increasing Cd concentrations in the growing media.[39] For instance, Cd concentrations in the roots of C. sativa fiber variety Santhica 27 exposed to 20 \u03bcM Cd (2.25 mg\u00b7L\u22121 Cd) for one week averaged 2,687 mg\u00b7kg\u22121 dw, while Cd concentrations in stems and leaves averaged 1,243 mg\u00b7kg\u22121 dry weight (dw) and 717 mg\u00b7kg\u22121 dw, respectively.[34] Additionally, the C. sativa medicinal variety Purple Tiger exposed to 2.5 mg\u00b7L\u22121 Cd for 68 days had average Cd concentrations of 1,982.6 mg\u00b7kg\u22121 dw in roots, 13.2 mg\u00b7kg\u22121 dw in leaves, 5.1 mg\u00b7kg\u22121 dw in stems, and 7.6 mg\u00b7kg\u22121 dw in flowers.[35]\n\n\n\n\n\n\n\nTable 3. Average cadmium (Cd) concentration at harvest in plant tissues of four hemp (Cannabis sativa L.) varieties exposed to 0 (control) and 2.5 (treated) mg\u00b7L\u22121 Cd in a nutrient solution on a dry weight (dw) basis. Notes: Cd concentration values measured as mg\u00b7kg\u22121 dw. ND = not detected. Values followed by the same uppercase letter(s) indicate no significant differences among hemp varieties for each plant tissue according to Tukey\u2019s Honest Significant Difference test (p < 0.05). Values followed by the same lowercase letter(s) within a hemp variety row indicate no significant differences among plant tissues for each treatment (control and treated) according to Tukey\u2019s Honest Significant Difference test (p < 0.05).\n\n\n\n\nFlower\n\nRoot\n\nLeaf\n\nStem\n\n\nVariety\n\nControl\n\nTreated\n\nControl\n\nTreated\n\nControl\n\nTreated\n\nControl\n\nTreated\n\n\nApricot Auto\n\n1.2 A ab\n\n38.1 A c\n\n3.8 A a\n\n1056.8 A a\n\n1.0 A b\n\n44.9 AB c\n\n0.5 A c\n\n116.3 A b\n\n\nAlpha Explorer\n\n0.6 A ab\n\n51.0 A d\n\n3.4 A a\n\n2274.2 A a\n\n3.0 A a\n\n92.5 A c\n\n0.3 A b\n\n176.4 A b\n\n\nVon\n\n0.4 A a\n\n11.9 B b\n\n1.1 A a\n\n512.4 A a\n\n2.1 A a\n\n18.2 BC b\n\nND A b\n\n1.8 C c\n\n\nT1\n\nND B a\n\n0.2 C b\n\nND B a\n\n16.1 B a\n\n0.8 A a\n\n8.22 C a\n\n0.6 A a\n\n26.8 B a\n\n\n\nAll varieties had increased concentrations of Cd in all plant tissues exposed to Cd in the nutrient solution, except in the floral tissues of T1. Cadmium-treated DN varieties, Apricot Auto and Alpha Explorer, had increased Cd concentrations in flower and stem tissues, and Alpha Explorer had significantly greater Cd concentrations in leaf tissue compared to the DLS varieties. These results suggest that Cd concentration in plant tissues is negatively correlated to biomass accumulation, as the DN varieties had a higher reduction in flower and leaf biomass when exposed to 2.5 mg\u00b7L\u22121 Cd, compared to the DLS varieties. It should be noted that detectable Cd levels were observed in the control tissues. We expected all the control tissues to be below detection and can only speculate on the source of the Cd. It could have been slight cross contamination due to the aeration system that connected all the tanks. Another possibility is the Cd was already in the plant tissues before the treatments were applied. We analyzed the propagation foam cubes and found some Cd in the material (0.25 mg\u00b7L\u22121), which potentially could have slightly contaminated the tanks.\nWhole-plant BCF ranged from 1,051.8 in Alpha Explorer to 20.9 in T1 (Table 4 and Figure S2). In Alpha Explorer, whole-plant, root, and leaf BCF were significantly greater than the other three varieties evaluated. The BCF in stems and flowers was significantly higher in the DN varieties, Alpha Explorer and Apricot Auto, when compared to the DLS varieties, Von and T1. In Alpha Explorer, Apricot Auto, and Von varieties, BCF values were the highest in roots, while in T1 plants BCF was the highest in stems. For all varieties, the BCF in floral tissue was significantly less than that of roots, except for T1, which accumulated most Cd in stem tissues instead of roots. Previous studies reported that BCF values were consistently higher in roots when compared to above-ground biomass.[39][41] Furthermore, in a review of the capacity of different varieties of C. sativa to accumulate heavy metals, root BCF values in plants exposed to different Cd treatments ranged from 0.08 to 30.99.[17] Most prior studies utilized contaminated soil or a soil-like substrate, which makes it challenging to determine the exact concentration of Cd that was available to plants. Nevertheless, our results support the previous literature demonstrating that plant genetics play a role in the Cd tolerance and accumulation potential of hemp plants.\n\n\n\n\n\n\n\nTable 4. Translocation factor (TF) and bioconcentration factor (BCF) in whole plants, roots, leaves, stems, and flowers of the hemp (Cannabis sativa L.) varieties Apricot Auto, Alpha Explorer, Von, and T1 exposed to 2.5 mg\u00b7L\u22121 Cd in a nutrient solution. Notes: Among the BCF values, those values followed by the same uppercase letter(s) within a column indicate no significant differences among hemp varieties for a plant tissue according to Tukey\u2019s Honest Significant Difference test (p < 0.05). Values followed by the same lowercase letter within each individual hemp variety row indicate no significant differences among plant tissues (root, leaf, stem, and flower) according to Tukey\u2019s Honest Significant Difference test (p < 0.05).\n\n\n\n\nBCF\n\nTF (%)\n\n\nVariety\n\nWhole plant\n\nRoot\n\nLeaf\n\nStem\n\nFlower\n\nWhole plant\n\n\nApricot Auto\n\n527.5 B\n\n446.1 B a\n\n18.3 B b\n\n47.6 AB b\n\n15.5 A b\n\n28.5 B\n\n\nAlpha Explorer\n\n1,051.8 A\n\n920.1 A a\n\n37.9 A b\n\n73.2 A b\n\n20.6 A b\n\n14.3 B\n\n\nVon\n\n213.8 BC\n\n201.4 BC a\n\n7.0 BC b\n\n0.7 C b\n\n4.8 B b\n\n6.5 B\n\n\nT1\n\n20.9 C\n\n6.2 C ab\n\n3.4 C b\n\n11.2 BC a\n\n0.1 B b\n\n193.0 A\n\n\n\nThe TF was calculated as the ratio between Cd concentration in aboveground tissue (stems, leaves, and flowers) and Cd concentration in roots. The variety T1 had the highest TF compared to the other three varieties (Table 4). Our results suggest T1 had the highest translocation factor due to increased Cd accumulation in stems, and relatively low accumulation in roots compared to other varieties. Therefore, this DLS variety may favor Cd sequestration in aboveground parts of the plant while the other three varieties accumulated more Cd in the roots. The tolerance index, which was calculated as the ratio between biomass accumulation (whole plant) in the 2.5 mg\u00b7L\u22121 treatment and biomass accumulation in the control treatment, was not significantly different among hemp varieties (data not shown).\nThe cadmium uptake rate was significantly greater in Alpha Explorer compared to the other three hemp varieties (Figure 2). Both BCF and Cd uptake rates were the highest in Alpha Explorer, suggesting that this variety was the most efficient in taking up Cd from the solution, accumulating it primarily in the roots but also translocating it to the shoots. However, Alpha Explorer had significantly lower whole-plant biomass than the DLS varieties, Von and T1, and biomass is considered a more accurate indicator of Cd toxicity than plant height.[39] In a typical field production scenario, DN varieties would generally have a shorter life cycle than the DLS varieties and significant differences in growing degree-day requirements for maturity in DN and DLS hemp varieties have been reported.[11] Although all plants in the present study were harvested at the same time, the increased maturation rate of the DN varieties likely led to increased rates of Cd uptake. Reduced Cd uptake rates in Von and T1 varieties could potentially allow these plants to better cope with Cd exposure.\n\r\n\n\n\n\n\n\n\n\n\n\nFig. 2. Average Cd uptake rates \u00b1 SE in four hemp (Cannabis sativa L.) varieties exposed to 2.5 mg\u00b7L\u22121 Cd. Bars associated with the same letter(s) indicate no significant differences among hemp varieties at p \u2264 0.05 according to Tukey\u2019s HSD test.\n\n\n\nAlthough the TF was the highest in T1, the other three hemp varieties had significant TF values within or above the range of approximately 2.5 to 12% that has been reported in previous studies[39][41], suggesting that hemp may tolerate Cd stress (Table 4). Furthermore, the TF of all hemp varieties was above 1% and, therefore, these varieties could be classified as high-efficiency plants for metal translocation from roots to above-ground organs.[42] Although BCF was above 100 mg\u00b7kg\u22121 DW in leaves, which is the minimal value for hyperaccumulator plants[14], whole plant biomass was reduced by Cd exposure. Therefore, these hemp varieties might not be candidates to be considered Cd hyperaccumulators as previously indicated.[34] Nevertheless, our data suggest that hemp characteristics related to metal uptake and distribution within plant tissues can fluctuate by variety.\n\nNutrient partitioning \nThe concentrations of the macronutrients N, P, K, and Ca were affected by both plant variety and Cd treatment (Table S2). However, no clear trends for any macronutrients were observed in response to Cd or among varieties. Previous data on nutrient distribution under metal stress is contradictory. For instance, the high-THC C. sativa variety \u201cNB100\u201d growing in uncontaminated substrate supplied with a commercial fertilizer (65, 17, 90 ppm N, P, and K, respectively) had higher N, P, and K concentrations in flower tissue, when compared to inflorescence leaves, fan leaves, and stems (roots were not analyzed), while Ca concentration was higher in fan leaves.[43] Conversely, there was a significant reduction in N, P, K, and Ca content in edible parts of tomato and lettuce grown in spiked soil containing 2.5 mg\u00b7kg\u22121 Cd, when compared to plants grown in non-contaminated soil.[44] Furthermore, chickpea plants (Cicer arietinum L.) exposed to Cd in nutrient solution (approximately 68 mg\u00b7L\u22121 Cd) showed a significant decrease in root and shoot Ca concentrations when compared to plants growing in an uncontaminated nutrient solution.[45] The N, P, and K concentrations in the shoots of Pfaffia glomerata were reported to increase with increasing Cd concentrations in the growing media.[46] These macronutrients are involved in the synthesis of Cd-detoxifying chelator molecules, such as glutathione and phytochelatins, and in the increase in activity of antioxidant enzymes, such as superoxide dismutase (SOD), catalase (CAT), and ascorbate peroxidade (APX).[46][47]\n\nTotal THC and CBD in hemp flower \nTotal CBD concentrations were significantly lower in Apricot Auto and Alpha Explorer plants exposed to 2.5 mg\u00b7L\u22121 Cd compared to the 0 mg\u00b7L\u22121 Cd treatment, while total CBD concentrations in T1 and Von plants were not significantly affected by Cd exposure (Table 5). Similarly, total THC was significantly lower (below the detection limit) in Apricot Auto and Alpha Explorer plants exposed to 2.5 mg\u00b7L\u22121 Cd, compared to the control. Interestingly, THC concentrations in the DLS varieties, Von and T1, were not significantly affected by Cd exposure. Both DN varieties exhibited a significant reduction in total THC and CBD when exposed to Cd, while the DLS varieties did not. In addition, DN varieties had significantly greater Cd uptake rates (Figure 2) and BCF (Figure S2) than DLS plants. Further, both DN varieties had significantly greater Cd accumulation in the floral tissue compared to the DLS varieties (Table 3). This suggests that the increased Cd uptake and accumulation in the floral tissue in Apricot Auto and Alpha Explorer may have led to a decrease in cannabinoid synthesis. A previous study[28] reported no differences in THC content in leaves of the DLS fiber hemp variety \u201cFibranova\u201d grown in substrate contaminated with 25 and 100 \u03bcg\u00b7g\u22121 Cd. Previous results suggest that plant genetics might play a role in cannabinoid synthesis under metal stress.[28][48]\n\n\n\n\n\n\n\nTable 5. Average total CBD and THC \u00b1 SE in the hemp (Cannabis sativa L.) varieties Apricot Auto, Alpha Explorer, Von, and T1 exposed to 0 (control) and 2.5 (treated) mg\u00b7L\u22121 Cd in a nutrient solution. Note: Values of concentration in flowers are measured as percentage of dry weight (% dw). ND = not detected. Values followed by the same uppercase letter(s) within a column indicate no significant differences among hemp varieties according to Tukey\u2019s Honest Significant Difference test (p < 0.05). Values followed by the same lowercase letter within a hemp variety row indicate no significant differences between control and treated plants according to Student\u2019s t-test (p < 0.05). ND = not detected.\n\n\n\n\nConcentration in flowers\n\n\n\n\nTotal THC\n\nTotal CBD\n\n\nVariety\n\nControl\n\nTreated\n\nControl\n\nTreated\n\n\nApricot Auto\n\n0.53 \u00b1 0.05 A a\n\nND B b\n\n9.18 \u00b1 0.69 A a\n\n1.82 \u00b1 0.86 B b\n\n\nAlpha Explorer\n\n0.36 \u00b1 0.01 A a\n\nND B b\n\n8.37 \u00b1 0.30 A a\n\n1.18 \u00b1 0.84 B b\n\n\nVon\n\n0.53 \u00b1 0.07 A a\n\n0.43 \u00b1 0.04 A a\n\n11.38 \u00b1 1.29 A a\n\n9.44 \u00b1 1.31 A a\n\n\nT1\n\n0.43 \u00b1 0.04 A a\n\n0.36 \u00b1 0.05 A a\n\n10.19 \u00b1 0.90 A a\n\n7.64 \u00b1 1.04 A a\n\n\n\nWhile total THC concentrations in all hemp varieties in the control treatment reached concentrations above the legal threshold (0.3%), this is not uncommon in both DLS and DN high-CBD hemp varieties[11][49] as age, genetics, and environmental factors may impact cannabinoid synthesis.[50] These results indicate that the impact of Cd stress on CBD and THC synthesis is variety-dependent, and exposure to 2.5 mg\u00b7L\u22121 Cd may affect cannabinoid synthesis in some varieties of C. sativa.\n\nConclusions \nThe impact of Cd on plant growth as well as BCF, uptake rate, and TF were affected by variety. Whole plant biomass yield in all four varieties was significantly reduced by the Cd treatment, suggesting that the hemps studied here may not be classified as hyperaccumulators as they may accumulate Cd with other ions in a nutrient solution until it becomes toxic. While Cd concentration was significantly higher in roots, all four varieties were efficient in translocating Cd from roots to shoots, with Cd concentrations in flowers ranging from 0.2 to 51 mg\u00b7kg\u22121 Cd DW in T1 and Alpha Explorer varieties, respectively. Flower and leaf biomass were significantly reduced in all four varieties in response to Cd. Further, the DN varieties, Alpha Explorer and Apricot Auto, had a significant decrease in total THC and CBD concentrations in plants exposed to Cd when compared to plants in the control treatment, while the DLS varieties did not. Additional studies are warranted to determine if there are different Cd tolerance mechanisms in DN compared to DLS hemp varieties. All four hemp varieties analyzed in this study are suitable for the medicinal market, given that heavy metal testing is conducted throughout production and on finished consumer products.\n\nSupplementary materials \nThe following supporting information can be downloaded at https:\/\/www.mdpi.com\/article\/10.3390\/w15122176\/s1: \n\nTable S1: Mean concentrations of cadmium (Cd) in nutrient solutions measured at the beginning and end of each cycle and daily uptake. Values are averages \u00b1 SE of four replications per treatment.\nTable S2: Average concentrations of nitrogen (N), phosphorus (P), potassium (K), and calcium (Ca) among different tissues in the hemp (Cannabis sativa L.) varieties Apricot Auto, Alpha Explorer, Von, and T1 on a dry weight (dw) basis exposed to 0 (control) and 2.5 (treated) mg\u00b7L\u22121 Cd in a nutrient solution.\nTable S3: ICP-OES parameters.\nTable S4. ICP-OES wavelengths used.\nFigure S1: Calibration curve of cadmium on ICP-OES.\nFigure S2. Bioconcentration factor (BCF) for Cd \u00b1 SE in plant tissues of four hemp (Cannabis sativa L.) varieties exposed to 2.5 mg\u00b7L\u22121 Cd.\n Abbreviations, acronyms, and initialisms \nAF: accumulation factor\nAl: aluminum\nAPX: ascorbate peroxidade\nB: boron\nBAC: biological absorption coefficient\nBCF: bioconcentration factor\nCa: calcium\nCAT: catalase\nCBD: cannabidiol\nCd: cadmium\nCu: copper\nDAT: days after treatment\nDLS: day-length-sensitive\nDN: day-neutral\nDW: dry weight\nEC: electrical conductivity\nFe: iron\nIBA: index of bioaccumulation\nICP-OES: inductively coupled plasma\u2013optical emission spectroscopy\nK: potassium\nMg: magnesium\nMn: manganese\nN: nitrogen\nNi: nickel\nP: phosphorous\nPTMI: provisional tolerable monthly intake\nRH: relative humidity\nS: sulfur\nSOD: superoxide dismutase\nTF: translocation factor\nTHC: tetrahydrocannabinol\nTI: tolerance index\nZn: zinc\nAcknowledgements \nAuthor contributions \nConceptualization, A.O.M. and T.W.C.; methodology, A.O.M. and T.W.C.; software, A.O.M. and T.W.C.; validation, T.W.C. and J.T.L.; formal analysis, A.O.M. and T.W.C.; investigation, A.O.M. and T.W.C.; resources, T.W.C. and J.T.L.; data curation, A.O.M. and T.W.C.; writing\u2014original draft preparation, A.O.M. and T.W.C.; writing\u2014review and editing, T.W.C. and J.T.L.; visualization, A.O.M. and T.W.C.; supervision, T.W.C.; project administration, T.W.C.; funding acquisition, T.W.C. All authors have read and agreed to the published version of the manuscript.\n\nFunding \nThis research received no external funding.\n\nData availability \nThe data presented in this study are available in the article and supplementary material.\n\nConflicts of interest \nThe authors declare no conflict of interest.\n\nReferences \n\n\n\u2191 Clarke, Robert Connell; Merlin, Mark David (2013). Cannabis: evolution and ethnobotany. Berkeley: University of California Press. pp. 29\u201357. ISBN 978-0-520-27048-0.   \n \n\n\u2191 Cherney, Jerome; Small, Ernest (12 November 2016). \"Industrial Hemp in North America: Production, Politics and Potential\" (in en). Agronomy 6 (4): 58. doi:10.3390\/agronomy6040058. ISSN 2073-4395. http:\/\/www.mdpi.com\/2073-4395\/6\/4\/58 .   \n \n\n\u2191 U.S Department of Agriculture (2019). \"Establishment of A Domestic Hemp Production Program\". Federal Register 84 (211): 58522\u201358564. https:\/\/www.govinfo.gov\/app\/details\/FR-2019-10-31\/2019-23749 .   \n \n\n\u2191 Clarke, Robert C.; Merlin, Mark D. (1 November 2016). \"Cannabis Domestication, Breeding History, Present-day Genetic Diversity, and Future Prospects\" (in en). Critical Reviews in Plant Sciences 35 (5-6): 293\u2013327. doi:10.1080\/07352689.2016.1267498. ISSN 0735-2689. https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/07352689.2016.1267498 .   \n \n\n\u2191 5.0 5.1 Potter, D.J. (2014). \"4. Cannabis Horticulture\". In Pertwee, R. G.. Handbook of cannabis. Handbooks in psychopharmacology (1st ed ed.). Oxford, United Kingdom ; New York, NY: Oxford University Press. pp. 65\u201388. ISBN 978-0-19-966268-5. OCLC 881139510. https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/881139510 .   \n \n\n\u2191 Ranalli, Paolo, ed. (1999). Advances in hemp research. New York: Food Products Press. ISBN 978-1-56022-872-1.   \n \n\n\u2191 Amaducci, Stefano; Colauzzi, Michele; Zatta, Alessandro; Venturi, Gianpietro (23 April 2008). \"Flowering Dynamics in Monoecious and Dioecious Hemp Genotypes\" (in en). Journal of Industrial Hemp 13 (1): 5\u201319. doi:10.1080\/15377880801898691. ISSN 1537-7881. http:\/\/www.tandfonline.com\/doi\/abs\/10.1080\/15377880801898691 .   \n \n\n\u2191 Hall, Jack; Bhattarai, Surya P.; Midmore, David J. (1 January 2012). \"Review of Flowering Control in Industrial Hemp\" (in en). Journal of Natural Fibers 9 (1): 23\u201336. doi:10.1080\/15440478.2012.651848. ISSN 1544-0478. http:\/\/www.tandfonline.com\/doi\/abs\/10.1080\/15440478.2012.651848 .   \n \n\n\u2191 Rosenthal, E. (2010). The Big Book of Buds: More Marijuana Varieties from the World\u2019s Great Seed Breeders. 4. Quick Trading Co.. ISBN 9781936807031.   \n \n\n\u2191 Small, Ernest (2017). Cannabis: a complete guide. Boca Raton: CRC Press Taylor & Francis Group. ISBN 978-1-4987-6163-5.   \n \n\n\u2191 11.0 11.1 11.2 11.3 11.4 Coolong, Timothy; Cassity-Duffey, Kate; Joy, Noelle (1 February 2023). \"Role of Planting Date on Yield and Cannabinoid Content of Day-neutral and Photoperiod-sensitive Hemp in Georgia, USA\". HortTechnology 33 (1): 138\u2013145. doi:10.21273\/HORTTECH05151-22. ISSN 1063-0198. https:\/\/journals.ashs.org\/view\/journals\/horttech\/33\/1\/article-p138.xml .   \n \n\n\u2191 12.0 12.1 Chaney, Rufus L.; Baklanov, Ilya A. (2017), \"Phytoremediation and Phytomining\" (in en), Advances in Botanical Research (Elsevier) 83: 189\u2013221, doi:10.1016\/bs.abr.2016.12.006, ISBN 978-0-12-802853-7, https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0065229616301252 . Retrieved 2023-09-27   \n \n\n\u2191 Nesler, A.; Furini, A. (2012). \"Phytoremediation: The utilization of Plants to Reclaim Polluted Sites\". In Furini, Antonella. Plants and heavy metals. SpringerBriefs in molecular science. Biometals. Dordrecht ; New York: Springer. pp. 75\u201386. ISBN 978-94-007-4440-0. OCLC 795742388. https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/795742388 .   \n \n\n\u2191 14.0 14.1 14.2 Greger, Maria (2004), Prasad, M. N. V., ed., \"Metal Availability, Uptake, Transport and Accumulation in Plants\" (in en), Heavy Metal Stress in Plants (Berlin, Heidelberg: Springer Berlin Heidelberg): 1\u201327, doi:10.1007\/978-3-662-07743-6_1, ISBN 978-3-642-07268-0, http:\/\/link.springer.com\/10.1007\/978-3-662-07743-6_1 . Retrieved 2023-09-27   \n \n\n\u2191 Kirkham, M.B. (1 December 2006). \"Cadmium in plants on polluted soils: Effects of soil factors, hyperaccumulation, and amendments\" (in en). Geoderma 137 (1-2): 19\u201332. doi:10.1016\/j.geoderma.2006.08.024. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0016706106002540 .   \n \n\n\u2191 16.0 16.1 De Vos, B\u00e9atrice; De Souza, Marcella Fernandez; Michels, Evi; Meers, Erik (13 January 2023). \"Industrial hemp (Cannabis sativa L.) field cultivation in a phytoattenuation strategy and valorization potential of the fibers for textile production\" (in en). Environmental Science and Pollution Research 30 (14): 41665\u201341681. doi:10.1007\/s11356-023-25198-z. ISSN 1614-7499. https:\/\/link.springer.com\/10.1007\/s11356-023-25198-z .   \n \n\n\u2191 17.0 17.1 17.2 Golia, Evangelia E.; Bethanis, John; Ntinopoulos, Nikolaos; Kaffe, Georgia-Garifalia; Komnou, Amalia Athanasia; Vasilou, Charicleia (1 April 2023). \"Investigating the potential of heavy metal accumulation from hemp. The use of industrial hemp (Cannabis Sativa L.) for phytoremediation of heavily and moderated polluted soils\" (in en). Sustainable Chemistry and Pharmacy 31: 100961. doi:10.1016\/j.scp.2022.100961. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2352554122003655 .   \n \n\n\u2191 Placido, Dante F.; Lee, Charles C. (23 February 2022). \"Potential of Industrial Hemp for Phytoremediation of Heavy Metals\" (in en). Plants 11 (5): 595. doi:10.3390\/plants11050595. ISSN 2223-7747. PMC PMC8912475. PMID 35270065. https:\/\/www.mdpi.com\/2223-7747\/11\/5\/595 .   \n \n\n\u2191 19.0 19.1 Rheay, Hanah T.; Omondi, Emmanuel C.; Brewer, Catherine E. (1 April 2021). \"Potential of hemp ( Cannabis sativa L.) for paired phytoremediation and bioenergy production\" (in en). GCB Bioenergy 13 (4): 525\u2013536. doi:10.1111\/gcbb.12782. ISSN 1757-1693. https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/gcbb.12782 .   \n \n\n\u2191 20.0 20.1 20.2 20.3 Shi, Gangrong; Cai, Qingsheng (7 May 2010). \"ZINC TOLERANCE AND ACCUMULATION IN EIGHT OIL CROPS\" (in en). Journal of Plant Nutrition 33 (7): 982\u2013997. doi:10.1080\/01904161003728669. ISSN 0190-4167. http:\/\/www.tandfonline.com\/doi\/abs\/10.1080\/01904161003728669 .   \n \n\n\u2191 DalCorso, Giovanni; Manara, Anna; Furini, Antonella (2013). \"An overview of heavy metal challenge in plants: from roots to shoots\" (in en). Metallomics 5 (9): 1117. doi:10.1039\/c3mt00038a. ISSN 1756-5901. https:\/\/academic.oup.com\/metallomics\/article\/5\/9\/1117-1132\/6015689 .   \n \n\n\u2191 Ismael, Marwa A.; Elyamine, Ali Mohamed; Moussa, Mohamed G.; Cai, Miaomiao; Zhao, Xiaohu; Hu, Chengxiao (2019). \"Cadmium in plants: uptake, toxicity, and its interactions with selenium fertilizers\" (in en). Metallomics 11 (2): 255\u2013277. doi:10.1039\/C8MT00247A. ISSN 1756-5901. https:\/\/academic.oup.com\/metallomics\/article\/11\/2\/255-277\/5957484 .   \n \n\n\u2191 23.0 23.1 Kabata-Pendias, Alina (18 October 2010) (in en). Trace Elements in Soils and Plants (0 ed.). CRC Press. doi:10.1201\/b10158. ISBN 978-0-429-19203-6. https:\/\/www.taylorfrancis.com\/books\/9781420093704 .   \n \n\n\u2191 Sebastian, Abin; Nangia, Ashwini; Prasad, Majeti Narasimha Vara; Rattanapolsan, Ladawan; Nakbanpote, Woranan (2019), \"Cadmium Toxicity and Tolerance in Micro- and Phytobiomes\" (in en), Cadmium Toxicity and Tolerance in Plants (Elsevier): 19\u201346, doi:10.1016\/b978-0-12-814864-8.00002-4, ISBN 978-0-12-814864-8, https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/B9780128148648000024 . Retrieved 2023-09-27   \n \n\n\u2191 Food and Agriculture Organization of the United Nations (2019). \"General Standard for Contaminants and Toxins in Food and Feed [CXS 193-1995\"] (PDF). Codex Alimentarius. pp. 47\u201348. https:\/\/www.fao.org\/fao-who-codexalimentarius\/sh-proxy\/en\/?lnk=1&url=https%253A%252F%252Fworkspace.fao.org%252Fsites%252Fcodex%252FStandards%252FCXS%2B193-1995%252FCXS_193e.pdf .   \n \n\n\u2191 26.0 26.1 Ahmad, Ayaz; Hadi, Fazal; Ali, Nasir (2 January 2015). \"Effective Phytoextraction of Cadmium (Cd) with Increasing Concentration of Total Phenolics and Free Proline in Cannabis sativa (L) Plant Under Various Treatments of Fertilizers, Plant Growth Regulators and Sodium Salt\" (in en). International Journal of Phytoremediation 17 (1): 56\u201365. doi:10.1080\/15226514.2013.828018. ISSN 1522-6514. http:\/\/www.tandfonline.com\/doi\/abs\/10.1080\/15226514.2013.828018 .   \n \n\n\u2191 27.0 27.1 27.2 Angelova, V.; Ivanova, R.; Delibaltova, V.; Ivanov, K. (1 May 2004). \"Bio-accumulation and distribution of heavy metals in fibre crops (flax, cotton and hemp)\" (in en). Industrial Crops and Products 19 (3): 197\u2013205. doi:10.1016\/j.indcrop.2003.10.001. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0926669003001110 .   \n \n\n\u2191 28.0 28.1 28.2 28.3 Citterio, Sandra; Santagostino, Angela; Fumagalli, Pietro; Prato, Nadia; Ranalli, Paolo; Sgorbati, Sergio (2003). \"[No title found\"]. Plant and Soil 256 (2): 243\u2013252. doi:10.1023\/A:1026113905129. http:\/\/link.springer.com\/10.1023\/A:1026113905129 .   \n \n\n\u2191 Kumar, Sanjeev; Singh, Ritu; Kumar, Virendra; Rani, Anita; Jain, Rajeev (2017), Bauddh, Kuldeep; Singh, Bhaskar; Korstad, John, eds., \"Cannabis sativa: A Plant Suitable for Phytoremediation and Bioenergy Production\" (in en), Phytoremediation Potential of Bioenergy Plants (Singapore: Springer Singapore): 269\u2013285, doi:10.1007\/978-981-10-3084-0_10, ISBN 978-981-10-3083-3, http:\/\/link.springer.com\/10.1007\/978-981-10-3084-0_10 . Retrieved 2023-09-27   \n \n\n\u2191 30.0 30.1 30.2 30.3 Ait Ali, Nadia; Bernal, M. Pilar; Ater, Mohammed (2002). \"[No title found\"]. Plant and Soil 239 (1): 103\u2013111. doi:10.1023\/A:1014995321560. http:\/\/link.springer.com\/10.1023\/A:1014995321560 .   \n \n\n\u2191 31.0 31.1 31.2 31.3 Liu, W.-X.; Liu, J.-W.; Wu, M.-Z.; Li, Y.; Zhao, Y.; Li, S.-R. (1 March 2009). \"Accumulation and Translocation of Toxic Heavy Metals in Winter Wheat (Triticum aestivum L.) Growing in Agricultural Soil of Zhengzhou, China\" (in en). Bulletin of Environmental Contamination and Toxicology 82 (3): 343\u2013347. doi:10.1007\/s00128-008-9575-6. ISSN 0007-4861. http:\/\/link.springer.com\/10.1007\/s00128-008-9575-6 .   \n \n\n\u2191 32.0 32.1 32.2 32.3 Pachura, Piotr; Ociepa-Kubicka, Agnieszka; Skowron-Grabowska, Beata (14 January 2016). \"Assessment of the availability of heavy metals to plants based on the translocation index and the bioaccumulation factor\" (in en). Desalination and Water Treatment 57 (3): 1469\u20131477. doi:10.1080\/19443994.2015.1017330. ISSN 1944-3994. http:\/\/www.tandfonline.com\/doi\/full\/10.1080\/19443994.2015.1017330 .   \n \n\n\u2191 33.0 33.1 Zayed, Adel; Gowthaman, Suvarnalatha; Terry, Norman (1 May 1998). \"Phytoaccumulation of Trace Elements by Wetland Plants: I. Duckweed\" (in en). Journal of Environmental Quality 27 (3): 715\u2013721. doi:10.2134\/jeq1998.00472425002700030032x. ISSN 0047-2425. https:\/\/acsess.onlinelibrary.wiley.com\/doi\/10.2134\/jeq1998.00472425002700030032x .   \n \n\n\u2191 34.0 34.1 34.2 34.3 34.4 34.5 Luyckx, Marie; Hausman, Jean-Fran\u00e7ois; Blanquet, Mathilde; Guerriero, Gea; Lutts, Stanley (1 July 2021). \"Silicon reduces cadmium absorption and increases root-to-shoot translocation without impacting growth in young plants of hemp (Cannabis sativa L.) on a short-term basis\" (in en). Environmental Science and Pollution Research 28 (28): 37963\u201337977. doi:10.1007\/s11356-021-12912-y. ISSN 0944-1344. https:\/\/link.springer.com\/10.1007\/s11356-021-12912-y .   \n \n\n\u2191 35.0 35.1 35.2 35.3 35.4 Marabesi, Amanda O.; Nambeesan, Savithri U.; van Iersel, Marc W.; Lessl, Jason T.; Coolong, Timothy W. (30 May 2023). \"Cadmium exposure is associated with increased transcript abundance of multiple heavy metal associated transporter genes in roots of hemp (Cannabis sativa L.)\". Frontiers in Plant Science 14: 1183249. doi:10.3389\/fpls.2023.1183249. ISSN 1664-462X. PMC PMC10265645. PMID 37324677. https:\/\/www.frontiersin.org\/articles\/10.3389\/fpls.2023.1183249\/full .   \n \n\n\u2191 Hoagland, D.R.; Arnon, D.I. (1950). \"The water-culture method for growing plants without soil\". California Agricultural Experiment Station, Circular. University of California. https:\/\/openlibrary.org\/books\/OL25240089M\/The_water-culture_method_for_growing_plants_without_soil .   \n \n\n\u2191 U.S. EPA (December 1996). \"Method 3052. Microwave assisted acid digestion of siliceous and organically based matrices\" (PDF). Test methods for evaluating solid waste. https:\/\/www.epa.gov\/sites\/default\/files\/2015-12\/documents\/3052.pdf .   \n \n\n\u2191 U.S. EPA (1994). \"EPA Method 200.8: Determination of Trace Elements in Waters and Wastes by Inductively Coupled Plasma-Mass Spectrometry\". Selected Analytical Methods for Environmental Remediation and Recovery (SAM). https:\/\/www.epa.gov\/esam\/epa-method-2008-determination-trace-elements-waters-and-wastes-inductively-coupled-plasma-mass .   \n \n\n\u2191 39.0 39.1 39.2 39.3 39.4 Shi, Gangrong; Cai, Qingsheng (1 September 2009). \"Cadmium tolerance and accumulation in eight potential energy crops\" (in en). Biotechnology Advances 27 (5): 555\u2013561. doi:10.1016\/j.biotechadv.2009.04.006. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0734975009000573 .   \n \n\n\u2191 \u0106a\u0107i\u0107, Marija; Per\u010din, Aleksandra; Zgorelec, \u017deljka; Kisi\u0107, Ivica (2019). \"Evaluation of heavy metals accumulation potential of hemp (Cannabis sativa L.)\" (in en). Journal of Central European Agriculture 20 (2): 700\u2013711. doi:10.5513\/JCEA01\/20.2.2201. ISSN 1332-9049. https:\/\/jcea.agr.hr\/en\/issues\/article\/2201 .   \n \n\n\u2191 41.0 41.1 41.2 Shi, Gangrong; Liu, Caifeng; Cui, Meicheng; Ma, Yuhua; Cai, Qingsheng (1 September 2012). \"Cadmium Tolerance and Bioaccumulation of 18 Hemp Accessions\" (in en). Applied Biochemistry and Biotechnology 168 (1): 163\u2013173. doi:10.1007\/s12010-011-9382-0. ISSN 0273-2289. http:\/\/link.springer.com\/10.1007\/s12010-011-9382-0 .   \n \n\n\u2191 University of Sulaimani; Majid, Salih N.; Khwakaram, Ahmed I.; University of Sulaimani; Rasul, Ghafoor A. Mam; University of Sulaimani (16 October 2014). \"Bioaccumulation, Enrichment and Translocation Factors of some Heavy Metals in Typha Angustifolia and Phragmites Australis Species Growing along Qalyasan Stream in Sulaimani City \/IKR\". Journal of Zankoy Sulaimani - Part A 16 (4): 93\u2013109. doi:10.17656\/jzs.10350. https:\/\/jzs.univsul.edu.iq\/index.php\/jzs\/article\/view\/jzs-10350 .   \n \n\n\u2191 Bernstein, Nirit; Gorelick, Jonathan; Zerahia, Roei; Koch, Sraya (17 June 2019). \"Impact of N, P, K, and Humic Acid Supplementation on the Chemical Profile of Medical Cannabis (Cannabis sativa L)\". Frontiers in Plant Science 10: 736. doi:10.3389\/fpls.2019.00736. ISSN 1664-462X. PMC PMC6589925. PMID 31263470. https:\/\/www.frontiersin.org\/article\/10.3389\/fpls.2019.00736\/full .   \n \n\n\u2191 Khan, Anwarzeb; Khan, Sardar; Alam, Mehboob; Khan, Muhammad Amjad; Aamir, Muhammad; Qamar, Zahir; Rehman, Zahir Ur; Perveen, Sajida (1 March 2016). \"Toxic metal interactions affect the bioaccumulation and dietary intake of macro- and micro-nutrients\" (in en). Chemosphere 146: 121\u2013128. doi:10.1016\/j.chemosphere.2015.12.014. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0045653515304616 .   \n \n\n\u2191 Ahmad, Parvaiz; Abdel Latef, Arafat A.; Abd_Allah, Elsayed F.; Hashem, Abeer; Sarwat, Maryam; Anjum, Naser A.; Gucel, Salih (27 April 2016). \"Calcium and Potassium Supplementation Enhanced Growth, Osmolyte Secondary Metabolite Production, and Enzymatic Antioxidant Machinery in Cadmium-Exposed Chickpea (Cicer arietinum L.)\". Frontiers in Plant Science 7. doi:10.3389\/fpls.2016.00513. ISSN 1664-462X. PMC PMC4847423. PMID 27200003. http:\/\/journal.frontiersin.org\/Article\/10.3389\/fpls.2016.00513\/abstract .   \n \n\n\u2191 46.0 46.1 Gomes, Marcelo Pedrosa; Marques, Teresa Cristina Lara Lanza S\u00e1 e Mel; Soares, Angela Maria (1 April 2013). \"Cadmium effects on mineral nutrition of the Cd-hyperaccumulator Pfaffia glomerata\" (in en). Biologia 68 (2): 223\u2013230. doi:10.2478\/s11756-013-0005-9. ISSN 0006-3088. http:\/\/link.springer.com\/10.2478\/s11756-013-0005-9 .   \n \n\n\u2191 Sarwar, Nadeem; Saifullah; Malhi, Sukhdev S; Zia, Munir Hussain; Naeem, Asif; Bibi, Sadia; Farid, Ghulam (30 April 2010). \"Role of mineral nutrition in minimizing cadmium accumulation by plants: Mineral nutrition for minimizing cadmium accumulation\" (in en). Journal of the Science of Food and Agriculture 90 (6): 925\u2013937. doi:10.1002\/jsfa.3916. https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/jsfa.3916 .   \n \n\n\u2191 Husain, Rabab; Weeden, Hannah; Bogush, Daniel; Deguchi, Michihito; Soliman, Mario; Potlakayala, Shobha; Katam, Ramesh; Goldman, Stephen et al. (29 August 2019). Thavamani, Palanisami. ed. \"Enhanced tolerance of industrial hemp (Cannabis sativa L.) plants on abandoned mine land soil leads to overexpression of cannabinoids\" (in en). PLOS ONE 14 (8): e0221570. doi:10.1371\/journal.pone.0221570. ISSN 1932-6203. PMC PMC6715179. PMID 31465423. https:\/\/dx.plos.org\/10.1371\/journal.pone.0221570 .   \n \n\n\u2191 Yang, Rui; Berthold, Erin C.; McCurdy, Christopher R.; da Silva Benevenute, Sarah; Brym, Zachary T.; Freeman, Joshua H. (3 June 2020). \"Development of Cannabinoids in Flowers of Industrial Hemp ( Cannabis sativa L.): A Pilot Study\" (in en). Journal of Agricultural and Food Chemistry 68 (22): 6058\u20136064. doi:10.1021\/acs.jafc.0c01211. ISSN 0021-8561. https:\/\/pubs.acs.org\/doi\/10.1021\/acs.jafc.0c01211 .   \n \n\n\u2191 Trancoso, Ingrid; de Souza, Guilherme A. R.; dos Santos, Paulo Ricardo; dos Santos, K\u00e9sia Dias; de Miranda, Rosana Maria dos Santos Nani; da Silva, Amanda L\u00facia Pereira Machado; Santos, Dennys Zsolt; Garc\u00eda-Tejero, Ivan F. et al. (22 June 2022). \"Cannabis sativa L.: Crop Management and Abiotic Factors That Affect Phytocannabinoid Production\" (in en). Agronomy 12 (7): 1492. doi:10.3390\/agronomy12071492. ISSN 2073-4395. https:\/\/www.mdpi.com\/2073-4395\/12\/7\/1492 .   \n \n\n\nNotes \nThis presentation is faithful to the original, with only a few minor changes to presentation. Some grammar and punctuation was cleaned up to improve readability. In some cases important information was missing from the references, and that information was added.\n\n\n\n\n\nSource: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\">https:\/\/www.cannaqa.wiki\/index.php?title=Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market<\/a>\nCategories: Pages containing cite templates with deprecated parametersCannaQAwiki journal articles (added in 2023)CannaQAwiki journal articles (all)CannaQAwiki journal articles on cannabis researchCannaQAwiki journal articles on cannabis testingNavigation menuPage actionsJournalDiscussionView sourceHistoryPage actionsJournalDiscussionMoreToolsIn other languagesPersonal toolsLog inNavigationMain pageList of articlesRandom pageRecent changesHelpSearch\u00a0 ToolsWhat links hereRelated changesSpecial pagesPrintable versionPermanent linkPage information This page was last edited on 27 September 2023, at 00:40.Content is available under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License unless otherwise noted.Privacy policyAbout CannaQAWikiDisclaimers\n","34fb9a0e0648cc8855dc5ea15c3c3a92_html":"<body class=\"mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-170 ns-subject page-Journal_Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market rootpage-Journal_Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market skin-monobook action-view skin--responsive\"><div id=\"rdp-ebb-globalWrapper\"><div id=\"rdp-ebb-column-content\"><div id=\"rdp-ebb-content\" class=\"mw-body\" role=\"main\"><a id=\"rdp-ebb-top\"><\/a>\n<h1 id=\"rdp-ebb-firstHeading\" class=\"firstHeading\" lang=\"en\">Journal:Cadmium bioconcentration and translocation potential in day-neutral and photoperiod-sensitive hemp grown hydroponically for the medicinal market<\/h1><div id=\"rdp-ebb-bodyContent\" class=\"mw-body-content\"><!-- start content --><div id=\"rdp-ebb-mw-content-text\" lang=\"en\" dir=\"ltr\" class=\"mw-content-ltr\"><div class=\"mw-parser-output\">\n\n\n<h2><span class=\"mw-headline\" id=\"Abstract\">Abstract<\/span><\/h2>\n<p><a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Heavy_metals\" title=\"Heavy metals\" class=\"wiki-link\" data-key=\"a485b8274a229bee7a5d842d06729007\">Heavy metal<\/a> <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Contamination\" title=\"Contamination\" class=\"wiki-link\" data-key=\"0663203ff7531a97a6d414b3ecc2c94d\">contamination<\/a> of agricultural soils is potentially concerning when growing crops for human consumption. Industrial <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Hemp\" title=\"Hemp\" class=\"wiki-link\" data-key=\"c23e30b6cf1df54f1dc338492c9f9da2\">hemp<\/a> (<i><a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabis_sativa\" title=\"Cannabis sativa\" class=\"wiki-link\" data-key=\"e003358742012354d1ff6002bc5781de\">Cannabis sativa L.<\/a><\/i>) has been reported to tolerate the presence of heavy metals such as <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cadmium\" title=\"Cadmium\" class=\"wiki-link\" data-key=\"35b033eb9996b45aa19489f6f4a5d566\">cadmium<\/a> (Cd) in the soil. Therefore, the objectives of this study were to evaluate Cd uptake and translocation in two day-length-sensitive (DLS) and two day-neutral (DN) hemp varieties grown for the medicinal market and to determine the impact of Cd exposure on <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabinoid\" title=\"Cannabinoid\" class=\"wiki-link\" data-key=\"c224c3041748677fcdce5b5209900b7b\">cannabinoid<\/a> concentrations in flowers. A hydroponic experiment was conducted by exposing plants to 0 mg\u00b7L<sup>\u22121<\/sup> Cd and 2.5 mg\u00b7L<sup>\u22121<\/sup> Cd in the nutrient solution. Cadmium concentrations ranged from 16.1 to 2274.2 mg\u00b7kg<sup>\u22121<\/sup> in roots, though all four varieties accumulated significant concentrations of Cd in aboveground tissues, with translocation factors ranging from 6.5 to 193. Whole-plant bioconcentration factors ranged from 20 to 1051 mg\u00b7kg<sup>\u22121<\/sup>. Cannabinoid concentrations were negatively impacted by Cd exposure in DN varieties but were unaffected in DLS varieties. Biomass was reduced by Cd exposure demonstrating that these varieties might not be suitable for growth on contaminated soil or for phytoremediation. There is potential for Cd accumulation in flowers, showing the need for heavy metal testing of <i>C. sativa<\/i> consumer products.\n<\/p><p><b>Keywords<\/b>: auto-flower, bioconcentration factor, cannabinoid, heavy metal, hemp, metal stress, translocation factor\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Introduction\">Introduction<\/span><\/h2>\n<p>For centuries, humans have cultivated <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Hemp\" title=\"Hemp\" class=\"wiki-link\" data-key=\"c23e30b6cf1df54f1dc338492c9f9da2\">hemp<\/a> (<i><a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabis_sativa\" title=\"Cannabis sativa\" class=\"wiki-link\" data-key=\"e003358742012354d1ff6002bc5781de\">Cannabis sativa L.<\/a><\/i>) for its fiber, seed, therapeutic, and <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Psychoactive_drug\" title=\"Psychoactive drug\" class=\"wiki-link\" data-key=\"abba12c7100bf1c8457208da20b4234b\">psychoactive<\/a> properties. During the domestication process, wild <i><a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabis\" title=\"Cannabis\" class=\"wiki-link\" data-key=\"a70b76268930d795518ff1f98d7e500d\">Cannabis<\/a><\/i> populations have been subject to selection, giving rise to the multiple varieties that exist today.<sup id=\"rdp-ebb-cite_ref-1\" class=\"reference\"><a href=\"#cite_note-1\">[1]<\/a><\/sup> The term \"industrial hemp\" is commonly used to refer to <i>C. sativa<\/i> plants<sup id=\"rdp-ebb-cite_ref-2\" class=\"reference\"><a href=\"#cite_note-2\">[2]<\/a><\/sup> with total <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Tetrahydrocannabinol\" title=\"Tetrahydrocannabinol\" class=\"wiki-link\" data-key=\"15f3b3e338baeb54c04c715818759ec9\">tetrahydrocannabinol<\/a> (THC) concentrations below 0.3%. Plants with total THC concentrations above 0.3% are classified as marijuana and subject to federal prohibition in the United States (U.S.)<sup id=\"rdp-ebb-cite_ref-3\" class=\"reference\"><a href=\"#cite_note-3\">[3]<\/a><\/sup> Current industrial hemp breeding efforts target fiber, seed, or <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabinoid\" title=\"Cannabinoid\" class=\"wiki-link\" data-key=\"c224c3041748677fcdce5b5209900b7b\">cannabinoid<\/a> production, based on the end user. Hemp varieties with high <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabidiol\" title=\"Cannabidiol\" class=\"wiki-link\" data-key=\"bbfcdd40d5eda5d20d1d5ab368c922a0\">cannabidiol<\/a> (CBD) concentrations are often selected for medicinal and therapeutic uses. There is also a focus on developing varieties that are day-neutral (DN) or minimally sensitive to photoperiod in order to expand production opportunities.<sup id=\"rdp-ebb-cite_ref-4\" class=\"reference\"><a href=\"#cite_note-4\">[4]<\/a><\/sup>\n<\/p><p>Hemp is generally considered a qualitative short-day plant that flowers in response to decreasing photoperiods. Hemp selections that flower in response to photoperiod are known as day-length-sensitive (DLS). After emergence, hemp undergoes a photoperiod-dependent vegetative phase maintained by exposure to approximately 14\u201318 hours or more of light daily.<sup id=\"rdp-ebb-cite_ref-:0_5-0\" class=\"reference\"><a href=\"#cite_note-:0-5\">[5]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-6\" class=\"reference\"><a href=\"#cite_note-6\">[6]<\/a><\/sup> When hemp is planted during periods of short days (<13 hours of light), it may flower prematurely. Premature flowering, prior to complete vegetative development, can result in yield reductions.<sup id=\"rdp-ebb-cite_ref-7\" class=\"reference\"><a href=\"#cite_note-7\">[7]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-8\" class=\"reference\"><a href=\"#cite_note-8\">[8]<\/a><\/sup> In contrast, some hemp varieties exhibit DN flowering tendencies known colloquially as \u201cauto-flower\u201d hemp. These DN varieties are relatively insensitive to photoperiod for flower induction. The DN trait is speculated to arise from <i><a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabis_ruderalis\" title=\"Cannabis ruderalis\" class=\"wiki-link\" data-key=\"42623e227b8db007679c59aa10cc1eab\">Cannabis ruderalis<\/a><\/i> (<i>C. sativa<\/i> ssp. <i>ruderalis<\/i>) and may have originated from hemp located in high latitudes where photoperiods can be long and growing seasons are typically short or regions with relatively short and constant photoperiods.<sup id=\"rdp-ebb-cite_ref-:0_5-1\" class=\"reference\"><a href=\"#cite_note-:0-5\">[5]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-9\" class=\"reference\"><a href=\"#cite_note-9\">[9]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-10\" class=\"reference\"><a href=\"#cite_note-10\">[10]<\/a><\/sup> Advantages of DN hemp varieties include the ability to flower in regions that have little variation in photoperiod throughout the year (tropics) or during times of the year when photoperiods may be inadequate to grow DLS varieties. However, many DN types of hemp have been reported to be particularly sensitive to environmental stressors such as high temperatures and may have lower yields than comparable DLS varieties.<sup id=\"rdp-ebb-cite_ref-:1_11-0\" class=\"reference\"><a href=\"#cite_note-:1-11\">[11]<\/a><\/sup>\n<\/p><p>In addition to uses for fiber, seed, and medicinal purposes, hemp has also been proposed as a candidate for use in phytoremediation, which utilizes plants to remove <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Contamination\" title=\"Contamination\" class=\"wiki-link\" data-key=\"0663203ff7531a97a6d414b3ecc2c94d\">contaminants<\/a>, such as <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Heavy_metals\" title=\"Heavy metals\" class=\"wiki-link\" data-key=\"a485b8274a229bee7a5d842d06729007\">heavy metals<\/a> or other chemicals from soils.<sup id=\"rdp-ebb-cite_ref-:2_12-0\" class=\"reference\"><a href=\"#cite_note-:2-12\">[12]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-13\" class=\"reference\"><a href=\"#cite_note-13\">[13]<\/a><\/sup> Accumulator plant species can uptake heavy metals from soils, even at low external concentrations, and concentrate them in plant tissues.<sup id=\"rdp-ebb-cite_ref-:3_14-0\" class=\"reference\"><a href=\"#cite_note-:3-14\">[14]<\/a><\/sup> By growing accumulator plants in contaminated soil, it is possible to realize <i>in situ<\/i> decontamination, an economically viable approach that preserves physicochemical soil characteristics, while removing contaminants.<sup id=\"rdp-ebb-cite_ref-15\" class=\"reference\"><a href=\"#cite_note-15\">[15]<\/a><\/sup> The morphophysiological characteristics of hemp, such as high biomass production, deep roots, and short life cycle, make it a potential candidate for phytoremediation.<sup id=\"rdp-ebb-cite_ref-:4_16-0\" class=\"reference\"><a href=\"#cite_note-:4-16\">[16]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:5_17-0\" class=\"reference\"><a href=\"#cite_note-:5-17\">[17]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-18\" class=\"reference\"><a href=\"#cite_note-18\">[18]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:6_19-0\" class=\"reference\"><a href=\"#cite_note-:6-19\">[19]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:7_20-0\" class=\"reference\"><a href=\"#cite_note-:7-20\">[20]<\/a><\/sup>\n<\/p><p>Heavy metal contamination of agricultural soils is a concern when growing crops for food or medicinal purposes, due to potential harm to human and animal health.<sup id=\"rdp-ebb-cite_ref-21\" class=\"reference\"><a href=\"#cite_note-21\">[21]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-22\" class=\"reference\"><a href=\"#cite_note-22\">[22]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:8_23-0\" class=\"reference\"><a href=\"#cite_note-:8-23\">[23]<\/a><\/sup> <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cadmium\" title=\"Cadmium\" class=\"wiki-link\" data-key=\"35b033eb9996b45aa19489f6f4a5d566\">Cadmium<\/a> (Cd) contamination in the environment has been linked to anthropogenic activities, such as mining and smelting. Further, Cd can be introduced to soils via contaminated manure, sewage sludge, and phosphate fertilizers.<sup id=\"rdp-ebb-cite_ref-24\" class=\"reference\"><a href=\"#cite_note-24\">[24]<\/a><\/sup> Cadmium is known to cause health issues when ingested in amounts greater than the provisional tolerable monthly intake (PTMI) of 25 \u03bcg\u00b7kg<sup>\u22121<\/sup> of body weight.<sup id=\"rdp-ebb-cite_ref-25\" class=\"reference\"><a href=\"#cite_note-25\">[25]<\/a><\/sup> In previous studies utilizing naturally and artificially contaminated soil and substrate containing from 0 to up to 200 mg\u00b7kg<sup>\u22121<\/sup> Cd, hemp varieties grown for fiber production accumulated Cd in aboveground tissues at levels that could be harmful to human health.<sup id=\"rdp-ebb-cite_ref-:9_26-0\" class=\"reference\"><a href=\"#cite_note-:9-26\">[26]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:10_27-0\" class=\"reference\"><a href=\"#cite_note-:10-27\">[27]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:11_28-0\" class=\"reference\"><a href=\"#cite_note-:11-28\">[28]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-29\" class=\"reference\"><a href=\"#cite_note-29\">[29]<\/a><\/sup> For instance, the hemp fiber variety Silistrinski grown in naturally contaminated soil containing 12.2 mg\u00b7kg<sup>\u22121<\/sup> Cd accumulated 1.22 mg\u00b7kg<sup>\u22121<\/sup> Cd in its flowers.<sup id=\"rdp-ebb-cite_ref-:10_27-1\" class=\"reference\"><a href=\"#cite_note-:10-27\">[27]<\/a><\/sup>\n<\/p><p>There are multiple indicators that can be used to determine the accumulation potential of a plant species. Bioconcentration factor (BCF) is the ratio between the metal concentration in plant tissues and the initial metal concentration in the soil or growing solution.<sup id=\"rdp-ebb-cite_ref-:7_20-1\" class=\"reference\"><a href=\"#cite_note-:7-20\">[20]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:12_30-0\" class=\"reference\"><a href=\"#cite_note-:12-30\">[30]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:13_31-0\" class=\"reference\"><a href=\"#cite_note-:13-31\">[31]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:14_32-0\" class=\"reference\"><a href=\"#cite_note-:14-32\">[32]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:15_33-0\" class=\"reference\"><a href=\"#cite_note-:15-33\">[33]<\/a><\/sup> This indicator has also been used interchangeably with terms such as accumulation factor (AF)<sup id=\"rdp-ebb-cite_ref-:11_28-1\" class=\"reference\"><a href=\"#cite_note-:11-28\">[28]<\/a><\/sup>, biological absorption coefficient (BAC) or index of bioaccumulation (IBA).<sup id=\"rdp-ebb-cite_ref-:8_23-1\" class=\"reference\"><a href=\"#cite_note-:8-23\">[23]<\/a><\/sup> A separate indicator of accumulation potential is the translocation factor (TF), which is the ratio between the metal concentration in the above ground biomass and the metal concentration in the roots.<sup id=\"rdp-ebb-cite_ref-:13_31-1\" class=\"reference\"><a href=\"#cite_note-:13-31\">[31]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:14_32-1\" class=\"reference\"><a href=\"#cite_note-:14-32\">[32]<\/a><\/sup> Additionally, plant growth parameters can be assessed to determine the tolerance index (TI), calculated as the ratio between growth in contaminated and non-contaminated soils.<sup id=\"rdp-ebb-cite_ref-:7_20-2\" class=\"reference\"><a href=\"#cite_note-:7-20\">[20]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:12_30-1\" class=\"reference\"><a href=\"#cite_note-:12-30\">[30]<\/a><\/sup> There is significant variability in BCF among plant species and chemical elements. It has been proposed that plants with BCF >100 mg\u00b7kg<sup>\u22121<\/sup> Cd on a dry weight (DW) basis in its leaves could be referred to as hyperaccumulators.<sup id=\"rdp-ebb-cite_ref-:3_14-1\" class=\"reference\"><a href=\"#cite_note-:3-14\">[14]<\/a><\/sup> Conversely, Chaney and Baklanov<sup id=\"rdp-ebb-cite_ref-:2_12-1\" class=\"reference\"><a href=\"#cite_note-:2-12\">[12]<\/a><\/sup> suggested that true hyperaccumulators are able to accumulate higher concentrations of metals in leaves than in roots (TF > 1).\n<\/p><p>Few studies have evaluated heavy metal accumulation in hemp flowers, with most research utilizing fiber hemp varieties to determine heavy metal uptake for phytoremediation purposes.<sup id=\"rdp-ebb-cite_ref-:4_16-1\" class=\"reference\"><a href=\"#cite_note-:4-16\">[16]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:5_17-1\" class=\"reference\"><a href=\"#cite_note-:5-17\">[17]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:6_19-1\" class=\"reference\"><a href=\"#cite_note-:6-19\">[19]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:16_34-0\" class=\"reference\"><a href=\"#cite_note-:16-34\">[34]<\/a><\/sup> Due to the harmful effects of Cd and other heavy metals on human health, the U.S. hemp industry has attempted to implement standards regarding maximum allowable levels of metals in <i>C. sativa<\/i> consumer products, which vary by state.<sup id=\"rdp-ebb-cite_ref-:17_35-0\" class=\"reference\"><a href=\"#cite_note-:17-35\">[35]<\/a><\/sup> As hemp flowers are increasingly grown for the medicinal market, determining Cd distribution among plant organs, as well as bioconcentration and root-to-shoot translocation factors, are of importance. We hypothesize that there are distinctions in Cd accumulation and distribution among plant tissues in hemp varieties with different growth and flowering habits. Therefore, the objectives of this study were to evaluate nutrient partitioning and Cd uptake, translocation, and accumulation in DLS and DN hemp varieties, and to determine the impact of Cd exposure to cannabinoids in plant flowers.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Materials_and_methods\">Materials and methods<\/span><\/h2>\n<h3><span class=\"mw-headline\" id=\"Experimental_settings\">Experimental settings<\/span><\/h3>\n<p>The experiment was conducted in a greenhouse in Watkinsville, GA, USA (lat. 33\u00b05\u2032 N, long. 83\u00b03\u2032 W) from January 2022 to April 2022. Feminized seed from two DLS hemp varieties, T1 and Von (Sunbelt Hemp Source, Moultrie, GA, USA), and two DN varieties Apricot Auto (Blue Forest Farms, New York, NY, USA) and Auto CBD Alpha Explorer (Alpha Explorer) (Phylos Bioscience, Portland, OR, USA) were sown into engineered foam cubes (3.33 cm L \u00d7 2.54 cm W \u00d7 3.81 cm D; Oasis Grower Solutions, Kent, OH, USA) for germination. Foam cubes were placed in plastic trays over a germination mat set at 24 \u00b0C exposed to a mist irrigation system, which applied water twice daily for one minute each. Supplemental lighting (approximately 100 \u00b5mol\u00b7m<sup>\u22122<\/sup>\u00b7s<sup>\u22121<\/sup>) was used during germination. Seedlings were maintained under these conditions for four weeks, after which they were placed into plastic netted containers (4.7 cm W \u00d7 5.1 cm D) and transferred to 37.9 L plastic containers (Rubbermaid Inc. Wooster, OH, USA) filled with 28 L of well water. The well water was analyzed for nutrient concentrations periodically throughout the experiment (Table 1). Four seedlings per replicate were placed equidistantly (24.3 cm apart) in holes drilled in the container lid. Welded wire mesh frames were attached to each lid to support plants. A 15.2 cm aquarium air stone attached to an air pump (Active Aqua; Hydrofarm, Petaluma, CA, USA) was placed inside the container to aerate the nutrient solution throughout the experiment. Container volume was maintained by adding well water every two to three days. At transplant, a nutrient solution was added to the plastic containers using a half-strength Hoagland\u2019s solution<sup id=\"rdp-ebb-cite_ref-36\" class=\"reference\"><a href=\"#cite_note-36\">[36]<\/a><\/sup> (Table 1).\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"12\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 1.<\/b> Mineral nutrient concentrations in the nutrient solution used in this study. Notes: Values shown in mg\u00b7L<sup>\u22121<\/sup>. ND = not detected. <sup>i<\/sup> Nutrient solution comprised from the following compounds: Ca(NO<sub>3<\/sub>)<sub>2<\/sub>\u22194H<sub>2<\/sub>O, KNO<sub>3<\/sub>, KH<sub>2<\/sub>PO<sub>4<\/sub>, MgSO<sub>4<\/sub>\u22197H<sub>2<\/sub>O, H<sub>3<\/sub>BO<sub>3<\/sub>, MnCl<sub>2<\/sub>\u22194H<sub>2<\/sub>O, ZnSO<sub>4<\/sub>\u22197H<sub>2<\/sub>O, CuSO<sub>4<\/sub>\u22195H<sub>2<\/sub>O, H<sub>2<\/sub>MoO<sub>4<\/sub>\u2219H<sub>2<\/sub>O, and Sequestrene 330.\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">N\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">P\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">K\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Ca\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Mg\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">B\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Cu\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Mo\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Fe\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Mn\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Zn\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Well water\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">ND\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><0.01\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2.7\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">12.1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2.1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><0.01\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><0.05\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><0.01\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><0.1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><0.1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><0.1\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Nutrient solution<sup>i<\/sup>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">105\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">15.5\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">117\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">100\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">24.3\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.3\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.01\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.005\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.5\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.25\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.025\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Total concentration\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">105\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">15.5\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">119.7\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">112.1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">26.4\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.3\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><0.05\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><0.01\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.5\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.25\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><0.1\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>Plants were grown for three weeks in the base nutrient solution after which nutrient solutions were replaced completely and Cd treatments were added using CdSO<sub>4<\/sub>\u00b78H<sub>2<\/sub>O, to achieve 0 (control) and 2.5 mg\u00b7L<sup>\u22121<\/sup> Cd. Cadmium concentrations were chosen based on the results of previous studies<sup id=\"rdp-ebb-cite_ref-:16_34-1\" class=\"reference\"><a href=\"#cite_note-:16-34\">[34]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:17_35-1\" class=\"reference\"><a href=\"#cite_note-:17-35\">[35]<\/a><\/sup>, which evaluated hemp exposure to Cd in hydroponic systems. The experimental treatments were arranged in a randomized complete block design, with four hemp varieties exposed to two levels of Cd with four replicates each. Nutrient solutions were maintained to a constant volume by adding water every two to three days, and nutrients were replaced every three weeks for the remainder of the experiment. The electrical conductivity (EC) and pH of the solutions were measured weekly. Solution pH was adjusted to 5.5 when necessary (pH down; General Hydroponics, Santa Rosa, CA, USA). Supplemental light (approximately 100 \u00b5mol\u00b7m<sup>\u22122<\/sup>\u00b7s<sup>\u22121<\/sup>) was used to provide 18\/6 light\/dark hours for four weeks of vegetative growth after transplanting seedlings into containers. Supplemental lights were turned off to allow for flower induction in the DLS varieties for the remaining seven weeks of production (average day length 12 hours and 38 minutes). The DN varieties (Apricot Auto and Auto CBD Alpha Explorer) exhibited visually detectable flower development one week prior to the induction of flowering in the DLS varieties (Von and T1). Therefore, the DN varieties were harvested one week prior to the DLS varieties to ensure that plants flowered for the same length of time.\n<\/p><p>Nutrient solutions were sampled at the beginning and end of each three-week cycle using 20 mL scintillation vials (HDPE; Thermo-Fisher Scientific\u2122, Waltham, MS, USA), and stored at \u22124 \u00b0C until the analysis of mineral nutrient concentrations (Table S1). Temperature and relative humidity (RH) of the greenhouse were monitored at canopy height hourly (VP4; Meter Group Inc., Pullman WA, USA) and averaged 19.1 \u00b1 3.1 \u00b0C and 74 \u00b1 0.1% RH for the experiment. Photosynthetic active radiation was also monitored hourly throughout the experiment (QSO-S; Meter Group Inc.) and plants were exposed to an average daily light integral (DLI) average of 21.6 \u00b1 9.0 mol\u00b7m<sup>\u22122<\/sup>\u00b7d<sup>\u22121<\/sup>.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Mineral_analysis\">Mineral analysis<\/span><\/h3>\n<p>Samples of fresh root, stem, leaf, and flower tissues were collected for Cd analysis at harvest. Composite samples (50 g fresh material) were taken from each of the four plants in a replicate (container). Roots were triple-washed with deionized water after removal. Ten of the youngest fully expanded leaves were collected from the top one-third of each plant (main stem and lateral branches) and rinsed with deionized water. Stem samples were collected from the bottom two-thirds of the main stem. Flower material was sampled from the top of the main stem and the top one-third of plants. Samples were placed in a forced air oven set at 55 \u00b0C for 72 hours until a constant weight was achieved. Dried plant material was then ground in a Wiley mill (Thomas Scientific, Swedesboro, NJ, USA) and passed through a 20-mesh screen. Samples were digested using EPA Method 3052.<sup id=\"rdp-ebb-cite_ref-37\" class=\"reference\"><a href=\"#cite_note-37\">[37]<\/a><\/sup> In brief, 0.5 gram samples were placed in fluorocarbon polymer microwave vessels, 10 mL of concentrated nitric acid were added to each vessel which was then sealed. The microwave digester (Mars 6 Microwave; CEM Corp., Matthews, NC, USA) was heated to 200 \u00b0C for 30 minutes and digested (solutions) were then transferred quantitatively into volumetric flasks and brought to 100 mL volume with deionized water prior to analysis.\n<\/p><p>Samples of the hydroponic solutions were filtered using a 0.45 \u00b5M PTFE membrane (Thermo-Fisher Scientific\u2122 Choice\u2122 Polypropylene Syringe Filters) and acidified using 2% (v\/v) high purity nitric acid (HNO<sub>3<\/sub>) (Certified ACS Plus, Fisher Scientific, Pittsburgh, PA, USA) prior to analysis. Hydroponic solutions and plant tissues were analyzed for multiple elements\u2014phosphorous (P), potassium (K), sulfur (S), calcium (Ca), magnesium (Mg), iron (Fe), manganese (Mn), aluminum (Al), boron (B), <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Copper\" title=\"Copper\" class=\"wiki-link\" data-key=\"8cd5b38f858ab566efabc49263a90089\">copper<\/a> (Cu), <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Zinc\" title=\"Zinc\" class=\"wiki-link\" data-key=\"e46c66bdffbc298c96ca3d9b96333048\">zinc<\/a> (Zn), <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Nickel\" title=\"Nickel\" class=\"wiki-link\" data-key=\"ed065345a0c45e5dadbb27aace1bbcd5\">nickel<\/a> (Ni), and cadmium (Cd)\u2014following EPA Method 200.8<sup id=\"rdp-ebb-cite_ref-38\" class=\"reference\"><a href=\"#cite_note-38\">[38]<\/a><\/sup> by <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Inductively_coupled_plasma_atomic_emission_spectroscopy\" title=\"Inductively coupled plasma atomic emission spectroscopy\" class=\"wiki-link\" data-key=\"c3d9e1c4597891de8e869097753f2c66\">inductively coupled plasma\u2013optical emission spectroscopy<\/a> (ICP-OES) (Spectro Arcos FHS16; Spectro Ametek USA, Wilmington, MA, USA). The instrument parameter settings and wavelengths used are displayed in the supplementary material (Tables S3 and S4, and Figure S1). The instrument reporting limit for Cd was <0.005 mg\u00b7L<sup>\u22121<\/sup>. Results were expressed as mg\u00b7L<sup>\u22121<\/sup>. Calibration standards utilized in this analysis were from a certified source (Inorganic Ventures, Christiansburg, VA, lot number: N2-MEB667614). Independent laboratory performance checks were also run with acceptable deviations for recoveries set at 100 \u00b1 5.0%.\n<\/p><p>The BCF was calculated by dividing the Cd concentration in plant tissues by the initial Cd concentration in the nutrient solution..<sup id=\"rdp-ebb-cite_ref-:7_20-3\" class=\"reference\"><a href=\"#cite_note-:7-20\">[20]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:12_30-2\" class=\"reference\"><a href=\"#cite_note-:12-30\">[30]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:13_31-2\" class=\"reference\"><a href=\"#cite_note-:13-31\">[31]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:14_32-2\" class=\"reference\"><a href=\"#cite_note-:14-32\">[32]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:15_33-1\" class=\"reference\"><a href=\"#cite_note-:15-33\">[33]<\/a><\/sup> The TF (%) was calculated by dividing the sum of Cd concentration in leaves, flowers, and stems by the Cd concentration in roots, and multiplying by 100.<sup id=\"rdp-ebb-cite_ref-:13_31-3\" class=\"reference\"><a href=\"#cite_note-:13-31\">[31]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:14_32-3\" class=\"reference\"><a href=\"#cite_note-:14-32\">[32]<\/a><\/sup> Cadmium uptake rates (\u00b5mol\u00b7plant<sup>\u22121<\/sup>\u00b7d<sup>\u22121<\/sup>) were calculated by the following equation (adapted from Ali <i>et al.<\/i><sup id=\"rdp-ebb-cite_ref-:12_30-3\" class=\"reference\"><a href=\"#cite_note-:12-30\">[30]<\/a><\/sup>): ((([Cd<sub>initial<\/sub> \u2212 Cd<sub>final<\/sub>])\/number of plants)\/treatment days)\/root biomass)\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Plant_growth_and_biomass_yield\">Plant growth and biomass yield<\/span><\/h3>\n<p>Plant height was determined by measuring the distance from the base of the stem to the tip of the apically dominant flower at harvest.<sup id=\"rdp-ebb-cite_ref-:1_11-1\" class=\"reference\"><a href=\"#cite_note-:1-11\">[11]<\/a><\/sup>\n<\/p><p>To quantify leaf, flower, stem, and root biomass, four whole plants per replicate were air-dried at ambient temperatures inside the greenhouse for two weeks and then separated into roots, stems, and leaf and flower biomass. Dry leaf and flower materials were manually pulled from plants following industry standards used for hemp biomass intended in cannabinoid extraction. Subsamples were taken from the air-dried materials and further dried in a forced air oven set at 55 \u00b0C for 48 hours until a constant weight was achieved. The dry weights of the whole plant samples were then normalized based on subsample moisture content.<sup id=\"rdp-ebb-cite_ref-:1_11-2\" class=\"reference\"><a href=\"#cite_note-:1-11\">[11]<\/a><\/sup>\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Cannabinoid_analysis\">Cannabinoid analysis<\/span><\/h3>\n<p>Approximately 25 g of fresh flower tissue sampled from inflorescences located on the top one-third of the plants were sampled during weeks six (DN varieties) and seven (DLS varieties) of flowering (49 and 56 days after treatment [DAT], respectively) and dried separately from other samples as follows. Flower material was placed on a perforated aluminum baking sheet and dried to approximately 15% moisture content in a walk-in cooler with a temperature set point of 13 \u00b0C and 55% relative humidity for 14 days. The appropriate relative humidity was maintained using a dehumidifier. The dried material was hand trimmed to remove leaves, sealed in a metalized resealable food bag (Uline, Braselton, GA, USA) and stored at \u22124 \u00b0C for cannabinoid analysis. The acidic and neutral forms of the cannabinoids, THC and CBD, were determined in dried flower material by a commercial <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Laboratory\" title=\"Laboratory\" class=\"wiki-link\" data-key=\"ed14e6a67b4b14ad2c190c28455725f6\">laboratory<\/a> using <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=High-performance_liquid_chromatography\" title=\"High-performance liquid chromatography\" class=\"wiki-link\" data-key=\"4c5d83ffa9785383c3eb0be7ea78dd2b\">high-performance liquid chromatography<\/a> and a <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Chromatography_detector\" title=\"Chromatography detector\" class=\"wiki-link\" data-key=\"7787a159a1c7c0a3b2cc28bbc3189587\">diode array detector<\/a> set to 230 nm (SJ Labs and Analytics, Macon, GA, USA). The limit of detection for THC and CBD was 0.02%. Total cannabinoid concentrations were calculated by the following formula: total cannabinoid = neutral + (acidic form \u00d7 0.877). The percentage of dry matter for all samples was recorded and the results were reported on a dry weight basis.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Statistical_analysis\">Statistical analysis<\/span><\/h3>\n<p>Statistical analysis was conducted using JMP Pro 15 (SAS, Cary, NC, USA). Data were subjected to a one-way ANOVA procedure with Student\u2019s <i>t<\/i>-test (<i>p<\/i> < 0.05) or Tukey\u2019s Honest Significant Difference test (<i>p<\/i> < 0.05) conducted for mean separation when appropriate. Tissue Cd and cannabinoid concentrations were log-transformed to ensure equal variance prior to statistical analysis. Non-transformed data are presented.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Results_and_discussion\">Results and discussion<\/span><\/h2>\n<h3><span class=\"mw-headline\" id=\"Plant_height_and_biomass_yield\">Plant height and biomass yield<\/span><\/h3>\n<p>Plant heights were significantly reduced by Cd exposure in DN varieties, Apricot Auto, and Alpha Explorer. Plants exposed to 2.5 mg\u00b7L<sup>\u22121<\/sup> Cd were shorter than plants in the control (0 mg\u00b7L<sup>\u22121<\/sup> Cd) treatment (Figure 1). Plant heights of the two DLS varieties, Von and T1, were not significantly affected by Cd treatment. Nevertheless, the average decline in plant height of plants exposed to Cd was 20.4% in the DLS varieties and 35.6% in the DN varieties. Exposure to Cd also significantly reduced flower and leaf, stem, and root biomass in the DN varieties, Apricot Auto, and Alpha Explorer (Table 2). In the DLS variety Von, the dry weight of flower and leaf tissues significantly decreased in the 2.5 mg\u00b7L<sup>\u22121<\/sup> Cd treatment compared to the control, while dry weights of stem and root tissues were unaffected by Cd. The dry weights of the flower and leaf, and root tissue were significantly reduced by exposure to Cd in the DLS variety T1. Whole plant biomass in all four varieties was significantly reduced by Cd treatments, with an average reduction of 74.4% in the DN varieties and 50.2% in the DLS varieties.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=File:Fig1_Marebesi_Water23_15-12.png\" class=\"image wiki-link\" data-key=\"ed826a6039d01a30c2eb50e5025b4b10\"><img alt=\"Fig1 Marebesi Water23 15-12.png\" src=\"https:\/\/www.cannaqa.wiki\/images\/8\/86\/Fig1_Marebesi_Water23_15-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Fig. 1.<\/b> Average plant height at harvest \u00b1 SE of four hemp (<i>Cannabis sativa<\/i> L.) varieties. Values correspond to averages of four replicates with four plants each. Bars associated with the same uppercase letter(s) indicate no significant differences among hemp varieties at <i>p<\/i> \u2264 0.05 according to Tukey\u2019s HSD test. Bars associated with the same lowercase letter indicate no significant differences between control and treated plants for a given hemp variety at <i>p<\/i> \u2264 0.05 according to Student\u2019s <i>t<\/i>-test.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"9\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 2.<\/b> Dry weight of flowers + leaves, stems, roots, and whole plant at harvest \u00b1 SE of four hemp (<i>Cannabis sativa<\/i> L.) varieties exposed to 0 (control) and 2.5 (treated) mg\u00b7L<sup>\u22121<\/sup> Cd in a nutrient solution. Notes: Biomass values measured as g\u00b7Plant<sup>\u22121<\/sup>. Values followed by the same uppercase letter(s) indicate no significant differences among hemp varieties for each plant tissue according to Tukey\u2019s Honest Significant Difference test (<i>p<\/i> < 0.05). Values followed by the same lowercase letter within each individual hemp variety row indicate no significant differences between control and Cd treated plants for each plant tissue according to Student\u2019s <i>t<\/i>-test (<i>p<\/i> < 0.05).\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">\n<\/th>\n<th colspan=\"2\" style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Flower + leaf\n<\/th>\n<th colspan=\"2\" style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Stems\n<\/th>\n<th colspan=\"2\" style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Roots\n<\/th>\n<th colspan=\"2\" style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Whole plant\n<\/th><\/tr>\n<tr>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Variety\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Control\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Treated\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Control\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Treated\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Control\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Treated\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Control\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Treated\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Apricot Auto\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">40.0 \u00b1 2.3 AB a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">9.9 \u00b1 2.0 B b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">17.7 \u00b1 1.2 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">6.5 \u00b1 1.0 A b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">8.3 \u00b1 0.3 B a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3.2 \u00b1 0.4 B b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">66.0 \u00b1 3.2 B a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">19.5 \u00b1 2.7 AB b\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Alpha Explorer\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">37.8 \u00b1 2.2 B a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">8.2 \u00b1 1.4 B b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">23.6 \u00b1 1.5 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">4.5 \u00b1 0.8 A b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">7.1 \u00b1 0.7 B a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2.1 \u00b1 0.4 B b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">68.4 \u00b1 3.3 B a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">14.8 \u00b1 2.4 B b\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Von\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">46.2 \u00b1 1.3 AB a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">22.2 \u00b1 7.2 AB b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">29.3 \u00b1 1.7 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">15.2 \u00b1 5.6 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">20.1 \u00b1 1.0 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">13.3 \u00b1 2.9 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">95.6 \u00b1 1.5 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">50.7 \u00b1 15.3 A b\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">T1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">48.6 \u00b1 2.4 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">28.5 \u00b1 5.6 A b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">27.7 \u00b1 6.5 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">13.9 \u00b1 2.7 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">22.9 \u00b1 2.0 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">9.7 \u00b1 2.7 AB b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">99.2 \u00b1 6.4 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">52.2 \u00b1 9.6 A b\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>The DLS varieties, Von and T1, generated significantly greater root biomass relative to the DN varieties when not exposed to Cd. In Cd-treated plants, root biomass was significantly greater in Von compared to the two DN varieties. Root biomass in T1 was not significantly different from any other variety when exposed to Cd. Flower and leaf tissue yields were highest in DLS variety T1 in both control and Cd-exposed treatments. Exposure to Cd resulted in a significant decrease in flower and leaf biomass of 75% and 78%, in Apricot Auto and Alpha Explorer DN varieties, respectively. Exposure to Cd significantly reduced flower and leaf biomass in the DLS varieties Von and T1 by 52% and 41%, respectively. Whole-plant, stem, and root biomass was also significantly reduced to a greater extent in DN varieties compared to the DLS varieties. This suggests that the DLS varieties used in the present study may be more tolerant to Cd exposure at 2.5 mg\u00b7L<sup>\u22121<\/sup> than the DN varieties evaluated. These results agree with previous studies that have reported a decrease in shoot biomass of hemp plants exposed to Cd.<sup id=\"rdp-ebb-cite_ref-:9_26-1\" class=\"reference\"><a href=\"#cite_note-:9-26\">[26]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:16_34-2\" class=\"reference\"><a href=\"#cite_note-:16-34\">[34]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:17_35-2\" class=\"reference\"><a href=\"#cite_note-:17-35\">[35]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:18_39-0\" class=\"reference\"><a href=\"#cite_note-:18-39\">[39]<\/a><\/sup>\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Cd_concentration_in_hemp_tissues\">Cd concentration in hemp tissues<\/span><\/h3>\n<p>Cd concentrations in hemp tissues were affected by plant variety and Cd treatment (Table 3). In DN varieties, Apricot Auto and Alpha Explorer, and the DLS variety Von, Cd concentrations were highest in roots. In contrast, the DLS variety, T1 had similar Cd concentrations in roots, leaf, and stem tissue upon exposure to Cd. These results are consistent with previous literature, which reported that roots were the preferred tissue for Cd accumulation in hemp<sup id=\"rdp-ebb-cite_ref-:10_27-2\" class=\"reference\"><a href=\"#cite_note-:10-27\">[27]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:16_34-3\" class=\"reference\"><a href=\"#cite_note-:16-34\">[34]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:17_35-3\" class=\"reference\"><a href=\"#cite_note-:17-35\">[35]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-40\" class=\"reference\"><a href=\"#cite_note-40\">[40]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:19_41-0\" class=\"reference\"><a href=\"#cite_note-:19-41\">[41]<\/a><\/sup> and that Cd accumulation increased with increasing Cd concentrations in the growing media.<sup id=\"rdp-ebb-cite_ref-:18_39-1\" class=\"reference\"><a href=\"#cite_note-:18-39\">[39]<\/a><\/sup> For instance, Cd concentrations in the roots of <i>C. sativa<\/i> fiber variety Santhica 27 exposed to 20 \u03bcM Cd (2.25 mg\u00b7L<sup>\u22121<\/sup> Cd) for one week averaged 2,687 mg\u00b7kg<sup>\u22121<\/sup> dw, while Cd concentrations in stems and leaves averaged 1,243 mg\u00b7kg<sup>\u22121<\/sup> dry weight (dw) and 717 mg\u00b7kg<sup>\u22121<\/sup> dw, respectively.<sup id=\"rdp-ebb-cite_ref-:16_34-4\" class=\"reference\"><a href=\"#cite_note-:16-34\">[34]<\/a><\/sup> Additionally, the <i>C. sativa<\/i> medicinal variety Purple Tiger exposed to 2.5 mg\u00b7L<sup>\u22121<\/sup> Cd for 68 days had average Cd concentrations of 1,982.6 mg\u00b7kg<sup>\u22121<\/sup> dw in roots, 13.2 mg\u00b7kg<sup>\u22121<\/sup> dw in leaves, 5.1 mg\u00b7kg<sup>\u22121<\/sup> dw in stems, and 7.6 mg\u00b7kg<sup>\u22121<\/sup> dw in flowers.<sup id=\"rdp-ebb-cite_ref-:17_35-4\" class=\"reference\"><a href=\"#cite_note-:17-35\">[35]<\/a><\/sup>\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"9\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 3.<\/b> Average cadmium (Cd) concentration at harvest in plant tissues of four hemp (<i>Cannabis sativa<\/i> L.) varieties exposed to 0 (control) and 2.5 (treated) mg\u00b7L<sup>\u22121<\/sup> Cd in a nutrient solution on a dry weight (dw) basis. Notes: Cd concentration values measured as mg\u00b7kg<sup>\u22121<\/sup> dw. ND = not detected. Values followed by the same uppercase letter(s) indicate no significant differences among hemp varieties for each plant tissue according to Tukey\u2019s Honest Significant Difference test (<i>p<\/i> < 0.05). Values followed by the same lowercase letter(s) within a hemp variety row indicate no significant differences among plant tissues for each treatment (control and treated) according to Tukey\u2019s Honest Significant Difference test (<i>p<\/i> < 0.05).\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">\n<\/th>\n<th colspan=\"2\" style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Flower\n<\/th>\n<th colspan=\"2\" style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Root\n<\/th>\n<th colspan=\"2\" style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Leaf\n<\/th>\n<th colspan=\"2\" style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Stem\n<\/th><\/tr>\n<tr>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Variety\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Control\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Treated\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Control\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Treated\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Control\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Treated\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Control\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Treated\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Apricot Auto\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1.2 A ab\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">38.1 A c\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3.8 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1056.8 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1.0 A b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">44.9 AB c\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.5 A c\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">116.3 A b\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Alpha Explorer\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.6 A ab\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">51.0 A d\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3.4 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2274.2 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3.0 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">92.5 A c\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.3 A b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">176.4 A b\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Von\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.4 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">11.9 B b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1.1 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">512.4 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2.1 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">18.2 BC b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">ND A b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1.8 C c\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">T1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">ND B a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.2 C b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">ND B a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">16.1 B a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.8 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">8.22 C a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.6 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">26.8 B a\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>All varieties had increased concentrations of Cd in all plant tissues exposed to Cd in the nutrient solution, except in the floral tissues of T1. Cadmium-treated DN varieties, Apricot Auto and Alpha Explorer, had increased Cd concentrations in flower and stem tissues, and Alpha Explorer had significantly greater Cd concentrations in leaf tissue compared to the DLS varieties. These results suggest that Cd concentration in plant tissues is negatively correlated to biomass accumulation, as the DN varieties had a higher reduction in flower and leaf biomass when exposed to 2.5 mg\u00b7L<sup>\u22121<\/sup> Cd, compared to the DLS varieties. It should be noted that detectable Cd levels were observed in the control tissues. We expected all the control tissues to be below detection and can only speculate on the source of the Cd. It could have been slight cross contamination due to the aeration system that connected all the tanks. Another possibility is the Cd was already in the plant tissues before the treatments were applied. We analyzed the propagation foam cubes and found some Cd in the material (0.25 mg\u00b7L<sup>\u22121<\/sup>), which potentially could have slightly contaminated the tanks.\n<\/p><p>Whole-plant BCF ranged from 1,051.8 in Alpha Explorer to 20.9 in T1 (Table 4 and Figure S2). In Alpha Explorer, whole-plant, root, and leaf BCF were significantly greater than the other three varieties evaluated. The BCF in stems and flowers was significantly higher in the DN varieties, Alpha Explorer and Apricot Auto, when compared to the DLS varieties, Von and T1. In Alpha Explorer, Apricot Auto, and Von varieties, BCF values were the highest in roots, while in T1 plants BCF was the highest in stems. For all varieties, the BCF in floral tissue was significantly less than that of roots, except for T1, which accumulated most Cd in stem tissues instead of roots. Previous studies reported that BCF values were consistently higher in roots when compared to above-ground biomass.<sup id=\"rdp-ebb-cite_ref-:18_39-2\" class=\"reference\"><a href=\"#cite_note-:18-39\">[39]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:19_41-1\" class=\"reference\"><a href=\"#cite_note-:19-41\">[41]<\/a><\/sup> Furthermore, in a review of the capacity of different varieties of <i>C. sativa<\/i> to accumulate heavy metals, root BCF values in plants exposed to different Cd treatments ranged from 0.08 to 30.99.<sup id=\"rdp-ebb-cite_ref-:5_17-2\" class=\"reference\"><a href=\"#cite_note-:5-17\">[17]<\/a><\/sup> Most prior studies utilized contaminated soil or a soil-like substrate, which makes it challenging to determine the exact concentration of Cd that was available to plants. Nevertheless, our results support the previous literature demonstrating that plant genetics play a role in the Cd tolerance and accumulation potential of hemp plants.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"7\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 4.<\/b> Translocation factor (TF) and bioconcentration factor (BCF) in whole plants, roots, leaves, stems, and flowers of the hemp (<i>Cannabis sativa<\/i> L.) varieties Apricot Auto, Alpha Explorer, Von, and T1 exposed to 2.5 mg\u00b7L<sup>\u22121<\/sup> Cd in a nutrient solution. Notes: Among the BCF values, those values followed by the same uppercase letter(s) within a column indicate no significant differences among hemp varieties for a plant tissue according to Tukey\u2019s Honest Significant Difference test (<i>p<\/i> < 0.05). Values followed by the same lowercase letter within each individual hemp variety row indicate no significant differences among plant tissues (root, leaf, stem, and flower) according to Tukey\u2019s Honest Significant Difference test (<i>p<\/i> < 0.05).\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">\n<\/th>\n<th colspan=\"5\" style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">BCF\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">TF (%)\n<\/th><\/tr>\n<tr>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Variety\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Whole plant\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Root\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Leaf\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Stem\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Flower\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Whole plant\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Apricot Auto\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">527.5 B\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">446.1 B a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">18.3 B b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">47.6 AB b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">15.5 A b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">28.5 B\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Alpha Explorer\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1,051.8 A\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">920.1 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">37.9 A b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">73.2 A b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">20.6 A b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">14.3 B\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Von\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">213.8 BC\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">201.4 BC a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">7.0 BC b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.7 C b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">4.8 B b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">6.5 B\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">T1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">20.9 C\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">6.2 C ab\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3.4 C b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">11.2 BC a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.1 B b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">193.0 A\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>The TF was calculated as the ratio between Cd concentration in aboveground tissue (stems, leaves, and flowers) and Cd concentration in roots. The variety T1 had the highest TF compared to the other three varieties (Table 4). Our results suggest T1 had the highest translocation factor due to increased Cd accumulation in stems, and relatively low accumulation in roots compared to other varieties. Therefore, this DLS variety may favor Cd sequestration in aboveground parts of the plant while the other three varieties accumulated more Cd in the roots. The tolerance index, which was calculated as the ratio between biomass accumulation (whole plant) in the 2.5 mg\u00b7L<sup>\u22121<\/sup> treatment and biomass accumulation in the control treatment, was not significantly different among hemp varieties (data not shown).\n<\/p><p>The cadmium uptake rate was significantly greater in Alpha Explorer compared to the other three hemp varieties (Figure 2). Both BCF and Cd uptake rates were the highest in Alpha Explorer, suggesting that this variety was the most efficient in taking up Cd from the solution, accumulating it primarily in the roots but also translocating it to the shoots. However, Alpha Explorer had significantly lower whole-plant biomass than the DLS varieties, Von and T1, and biomass is considered a more accurate indicator of Cd toxicity than plant height.<sup id=\"rdp-ebb-cite_ref-:18_39-3\" class=\"reference\"><a href=\"#cite_note-:18-39\">[39]<\/a><\/sup> In a typical field production scenario, DN varieties would generally have a shorter life cycle than the DLS varieties and significant differences in growing degree-day requirements for maturity in DN and DLS hemp varieties have been reported.<sup id=\"rdp-ebb-cite_ref-:1_11-3\" class=\"reference\"><a href=\"#cite_note-:1-11\">[11]<\/a><\/sup> Although all plants in the present study were harvested at the same time, the increased maturation rate of the DN varieties likely led to increased rates of Cd uptake. Reduced Cd uptake rates in Von and T1 varieties could potentially allow these plants to better cope with Cd exposure.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=File:Fig2_Marebesi_Water23_15-12.png\" class=\"image wiki-link\" data-key=\"4001bc2963da12fa2938331487e0ffa5\"><img alt=\"Fig2 Marebesi Water23 15-12.png\" src=\"https:\/\/www.cannaqa.wiki\/images\/2\/20\/Fig2_Marebesi_Water23_15-12.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Fig. 2.<\/b> Average Cd uptake rates \u00b1 SE in four hemp (<i>Cannabis sativa<\/i> L.) varieties exposed to 2.5 mg\u00b7L<sup>\u22121<\/sup> Cd. Bars associated with the same letter(s) indicate no significant differences among hemp varieties at <i>p<\/i> \u2264 0.05 according to Tukey\u2019s HSD test.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>Although the TF was the highest in T1, the other three hemp varieties had significant TF values within or above the range of approximately 2.5 to 12% that has been reported in previous studies<sup id=\"rdp-ebb-cite_ref-:18_39-4\" class=\"reference\"><a href=\"#cite_note-:18-39\">[39]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:19_41-2\" class=\"reference\"><a href=\"#cite_note-:19-41\">[41]<\/a><\/sup>, suggesting that hemp may tolerate Cd stress (Table 4). Furthermore, the TF of all hemp varieties was above 1% and, therefore, these varieties could be classified as high-efficiency plants for metal translocation from roots to above-ground organs.<sup id=\"rdp-ebb-cite_ref-42\" class=\"reference\"><a href=\"#cite_note-42\">[42]<\/a><\/sup> Although BCF was above 100 mg\u00b7kg<sup>\u22121<\/sup> DW in leaves, which is the minimal value for hyperaccumulator plants<sup id=\"rdp-ebb-cite_ref-:3_14-2\" class=\"reference\"><a href=\"#cite_note-:3-14\">[14]<\/a><\/sup>, whole plant biomass was reduced by Cd exposure. Therefore, these hemp varieties might not be candidates to be considered Cd hyperaccumulators as previously indicated.<sup id=\"rdp-ebb-cite_ref-:16_34-5\" class=\"reference\"><a href=\"#cite_note-:16-34\">[34]<\/a><\/sup> Nevertheless, our data suggest that hemp characteristics related to metal uptake and distribution within plant tissues can fluctuate by variety.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Nutrient_partitioning\">Nutrient partitioning<\/span><\/h3>\n<p>The concentrations of the macronutrients N, P, K, and Ca were affected by both plant variety and Cd treatment (Table S2). However, no clear trends for any macronutrients were observed in response to Cd or among varieties. Previous data on nutrient distribution under metal stress is contradictory. For instance, the high-THC <i>C. sativa<\/i> variety \u201cNB100\u201d growing in uncontaminated substrate supplied with a commercial fertilizer (65, 17, 90 ppm N, P, and K, respectively) had higher N, P, and K concentrations in flower tissue, when compared to inflorescence leaves, fan leaves, and stems (roots were not analyzed), while Ca concentration was higher in fan leaves.<sup id=\"rdp-ebb-cite_ref-43\" class=\"reference\"><a href=\"#cite_note-43\">[43]<\/a><\/sup> Conversely, there was a significant reduction in N, P, K, and Ca content in edible parts of tomato and lettuce grown in spiked soil containing 2.5 mg\u00b7kg<sup>\u22121<\/sup> Cd, when compared to plants grown in non-contaminated soil.<sup id=\"rdp-ebb-cite_ref-44\" class=\"reference\"><a href=\"#cite_note-44\">[44]<\/a><\/sup> Furthermore, chickpea plants (<i>Cicer arietinum<\/i> L.) exposed to Cd in nutrient solution (approximately 68 mg\u00b7L<sup>\u22121<\/sup> Cd) showed a significant decrease in root and shoot Ca concentrations when compared to plants growing in an uncontaminated nutrient solution.<sup id=\"rdp-ebb-cite_ref-45\" class=\"reference\"><a href=\"#cite_note-45\">[45]<\/a><\/sup> The N, P, and K concentrations in the shoots of <i>Pfaffia glomerata<\/i> were reported to increase with increasing Cd concentrations in the growing media.<sup id=\"rdp-ebb-cite_ref-:20_46-0\" class=\"reference\"><a href=\"#cite_note-:20-46\">[46]<\/a><\/sup> These macronutrients are involved in the synthesis of Cd-detoxifying chelator molecules, such as glutathione and phytochelatins, and in the increase in activity of antioxidant enzymes, such as superoxide dismutase (SOD), catalase (CAT), and ascorbate peroxidade (APX).<sup id=\"rdp-ebb-cite_ref-:20_46-1\" class=\"reference\"><a href=\"#cite_note-:20-46\">[46]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-47\" class=\"reference\"><a href=\"#cite_note-47\">[47]<\/a><\/sup>\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Total_THC_and_CBD_in_hemp_flower\">Total THC and CBD in hemp flower<\/span><\/h3>\n<p>Total CBD concentrations were significantly lower in Apricot Auto and Alpha Explorer plants exposed to 2.5 mg\u00b7L<sup>\u22121<\/sup> Cd compared to the 0 mg\u00b7L<sup>\u22121<\/sup> Cd treatment, while total CBD concentrations in T1 and Von plants were not significantly affected by Cd exposure (Table 5). Similarly, total THC was significantly lower (below the detection limit) in Apricot Auto and Alpha Explorer plants exposed to 2.5 mg\u00b7L<sup>\u22121<\/sup> Cd, compared to the control. Interestingly, THC concentrations in the DLS varieties, Von and T1, were not significantly affected by Cd exposure. Both DN varieties exhibited a significant reduction in total THC and CBD when exposed to Cd, while the DLS varieties did not. In addition, DN varieties had significantly greater Cd uptake rates (Figure 2) and BCF (Figure S2) than DLS plants. Further, both DN varieties had significantly greater Cd accumulation in the floral tissue compared to the DLS varieties (Table 3). This suggests that the increased Cd uptake and accumulation in the floral tissue in Apricot Auto and Alpha Explorer may have led to a decrease in cannabinoid synthesis. A previous study<sup id=\"rdp-ebb-cite_ref-:11_28-2\" class=\"reference\"><a href=\"#cite_note-:11-28\">[28]<\/a><\/sup> reported no differences in THC content in leaves of the DLS fiber hemp variety \u201cFibranova\u201d grown in substrate contaminated with 25 and 100 \u03bcg\u00b7g<sup>\u22121<\/sup> Cd. Previous results suggest that plant genetics might play a role in cannabinoid synthesis under metal stress.<sup id=\"rdp-ebb-cite_ref-:11_28-3\" class=\"reference\"><a href=\"#cite_note-:11-28\">[28]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-48\" class=\"reference\"><a href=\"#cite_note-48\">[48]<\/a><\/sup>\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"5\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 5.<\/b> Average total CBD and THC \u00b1 SE in the hemp (<i>Cannabis sativa<\/i> L.) varieties Apricot Auto, Alpha Explorer, Von, and T1 exposed to 0 (control) and 2.5 (treated) mg\u00b7L<sup>\u22121<\/sup> Cd in a nutrient solution. Note: Values of concentration in flowers are measured as percentage of dry weight (% dw). ND = not detected. Values followed by the same uppercase letter(s) within a column indicate no significant differences among hemp varieties according to Tukey\u2019s Honest Significant Difference test (<i>p<\/i> < 0.05). Values followed by the same lowercase letter within a hemp variety row indicate no significant differences between control and treated plants according to Student\u2019s <i>t<\/i>-test (<i>p<\/i> < 0.05). ND = not detected.\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">\n<\/th>\n<th colspan=\"4\" style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Concentration in flowers\n<\/th><\/tr>\n<tr>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">\n<\/th>\n<th colspan=\"2\" style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Total THC\n<\/th>\n<th colspan=\"2\" style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Total CBD\n<\/th><\/tr>\n<tr>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Variety\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Control\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Treated\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Control\n<\/th>\n<th style=\"background-color:#dddddd; padding-left:10px; padding-right:10px;\">Treated\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Apricot Auto\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.53 \u00b1 0.05 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">ND B b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">9.18 \u00b1 0.69 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1.82 \u00b1 0.86 B b\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Alpha Explorer\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.36 \u00b1 0.01 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">ND B b\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">8.37 \u00b1 0.30 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1.18 \u00b1 0.84 B b\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Von\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.53 \u00b1 0.07 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.43 \u00b1 0.04 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">11.38 \u00b1 1.29 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">9.44 \u00b1 1.31 A a\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">T1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.43 \u00b1 0.04 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.36 \u00b1 0.05 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">10.19 \u00b1 0.90 A a\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">7.64 \u00b1 1.04 A a\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>While total THC concentrations in all hemp varieties in the control treatment reached concentrations above the legal threshold (0.3%), this is not uncommon in both DLS and DN high-CBD hemp varieties<sup id=\"rdp-ebb-cite_ref-:1_11-4\" class=\"reference\"><a href=\"#cite_note-:1-11\">[11]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-49\" class=\"reference\"><a href=\"#cite_note-49\">[49]<\/a><\/sup> as age, genetics, and environmental factors may impact cannabinoid synthesis.<sup id=\"rdp-ebb-cite_ref-50\" class=\"reference\"><a href=\"#cite_note-50\">[50]<\/a><\/sup> These results indicate that the impact of Cd stress on CBD and THC synthesis is variety-dependent, and exposure to 2.5 mg\u00b7L<sup>\u22121<\/sup> Cd may affect cannabinoid synthesis in some varieties of <i>C. sativa<\/i>.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Conclusions\">Conclusions<\/span><\/h2>\n<p>The impact of Cd on plant growth as well as BCF, uptake rate, and TF were affected by variety. Whole plant biomass yield in all four varieties was significantly reduced by the Cd treatment, suggesting that the hemps studied here may not be classified as hyperaccumulators as they may accumulate Cd with other ions in a nutrient solution until it becomes toxic. While Cd concentration was significantly higher in roots, all four varieties were efficient in translocating Cd from roots to shoots, with Cd concentrations in flowers ranging from 0.2 to 51 mg\u00b7kg<sup>\u22121<\/sup> Cd DW in T1 and Alpha Explorer varieties, respectively. Flower and leaf biomass were significantly reduced in all four varieties in response to Cd. Further, the DN varieties, Alpha Explorer and Apricot Auto, had a significant decrease in total THC and CBD concentrations in plants exposed to Cd when compared to plants in the control treatment, while the DLS varieties did not. Additional studies are warranted to determine if there are different Cd tolerance mechanisms in DN compared to DLS hemp varieties. All four hemp varieties analyzed in this study are suitable for the medicinal market, given that heavy metal testing is conducted throughout production and on finished consumer products.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Supplementary_materials\">Supplementary materials<\/span><\/h2>\n<p>The following supporting information can be downloaded at <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.mdpi.com\/article\/10.3390\/w15122176\/s1\" target=\"_blank\">https:\/\/www.mdpi.com\/article\/10.3390\/w15122176\/s1<\/a>: \n<\/p>\n<ul><li>Table S1: Mean concentrations of cadmium (Cd) in nutrient solutions measured at the beginning and end of each cycle and daily uptake. Values are averages \u00b1 SE of four replications per treatment.<\/li>\n<li>Table S2: Average concentrations of nitrogen (N), phosphorus (P), potassium (K), and calcium (Ca) among different tissues in the hemp (<i>Cannabis sativa<\/i> L.) varieties Apricot Auto, Alpha Explorer, Von, and T1 on a dry weight (dw) basis exposed to 0 (control) and 2.5 (treated) mg\u00b7L<sup>\u22121<\/sup> Cd in a nutrient solution.<\/li>\n<li>Table S3: ICP-OES parameters.<\/li>\n<li>Table S4. ICP-OES wavelengths used.<\/li>\n<li>Figure S1: Calibration curve of cadmium on ICP-OES.<\/li>\n<li>Figure S2. Bioconcentration factor (BCF) for Cd \u00b1 SE in plant tissues of four hemp (<i>Cannabis sativa<\/i> L.) varieties exposed to 2.5 mg\u00b7L<sup>\u22121<\/sup> Cd.<\/li><\/ul>\n<h2><span id=\"rdp-ebb-Abbreviations,_acronyms,_and_initialisms\"><\/span><span class=\"mw-headline\" id=\"Abbreviations.2C_acronyms.2C_and_initialisms\">Abbreviations, acronyms, and initialisms<\/span><\/h2>\n<ul><li><b>AF<\/b>: accumulation factor<\/li>\n<li><b>Al<\/b>: aluminum<\/li>\n<li><b>APX<\/b>: ascorbate peroxidade<\/li>\n<li><b>B<\/b>: boron<\/li>\n<li><b>BAC<\/b>: biological absorption coefficient<\/li>\n<li><b>BCF<\/b>: bioconcentration factor<\/li>\n<li><b>Ca<\/b>: calcium<\/li>\n<li><b>CAT<\/b>: catalase<\/li>\n<li><b>CBD<\/b>: cannabidiol<\/li>\n<li><b>Cd<\/b>: cadmium<\/li>\n<li><b>Cu<\/b>: copper<\/li>\n<li><b>DAT<\/b>: days after treatment<\/li>\n<li><b>DLS<\/b>: day-length-sensitive<\/li>\n<li><b>DN<\/b>: day-neutral<\/li>\n<li><b>DW<\/b>: dry weight<\/li>\n<li><b>EC<\/b>: electrical conductivity<\/li>\n<li><b>Fe<\/b>: iron<\/li>\n<li><b>IBA<\/b>: index of bioaccumulation<\/li>\n<li><b>ICP-OES<\/b>: inductively coupled plasma\u2013optical emission spectroscopy<\/li>\n<li><b>K<\/b>: potassium<\/li>\n<li><b>Mg<\/b>: magnesium<\/li>\n<li><b>Mn<\/b>: manganese<\/li>\n<li><b>N<\/b>: nitrogen<\/li>\n<li><b>Ni<\/b>: nickel<\/li>\n<li><b>P<\/b>: phosphorous<\/li>\n<li><b>PTMI<\/b>: provisional tolerable monthly intake<\/li>\n<li><b>RH<\/b>: relative humidity<\/li>\n<li><b>S<\/b>: sulfur<\/li>\n<li><b>SOD<\/b>: superoxide dismutase<\/li>\n<li><b>TF<\/b>: translocation factor<\/li>\n<li><b>THC<\/b>: tetrahydrocannabinol<\/li>\n<li><b>TI<\/b>: tolerance index<\/li>\n<li><b>Zn<\/b>: zinc<\/li><\/ul>\n<h2><span class=\"mw-headline\" id=\"Acknowledgements\">Acknowledgements<\/span><\/h2>\n<h3><span class=\"mw-headline\" id=\"Author_contributions\">Author contributions<\/span><\/h3>\n<p>Conceptualization, A.O.M. and T.W.C.; methodology, A.O.M. and T.W.C.; software, A.O.M. and T.W.C.; validation, T.W.C. and J.T.L.; formal analysis, A.O.M. and T.W.C.; investigation, A.O.M. and T.W.C.; resources, T.W.C. and J.T.L.; data curation, A.O.M. and T.W.C.; writing\u2014original draft preparation, A.O.M. and T.W.C.; writing\u2014review and editing, T.W.C. and J.T.L.; visualization, A.O.M. and T.W.C.; supervision, T.W.C.; project administration, T.W.C.; funding acquisition, T.W.C. All authors have read and agreed to the published version of the manuscript.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Funding\">Funding<\/span><\/h3>\n<p>This research received no external funding.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Data_availability\">Data availability<\/span><\/h3>\n<p>The data presented in this study are available in the article and supplementary material.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Conflicts_of_interest\">Conflicts of interest<\/span><\/h3>\n<p>The authors declare no conflict of interest.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"References\">References<\/span><\/h2>\n<div class=\"reflist references-column-width\" style=\"-moz-column-width: 30em; -webkit-column-width: 30em; column-width: 30em; list-style-type: decimal;\">\n<div class=\"mw-references-wrap mw-references-columns\"><ol class=\"references\">\n<li id=\"cite_note-1\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-1\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation book\">Clarke, Robert Connell; Merlin, Mark David (2013). <i>Cannabis: evolution and ethnobotany<\/i>. Berkeley: University of California Press. pp. 29\u201357. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-0-520-27048-0.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=Cannabis%3A+evolution+and+ethnobotany&rft.aulast=Clarke&rft.aufirst=Robert+Connell&rft.au=Clarke%2C%26%2332%3BRobert+Connell&rft.au=Merlin%2C%26%2332%3BMark+David&rft.date=2013&rft.pages=pp.%26nbsp%3B29%E2%80%9357&rft.place=Berkeley&rft.pub=University+of+California+Press&rft.isbn=978-0-520-27048-0&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-2\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-2\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Cherney, Jerome; Small, Ernest (12 November 2016). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.mdpi.com\/2073-4395\/6\/4\/58\" target=\"_blank\">\"Industrial Hemp in North America: Production, Politics and Potential\"<\/a> (in en). <i>Agronomy<\/i> <b>6<\/b> (4): 58. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3390%2Fagronomy6040058\" target=\"_blank\">10.3390\/agronomy6040058<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2073-4395\" target=\"_blank\">2073-4395<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.mdpi.com\/2073-4395\/6\/4\/58\" target=\"_blank\">http:\/\/www.mdpi.com\/2073-4395\/6\/4\/58<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Industrial+Hemp+in+North+America%3A+Production%2C+Politics+and+Potential&rft.jtitle=Agronomy&rft.aulast=Cherney&rft.aufirst=Jerome&rft.au=Cherney%2C%26%2332%3BJerome&rft.au=Small%2C%26%2332%3BErnest&rft.date=12+November+2016&rft.volume=6&rft.issue=4&rft.pages=58&rft_id=info:doi\/10.3390%2Fagronomy6040058&rft.issn=2073-4395&rft_id=http%3A%2F%2Fwww.mdpi.com%2F2073-4395%2F6%2F4%2F58&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-3\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-3\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">U.S Department of Agriculture (2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.govinfo.gov\/app\/details\/FR-2019-10-31\/2019-23749\" target=\"_blank\">\"Establishment of A Domestic Hemp Production Program\"<\/a>. <i>Federal Register<\/i> <b>84<\/b> (211): 58522\u201358564<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.govinfo.gov\/app\/details\/FR-2019-10-31\/2019-23749\" target=\"_blank\">https:\/\/www.govinfo.gov\/app\/details\/FR-2019-10-31\/2019-23749<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Establishment+of+A+Domestic+Hemp+Production+Program&rft.jtitle=Federal+Register&rft.aulast=U.S+Department+of+Agriculture&rft.au=U.S+Department+of+Agriculture&rft.date=2019&rft.volume=84&rft.issue=211&rft.pages=58522%E2%80%9358564&rft_id=https%3A%2F%2Fwww.govinfo.gov%2Fapp%2Fdetails%2FFR-2019-10-31%2F2019-23749&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-4\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-4\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Clarke, Robert C.; Merlin, Mark D. (1 November 2016). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/07352689.2016.1267498\" target=\"_blank\">\"Cannabis Domestication, Breeding History, Present-day Genetic Diversity, and Future Prospects\"<\/a> (in en). <i>Critical Reviews in Plant Sciences<\/i> <b>35<\/b> (5-6): 293\u2013327. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1080%2F07352689.2016.1267498\" target=\"_blank\">10.1080\/07352689.2016.1267498<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0735-2689\" target=\"_blank\">0735-2689<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/07352689.2016.1267498\" target=\"_blank\">https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/07352689.2016.1267498<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Cannabis+Domestication%2C+Breeding+History%2C+Present-day+Genetic+Diversity%2C+and+Future+Prospects&rft.jtitle=Critical+Reviews+in+Plant+Sciences&rft.aulast=Clarke&rft.aufirst=Robert+C.&rft.au=Clarke%2C%26%2332%3BRobert+C.&rft.au=Merlin%2C%26%2332%3BMark+D.&rft.date=1+November+2016&rft.volume=35&rft.issue=5-6&rft.pages=293%E2%80%93327&rft_id=info:doi\/10.1080%2F07352689.2016.1267498&rft.issn=0735-2689&rft_id=https%3A%2F%2Fwww.tandfonline.com%2Fdoi%2Ffull%2F10.1080%2F07352689.2016.1267498&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:0-5\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:0_5-0\">5.0<\/a><\/sup> <sup><a href=\"#cite_ref-:0_5-1\">5.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation book\">Potter, D.J. (2014). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/881139510\" target=\"_blank\">\"4. Cannabis Horticulture\"<\/a>. In Pertwee, R. G.. <i>Handbook of cannabis<\/i>. Handbooks in psychopharmacology (1st ed ed.). Oxford, United Kingdom ; New York, NY: Oxford University Press. pp. 65\u201388. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-0-19-966268-5. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Online_Computer_Library_Center\" data-key=\"b53206e2204c7e657858a88b56c8ac4a\">OCLC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/oclc\/881139510\" target=\"_blank\">881139510<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/881139510\" target=\"_blank\">https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/881139510<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=4.+Cannabis+Horticulture&rft.atitle=Handbook+of+cannabis&rft.aulast=Potter%2C+D.J.&rft.au=Potter%2C+D.J.&rft.date=2014&rft.series=Handbooks+in+psychopharmacology&rft.pages=pp.%26nbsp%3B65%E2%80%9388&rft.edition=1st+ed&rft.place=Oxford%2C+United+Kingdom+%3B+New+York%2C+NY&rft.pub=Oxford+University+Press&rft.isbn=978-0-19-966268-5&rft_id=info:oclcnum\/881139510&rft_id=https%3A%2F%2Fwww.worldcat.org%2Ftitle%2Fmediawiki%2Foclc%2F881139510&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-6\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-6\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation book\">Ranalli, Paolo, ed. (1999). <i>Advances in hemp research<\/i>. New York: Food Products Press. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-56022-872-1.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=Advances+in+hemp+research&rft.date=1999&rft.place=New+York&rft.pub=Food+Products+Press&rft.isbn=978-1-56022-872-1&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-7\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-7\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Amaducci, Stefano; Colauzzi, Michele; Zatta, Alessandro; Venturi, Gianpietro (23 April 2008). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.tandfonline.com\/doi\/abs\/10.1080\/15377880801898691\" target=\"_blank\">\"Flowering Dynamics in Monoecious and Dioecious Hemp Genotypes\"<\/a> (in en). <i>Journal of Industrial Hemp<\/i> <b>13<\/b> (1): 5\u201319. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1080%2F15377880801898691\" target=\"_blank\">10.1080\/15377880801898691<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1537-7881\" target=\"_blank\">1537-7881<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.tandfonline.com\/doi\/abs\/10.1080\/15377880801898691\" target=\"_blank\">http:\/\/www.tandfonline.com\/doi\/abs\/10.1080\/15377880801898691<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Flowering+Dynamics+in+Monoecious+and+Dioecious+Hemp+Genotypes&rft.jtitle=Journal+of+Industrial+Hemp&rft.aulast=Amaducci&rft.aufirst=Stefano&rft.au=Amaducci%2C%26%2332%3BStefano&rft.au=Colauzzi%2C%26%2332%3BMichele&rft.au=Zatta%2C%26%2332%3BAlessandro&rft.au=Venturi%2C%26%2332%3BGianpietro&rft.date=23+April+2008&rft.volume=13&rft.issue=1&rft.pages=5%E2%80%9319&rft_id=info:doi\/10.1080%2F15377880801898691&rft.issn=1537-7881&rft_id=http%3A%2F%2Fwww.tandfonline.com%2Fdoi%2Fabs%2F10.1080%2F15377880801898691&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-8\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-8\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Hall, Jack; Bhattarai, Surya P.; Midmore, David J. (1 January 2012). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.tandfonline.com\/doi\/abs\/10.1080\/15440478.2012.651848\" target=\"_blank\">\"Review of Flowering Control in Industrial Hemp\"<\/a> (in en). <i>Journal of Natural Fibers<\/i> <b>9<\/b> (1): 23\u201336. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1080%2F15440478.2012.651848\" target=\"_blank\">10.1080\/15440478.2012.651848<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1544-0478\" target=\"_blank\">1544-0478<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.tandfonline.com\/doi\/abs\/10.1080\/15440478.2012.651848\" target=\"_blank\">http:\/\/www.tandfonline.com\/doi\/abs\/10.1080\/15440478.2012.651848<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Review+of+Flowering+Control+in+Industrial+Hemp&rft.jtitle=Journal+of+Natural+Fibers&rft.aulast=Hall&rft.aufirst=Jack&rft.au=Hall%2C%26%2332%3BJack&rft.au=Bhattarai%2C%26%2332%3BSurya+P.&rft.au=Midmore%2C%26%2332%3BDavid+J.&rft.date=1+January+2012&rft.volume=9&rft.issue=1&rft.pages=23%E2%80%9336&rft_id=info:doi\/10.1080%2F15440478.2012.651848&rft.issn=1544-0478&rft_id=http%3A%2F%2Fwww.tandfonline.com%2Fdoi%2Fabs%2F10.1080%2F15440478.2012.651848&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-9\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-9\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation book\">Rosenthal, E. (2010). <i>The Big Book of Buds: More Marijuana Varieties from the World\u2019s Great Seed Breeders<\/i>. <b>4<\/b>. Quick Trading Co.. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 9781936807031.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=The+Big+Book+of+Buds%3A+More+Marijuana+Varieties+from+the+World%E2%80%99s+Great+Seed+Breeders&rft.aulast=Rosenthal%2C+E.&rft.au=Rosenthal%2C+E.&rft.date=2010&rft.volume=4&rft.pub=Quick+Trading+Co.&rft.isbn=9781936807031&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-10\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-10\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation book\">Small, Ernest (2017). <i>Cannabis: a complete guide<\/i>. Boca Raton: CRC Press Taylor & Francis Group. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-1-4987-6163-5.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=Cannabis%3A+a+complete+guide&rft.aulast=Small&rft.aufirst=Ernest&rft.au=Small%2C%26%2332%3BErnest&rft.date=2017&rft.place=Boca+Raton&rft.pub=CRC+Press+Taylor+%26+Francis+Group&rft.isbn=978-1-4987-6163-5&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:1-11\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:1_11-0\">11.0<\/a><\/sup> <sup><a href=\"#cite_ref-:1_11-1\">11.1<\/a><\/sup> <sup><a href=\"#cite_ref-:1_11-2\">11.2<\/a><\/sup> <sup><a href=\"#cite_ref-:1_11-3\">11.3<\/a><\/sup> <sup><a href=\"#cite_ref-:1_11-4\">11.4<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Coolong, Timothy; Cassity-Duffey, Kate; Joy, Noelle (1 February 2023). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/journals.ashs.org\/view\/journals\/horttech\/33\/1\/article-p138.xml\" target=\"_blank\">\"Role of Planting Date on Yield and Cannabinoid Content of Day-neutral and Photoperiod-sensitive Hemp in Georgia, USA\"<\/a>. <i>HortTechnology<\/i> <b>33<\/b> (1): 138\u2013145. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.21273%2FHORTTECH05151-22\" target=\"_blank\">10.21273\/HORTTECH05151-22<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1063-0198\" target=\"_blank\">1063-0198<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/journals.ashs.org\/view\/journals\/horttech\/33\/1\/article-p138.xml\" target=\"_blank\">https:\/\/journals.ashs.org\/view\/journals\/horttech\/33\/1\/article-p138.xml<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Role+of+Planting+Date+on+Yield+and+Cannabinoid+Content+of+Day-neutral+and+Photoperiod-sensitive+Hemp+in+Georgia%2C+USA&rft.jtitle=HortTechnology&rft.aulast=Coolong&rft.aufirst=Timothy&rft.au=Coolong%2C%26%2332%3BTimothy&rft.au=Cassity-Duffey%2C%26%2332%3BKate&rft.au=Joy%2C%26%2332%3BNoelle&rft.date=1+February+2023&rft.volume=33&rft.issue=1&rft.pages=138%E2%80%93145&rft_id=info:doi\/10.21273%2FHORTTECH05151-22&rft.issn=1063-0198&rft_id=https%3A%2F%2Fjournals.ashs.org%2Fview%2Fjournals%2Fhorttech%2F33%2F1%2Farticle-p138.xml&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:2-12\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:2_12-0\">12.0<\/a><\/sup> <sup><a href=\"#cite_ref-:2_12-1\">12.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation\" id=\"rdp-ebb-CITEREFChaneyBaklanov2017\">Chaney, Rufus L.; Baklanov, Ilya A. (2017), <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0065229616301252\" target=\"_blank\">\"Phytoremediation and Phytomining\"<\/a> (in en), <i>Advances in Botanical Research<\/i> (Elsevier) <b>83<\/b>: 189\u2013221, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fbs.abr.2016.12.006\" target=\"_blank\">10.1016\/bs.abr.2016.12.006<\/a>, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-0-12-802853-7<span class=\"printonly\">, <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0065229616301252\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0065229616301252<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 2023-09-27<\/span><\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Phytoremediation+and+Phytomining&rft.jtitle=Advances+in+Botanical+Research&rft.aulast=Chaney&rft.aufirst=Rufus+L.&rft.au=Chaney%2C%26%2332%3BRufus+L.&rft.au=Baklanov%2C%26%2332%3BIlya+A.&rft.date=2017&rft.volume=83&rft.pages=189%E2%80%93221&rft.pub=Elsevier&rft_id=info:doi\/10.1016%2Fbs.abr.2016.12.006&rft.isbn=978-0-12-802853-7&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0065229616301252&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-13\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-13\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation book\">Nesler, A.; Furini, A. (2012). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/795742388\" target=\"_blank\">\"Phytoremediation: The utilization of Plants to Reclaim Polluted Sites\"<\/a>. In Furini, Antonella. <i>Plants and heavy metals<\/i>. SpringerBriefs in molecular science. Biometals. Dordrecht ; New York: Springer. pp. 75\u201386. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-94-007-4440-0. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Online_Computer_Library_Center\" data-key=\"b53206e2204c7e657858a88b56c8ac4a\">OCLC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/oclc\/795742388\" target=\"_blank\">795742388<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/795742388\" target=\"_blank\">https:\/\/www.worldcat.org\/title\/mediawiki\/oclc\/795742388<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Phytoremediation%3A+The+utilization+of+Plants+to+Reclaim+Polluted+Sites&rft.atitle=Plants+and+heavy+metals&rft.aulast=Nesler%2C+A.%3B+Furini%2C+A.&rft.au=Nesler%2C+A.%3B+Furini%2C+A.&rft.date=2012&rft.series=SpringerBriefs+in+molecular+science.+Biometals&rft.pages=pp.%26nbsp%3B75%E2%80%9386&rft.place=Dordrecht+%3B+New+York&rft.pub=Springer&rft.isbn=978-94-007-4440-0&rft_id=info:oclcnum\/795742388&rft_id=https%3A%2F%2Fwww.worldcat.org%2Ftitle%2Fmediawiki%2Foclc%2F795742388&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:3-14\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:3_14-0\">14.0<\/a><\/sup> <sup><a href=\"#cite_ref-:3_14-1\">14.1<\/a><\/sup> <sup><a href=\"#cite_ref-:3_14-2\">14.2<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation\" id=\"rdp-ebb-CITEREFGreger2004\">Greger, Maria (2004), Prasad, M. N. V., ed., <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/978-3-662-07743-6_1\" target=\"_blank\">\"Metal Availability, Uptake, Transport and Accumulation in Plants\"<\/a> (in en), <i>Heavy Metal Stress in Plants<\/i> (Berlin, Heidelberg: Springer Berlin Heidelberg): 1\u201327, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2F978-3-662-07743-6_1\" target=\"_blank\">10.1007\/978-3-662-07743-6_1<\/a>, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-3-642-07268-0<span class=\"printonly\">, <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/978-3-662-07743-6_1\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/978-3-662-07743-6_1<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 2023-09-27<\/span><\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Metal+Availability%2C+Uptake%2C+Transport+and+Accumulation+in+Plants&rft.jtitle=Heavy+Metal+Stress+in+Plants&rft.aulast=Greger&rft.aufirst=Maria&rft.au=Greger%2C%26%2332%3BMaria&rft.date=2004&rft.pages=1%E2%80%9327&rft.place=Berlin%2C+Heidelberg&rft.pub=Springer+Berlin+Heidelberg&rft_id=info:doi\/10.1007%2F978-3-662-07743-6_1&rft.isbn=978-3-642-07268-0&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2F978-3-662-07743-6_1&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-15\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-15\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Kirkham, M.B. (1 December 2006). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0016706106002540\" target=\"_blank\">\"Cadmium in plants on polluted soils: Effects of soil factors, hyperaccumulation, and amendments\"<\/a> (in en). <i>Geoderma<\/i> <b>137<\/b> (1-2): 19\u201332. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.geoderma.2006.08.024\" target=\"_blank\">10.1016\/j.geoderma.2006.08.024<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0016706106002540\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0016706106002540<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Cadmium+in+plants+on+polluted+soils%3A+Effects+of+soil+factors%2C+hyperaccumulation%2C+and+amendments&rft.jtitle=Geoderma&rft.aulast=Kirkham&rft.aufirst=M.B.&rft.au=Kirkham%2C%26%2332%3BM.B.&rft.date=1+December+2006&rft.volume=137&rft.issue=1-2&rft.pages=19%E2%80%9332&rft_id=info:doi\/10.1016%2Fj.geoderma.2006.08.024&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0016706106002540&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:4-16\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:4_16-0\">16.0<\/a><\/sup> <sup><a href=\"#cite_ref-:4_16-1\">16.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">De Vos, B\u00e9atrice; De Souza, Marcella Fernandez; Michels, Evi; Meers, Erik (13 January 2023). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/link.springer.com\/10.1007\/s11356-023-25198-z\" target=\"_blank\">\"Industrial hemp (Cannabis sativa L.) field cultivation in a phytoattenuation strategy and valorization potential of the fibers for textile production\"<\/a> (in en). <i>Environmental Science and Pollution Research<\/i> <b>30<\/b> (14): 41665\u201341681. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2Fs11356-023-25198-z\" target=\"_blank\">10.1007\/s11356-023-25198-z<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1614-7499\" target=\"_blank\">1614-7499<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/link.springer.com\/10.1007\/s11356-023-25198-z\" target=\"_blank\">https:\/\/link.springer.com\/10.1007\/s11356-023-25198-z<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Industrial+hemp+%28Cannabis+sativa+L.%29+field+cultivation+in+a+phytoattenuation+strategy+and+valorization+potential+of+the+fibers+for+textile+production&rft.jtitle=Environmental+Science+and+Pollution+Research&rft.aulast=De+Vos&rft.aufirst=B%C3%A9atrice&rft.au=De+Vos%2C%26%2332%3BB%C3%A9atrice&rft.au=De+Souza%2C%26%2332%3BMarcella+Fernandez&rft.au=Michels%2C%26%2332%3BEvi&rft.au=Meers%2C%26%2332%3BErik&rft.date=13+January+2023&rft.volume=30&rft.issue=14&rft.pages=41665%E2%80%9341681&rft_id=info:doi\/10.1007%2Fs11356-023-25198-z&rft.issn=1614-7499&rft_id=https%3A%2F%2Flink.springer.com%2F10.1007%2Fs11356-023-25198-z&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:5-17\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:5_17-0\">17.0<\/a><\/sup> <sup><a href=\"#cite_ref-:5_17-1\">17.1<\/a><\/sup> <sup><a href=\"#cite_ref-:5_17-2\">17.2<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Golia, Evangelia E.; Bethanis, John; Ntinopoulos, Nikolaos; Kaffe, Georgia-Garifalia; Komnou, Amalia Athanasia; Vasilou, Charicleia (1 April 2023). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2352554122003655\" target=\"_blank\">\"Investigating the potential of heavy metal accumulation from hemp. The use of industrial hemp (Cannabis Sativa L.) for phytoremediation of heavily and moderated polluted soils\"<\/a> (in en). <i>Sustainable Chemistry and Pharmacy<\/i> <b>31<\/b>: 100961. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.scp.2022.100961\" target=\"_blank\">10.1016\/j.scp.2022.100961<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2352554122003655\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2352554122003655<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Investigating+the+potential+of+heavy+metal+accumulation+from+hemp.+The+use+of+industrial+hemp+%28Cannabis+Sativa+L.%29+for+phytoremediation+of+heavily+and+moderated+polluted+soils&rft.jtitle=Sustainable+Chemistry+and+Pharmacy&rft.aulast=Golia&rft.aufirst=Evangelia+E.&rft.au=Golia%2C%26%2332%3BEvangelia+E.&rft.au=Bethanis%2C%26%2332%3BJohn&rft.au=Ntinopoulos%2C%26%2332%3BNikolaos&rft.au=Kaffe%2C%26%2332%3BGeorgia-Garifalia&rft.au=Komnou%2C%26%2332%3BAmalia+Athanasia&rft.au=Vasilou%2C%26%2332%3BCharicleia&rft.date=1+April+2023&rft.volume=31&rft.pages=100961&rft_id=info:doi\/10.1016%2Fj.scp.2022.100961&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2352554122003655&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-18\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-18\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Placido, Dante F.; Lee, Charles C. (23 February 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.mdpi.com\/2223-7747\/11\/5\/595\" target=\"_blank\">\"Potential of Industrial Hemp for Phytoremediation of Heavy Metals\"<\/a> (in en). <i>Plants<\/i> <b>11<\/b> (5): 595. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3390%2Fplants11050595\" target=\"_blank\">10.3390\/plants11050595<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2223-7747\" target=\"_blank\">2223-7747<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.pubmedcentral.gov\/articlerender.fcgi?tool=pmcentrez&artid=PMC8912475\" target=\"_blank\">PMC8912475<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/35270065\" target=\"_blank\">35270065<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.mdpi.com\/2223-7747\/11\/5\/595\" target=\"_blank\">https:\/\/www.mdpi.com\/2223-7747\/11\/5\/595<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Potential+of+Industrial+Hemp+for+Phytoremediation+of+Heavy+Metals&rft.jtitle=Plants&rft.aulast=Placido&rft.aufirst=Dante+F.&rft.au=Placido%2C%26%2332%3BDante+F.&rft.au=Lee%2C%26%2332%3BCharles+C.&rft.date=23+February+2022&rft.volume=11&rft.issue=5&rft.pages=595&rft_id=info:doi\/10.3390%2Fplants11050595&rft.issn=2223-7747&rft_id=info:pmc\/PMC8912475&rft_id=info:pmid\/35270065&rft_id=https%3A%2F%2Fwww.mdpi.com%2F2223-7747%2F11%2F5%2F595&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:6-19\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:6_19-0\">19.0<\/a><\/sup> <sup><a href=\"#cite_ref-:6_19-1\">19.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Rheay, Hanah T.; Omondi, Emmanuel C.; Brewer, Catherine E. (1 April 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/gcbb.12782\" target=\"_blank\">\"Potential of hemp ( Cannabis sativa L.) for paired phytoremediation and bioenergy production\"<\/a> (in en). <i>GCB Bioenergy<\/i> <b>13<\/b> (4): 525\u2013536. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1111%2Fgcbb.12782\" target=\"_blank\">10.1111\/gcbb.12782<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1757-1693\" target=\"_blank\">1757-1693<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/gcbb.12782\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/gcbb.12782<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Potential+of+hemp+%28+Cannabis+sativa+L.%29+for+paired+phytoremediation+and+bioenergy+production&rft.jtitle=GCB+Bioenergy&rft.aulast=Rheay&rft.aufirst=Hanah+T.&rft.au=Rheay%2C%26%2332%3BHanah+T.&rft.au=Omondi%2C%26%2332%3BEmmanuel+C.&rft.au=Brewer%2C%26%2332%3BCatherine+E.&rft.date=1+April+2021&rft.volume=13&rft.issue=4&rft.pages=525%E2%80%93536&rft_id=info:doi\/10.1111%2Fgcbb.12782&rft.issn=1757-1693&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1111%2Fgcbb.12782&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:7-20\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:7_20-0\">20.0<\/a><\/sup> <sup><a href=\"#cite_ref-:7_20-1\">20.1<\/a><\/sup> <sup><a href=\"#cite_ref-:7_20-2\">20.2<\/a><\/sup> <sup><a href=\"#cite_ref-:7_20-3\">20.3<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Shi, Gangrong; Cai, Qingsheng (7 May 2010). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.tandfonline.com\/doi\/abs\/10.1080\/01904161003728669\" target=\"_blank\">\"ZINC TOLERANCE AND ACCUMULATION IN EIGHT OIL CROPS\"<\/a> (in en). <i>Journal of Plant Nutrition<\/i> <b>33<\/b> (7): 982\u2013997. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1080%2F01904161003728669\" target=\"_blank\">10.1080\/01904161003728669<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0190-4167\" target=\"_blank\">0190-4167<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.tandfonline.com\/doi\/abs\/10.1080\/01904161003728669\" target=\"_blank\">http:\/\/www.tandfonline.com\/doi\/abs\/10.1080\/01904161003728669<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=ZINC+TOLERANCE+AND+ACCUMULATION+IN+EIGHT+OIL+CROPS&rft.jtitle=Journal+of+Plant+Nutrition&rft.aulast=Shi&rft.aufirst=Gangrong&rft.au=Shi%2C%26%2332%3BGangrong&rft.au=Cai%2C%26%2332%3BQingsheng&rft.date=7+May+2010&rft.volume=33&rft.issue=7&rft.pages=982%E2%80%93997&rft_id=info:doi\/10.1080%2F01904161003728669&rft.issn=0190-4167&rft_id=http%3A%2F%2Fwww.tandfonline.com%2Fdoi%2Fabs%2F10.1080%2F01904161003728669&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-21\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-21\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">DalCorso, Giovanni; Manara, Anna; Furini, Antonella (2013). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/academic.oup.com\/metallomics\/article\/5\/9\/1117-1132\/6015689\" target=\"_blank\">\"An overview of heavy metal challenge in plants: from roots to shoots\"<\/a> (in en). <i>Metallomics<\/i> <b>5<\/b> (9): 1117. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1039%2Fc3mt00038a\" target=\"_blank\">10.1039\/c3mt00038a<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1756-5901\" target=\"_blank\">1756-5901<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/academic.oup.com\/metallomics\/article\/5\/9\/1117-1132\/6015689\" target=\"_blank\">https:\/\/academic.oup.com\/metallomics\/article\/5\/9\/1117-1132\/6015689<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=An+overview+of+heavy+metal+challenge+in+plants%3A+from+roots+to+shoots&rft.jtitle=Metallomics&rft.aulast=DalCorso&rft.aufirst=Giovanni&rft.au=DalCorso%2C%26%2332%3BGiovanni&rft.au=Manara%2C%26%2332%3BAnna&rft.au=Furini%2C%26%2332%3BAntonella&rft.date=2013&rft.volume=5&rft.issue=9&rft.pages=1117&rft_id=info:doi\/10.1039%2Fc3mt00038a&rft.issn=1756-5901&rft_id=https%3A%2F%2Facademic.oup.com%2Fmetallomics%2Farticle%2F5%2F9%2F1117-1132%2F6015689&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-22\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-22\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ismael, Marwa A.; Elyamine, Ali Mohamed; Moussa, Mohamed G.; Cai, Miaomiao; Zhao, Xiaohu; Hu, Chengxiao (2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/academic.oup.com\/metallomics\/article\/11\/2\/255-277\/5957484\" target=\"_blank\">\"Cadmium in plants: uptake, toxicity, and its interactions with selenium fertilizers\"<\/a> (in en). <i>Metallomics<\/i> <b>11<\/b> (2): 255\u2013277. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1039%2FC8MT00247A\" target=\"_blank\">10.1039\/C8MT00247A<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1756-5901\" target=\"_blank\">1756-5901<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/academic.oup.com\/metallomics\/article\/11\/2\/255-277\/5957484\" target=\"_blank\">https:\/\/academic.oup.com\/metallomics\/article\/11\/2\/255-277\/5957484<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Cadmium+in+plants%3A+uptake%2C+toxicity%2C+and+its+interactions+with+selenium+fertilizers&rft.jtitle=Metallomics&rft.aulast=Ismael&rft.aufirst=Marwa+A.&rft.au=Ismael%2C%26%2332%3BMarwa+A.&rft.au=Elyamine%2C%26%2332%3BAli+Mohamed&rft.au=Moussa%2C%26%2332%3BMohamed+G.&rft.au=Cai%2C%26%2332%3BMiaomiao&rft.au=Zhao%2C%26%2332%3BXiaohu&rft.au=Hu%2C%26%2332%3BChengxiao&rft.date=2019&rft.volume=11&rft.issue=2&rft.pages=255%E2%80%93277&rft_id=info:doi\/10.1039%2FC8MT00247A&rft.issn=1756-5901&rft_id=https%3A%2F%2Facademic.oup.com%2Fmetallomics%2Farticle%2F11%2F2%2F255-277%2F5957484&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:8-23\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:8_23-0\">23.0<\/a><\/sup> <sup><a href=\"#cite_ref-:8_23-1\">23.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation book\">Kabata-Pendias, Alina (18 October 2010) (in en). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.taylorfrancis.com\/books\/9781420093704\" target=\"_blank\"><i>Trace Elements in Soils and Plants<\/i><\/a> (0 ed.). CRC Press. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1201%2Fb10158\" target=\"_blank\">10.1201\/b10158<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-0-429-19203-6<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.taylorfrancis.com\/books\/9781420093704\" target=\"_blank\">https:\/\/www.taylorfrancis.com\/books\/9781420093704<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=Trace+Elements+in+Soils+and+Plants&rft.aulast=Kabata-Pendias&rft.aufirst=Alina&rft.au=Kabata-Pendias%2C%26%2332%3BAlina&rft.date=18+October+2010&rft.edition=0&rft.pub=CRC+Press&rft_id=info:doi\/10.1201%2Fb10158&rft.isbn=978-0-429-19203-6&rft_id=https%3A%2F%2Fwww.taylorfrancis.com%2Fbooks%2F9781420093704&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-24\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-24\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation\" id=\"rdp-ebb-CITEREFSebastianNangiaPrasadRattanapolsan2019\">Sebastian, Abin; Nangia, Ashwini; Prasad, Majeti Narasimha Vara; Rattanapolsan, Ladawan; Nakbanpote, Woranan (2019), <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/B9780128148648000024\" target=\"_blank\">\"Cadmium Toxicity and Tolerance in Micro- and Phytobiomes\"<\/a> (in en), <i>Cadmium Toxicity and Tolerance in Plants<\/i> (Elsevier): 19\u201346, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fb978-0-12-814864-8.00002-4\" target=\"_blank\">10.1016\/b978-0-12-814864-8.00002-4<\/a>, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-0-12-814864-8<span class=\"printonly\">, <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/B9780128148648000024\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/B9780128148648000024<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 2023-09-27<\/span><\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Cadmium+Toxicity+and+Tolerance+in+Micro-+and+Phytobiomes&rft.jtitle=Cadmium+Toxicity+and+Tolerance+in+Plants&rft.aulast=Sebastian&rft.aufirst=Abin&rft.au=Sebastian%2C%26%2332%3BAbin&rft.au=Nangia%2C%26%2332%3BAshwini&rft.au=Prasad%2C%26%2332%3BMajeti+Narasimha+Vara&rft.au=Rattanapolsan%2C%26%2332%3BLadawan&rft.au=Nakbanpote%2C%26%2332%3BWoranan&rft.date=2019&rft.pages=19%E2%80%9346&rft.pub=Elsevier&rft_id=info:doi\/10.1016%2Fb978-0-12-814864-8.00002-4&rft.isbn=978-0-12-814864-8&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FB9780128148648000024&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-25\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-25\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Food and Agriculture Organization of the United Nations (2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.fao.org\/fao-who-codexalimentarius\/sh-proxy\/en\/?lnk=1&url=https%253A%252F%252Fworkspace.fao.org%252Fsites%252Fcodex%252FStandards%252FCXS%2B193-1995%252FCXS_193e.pdf\" target=\"_blank\">\"General Standard for Contaminants and Toxins in Food and Feed [CXS 193-1995<\/a>\"] (PDF). <i>Codex Alimentarius<\/i>. pp. 47\u201348<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.fao.org\/fao-who-codexalimentarius\/sh-proxy\/en\/?lnk=1&url=https%253A%252F%252Fworkspace.fao.org%252Fsites%252Fcodex%252FStandards%252FCXS%2B193-1995%252FCXS_193e.pdf\" target=\"_blank\">https:\/\/www.fao.org\/fao-who-codexalimentarius\/sh-proxy\/en\/?lnk=1&url=https%253A%252F%252Fworkspace.fao.org%252Fsites%252Fcodex%252FStandards%252FCXS%2B193-1995%252FCXS_193e.pdf<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=General+Standard+for+Contaminants+and+Toxins+in+Food+and+Feed+%5BCXS+193-1995%5D&rft.atitle=Codex+Alimentarius&rft.aulast=Food+and+Agriculture+Organization+of+the+United+Nations&rft.au=Food+and+Agriculture+Organization+of+the+United+Nations&rft.date=2019&rft.pages=pp.+47%E2%80%9348&rft_id=https%3A%2F%2Fwww.fao.org%2Ffao-who-codexalimentarius%2Fsh-proxy%2Fen%2F%3Flnk%3D1%26url%3Dhttps%25253A%25252F%25252Fworkspace.fao.org%25252Fsites%25252Fcodex%25252FStandards%25252FCXS%252B193-1995%25252FCXS_193e.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:9-26\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:9_26-0\">26.0<\/a><\/sup> <sup><a href=\"#cite_ref-:9_26-1\">26.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ahmad, Ayaz; Hadi, Fazal; Ali, Nasir (2 January 2015). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.tandfonline.com\/doi\/abs\/10.1080\/15226514.2013.828018\" target=\"_blank\">\"Effective Phytoextraction of Cadmium (Cd) with Increasing Concentration of Total Phenolics and Free Proline in Cannabis sativa (L) Plant Under Various Treatments of Fertilizers, Plant Growth Regulators and Sodium Salt\"<\/a> (in en). <i>International Journal of Phytoremediation<\/i> <b>17<\/b> (1): 56\u201365. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1080%2F15226514.2013.828018\" target=\"_blank\">10.1080\/15226514.2013.828018<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1522-6514\" target=\"_blank\">1522-6514<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.tandfonline.com\/doi\/abs\/10.1080\/15226514.2013.828018\" target=\"_blank\">http:\/\/www.tandfonline.com\/doi\/abs\/10.1080\/15226514.2013.828018<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Effective+Phytoextraction+of+Cadmium+%28Cd%29+with+Increasing+Concentration+of+Total+Phenolics+and+Free+Proline+in+Cannabis+sativa+%28L%29+Plant+Under+Various+Treatments+of+Fertilizers%2C+Plant+Growth+Regulators+and+Sodium+Salt&rft.jtitle=International+Journal+of+Phytoremediation&rft.aulast=Ahmad&rft.aufirst=Ayaz&rft.au=Ahmad%2C%26%2332%3BAyaz&rft.au=Hadi%2C%26%2332%3BFazal&rft.au=Ali%2C%26%2332%3BNasir&rft.date=2+January+2015&rft.volume=17&rft.issue=1&rft.pages=56%E2%80%9365&rft_id=info:doi\/10.1080%2F15226514.2013.828018&rft.issn=1522-6514&rft_id=http%3A%2F%2Fwww.tandfonline.com%2Fdoi%2Fabs%2F10.1080%2F15226514.2013.828018&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:10-27\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:10_27-0\">27.0<\/a><\/sup> <sup><a href=\"#cite_ref-:10_27-1\">27.1<\/a><\/sup> <sup><a href=\"#cite_ref-:10_27-2\">27.2<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Angelova, V.; Ivanova, R.; Delibaltova, V.; Ivanov, K. (1 May 2004). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0926669003001110\" target=\"_blank\">\"Bio-accumulation and distribution of heavy metals in fibre crops (flax, cotton and hemp)\"<\/a> (in en). <i>Industrial Crops and Products<\/i> <b>19<\/b> (3): 197\u2013205. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.indcrop.2003.10.001\" target=\"_blank\">10.1016\/j.indcrop.2003.10.001<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0926669003001110\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0926669003001110<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Bio-accumulation+and+distribution+of+heavy+metals+in+fibre+crops+%28flax%2C+cotton+and+hemp%29&rft.jtitle=Industrial+Crops+and+Products&rft.aulast=Angelova&rft.aufirst=V.&rft.au=Angelova%2C%26%2332%3BV.&rft.au=Ivanova%2C%26%2332%3BR.&rft.au=Delibaltova%2C%26%2332%3BV.&rft.au=Ivanov%2C%26%2332%3BK.&rft.date=1+May+2004&rft.volume=19&rft.issue=3&rft.pages=197%E2%80%93205&rft_id=info:doi\/10.1016%2Fj.indcrop.2003.10.001&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0926669003001110&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:11-28\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:11_28-0\">28.0<\/a><\/sup> <sup><a href=\"#cite_ref-:11_28-1\">28.1<\/a><\/sup> <sup><a href=\"#cite_ref-:11_28-2\">28.2<\/a><\/sup> <sup><a href=\"#cite_ref-:11_28-3\">28.3<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Citterio, Sandra; Santagostino, Angela; Fumagalli, Pietro; Prato, Nadia; Ranalli, Paolo; Sgorbati, Sergio (2003). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1023\/A:1026113905129\" target=\"_blank\">\"[No title found<\/a>\"]. <i>Plant and Soil<\/i> <b>256<\/b> (2): 243\u2013252. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1023%2FA%3A1026113905129\" target=\"_blank\">10.1023\/A:1026113905129<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1023\/A:1026113905129\" target=\"_blank\">http:\/\/link.springer.com\/10.1023\/A:1026113905129<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=%5BNo+title+found%5D&rft.jtitle=Plant+and+Soil&rft.aulast=Citterio&rft.aufirst=Sandra&rft.au=Citterio%2C%26%2332%3BSandra&rft.au=Santagostino%2C%26%2332%3BAngela&rft.au=Fumagalli%2C%26%2332%3BPietro&rft.au=Prato%2C%26%2332%3BNadia&rft.au=Ranalli%2C%26%2332%3BPaolo&rft.au=Sgorbati%2C%26%2332%3BSergio&rft.date=2003&rft.volume=256&rft.issue=2&rft.pages=243%E2%80%93252&rft_id=info:doi\/10.1023%2FA%3A1026113905129&rft_id=http%3A%2F%2Flink.springer.com%2F10.1023%2FA%3A1026113905129&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-29\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-29\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation\" id=\"rdp-ebb-CITEREFKumarSinghKumarRani2017\">Kumar, Sanjeev; Singh, Ritu; Kumar, Virendra; Rani, Anita; Jain, Rajeev (2017), Bauddh, Kuldeep; Singh, Bhaskar; Korstad, John, eds., <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/978-981-10-3084-0_10\" target=\"_blank\">\"Cannabis sativa: A Plant Suitable for Phytoremediation and Bioenergy Production\"<\/a> (in en), <i>Phytoremediation Potential of Bioenergy Plants<\/i> (Singapore: Springer Singapore): 269\u2013285, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2F978-981-10-3084-0_10\" target=\"_blank\">10.1007\/978-981-10-3084-0_10<\/a>, <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-981-10-3083-3<span class=\"printonly\">, <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/978-981-10-3084-0_10\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/978-981-10-3084-0_10<\/a><\/span><span class=\"reference-accessdate\">. Retrieved 2023-09-27<\/span><\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Cannabis+sativa%3A+A+Plant+Suitable+for+Phytoremediation+and+Bioenergy+Production&rft.jtitle=Phytoremediation+Potential+of+Bioenergy+Plants&rft.aulast=Kumar&rft.aufirst=Sanjeev&rft.au=Kumar%2C%26%2332%3BSanjeev&rft.au=Singh%2C%26%2332%3BRitu&rft.au=Kumar%2C%26%2332%3BVirendra&rft.au=Rani%2C%26%2332%3BAnita&rft.au=Jain%2C%26%2332%3BRajeev&rft.date=2017&rft.pages=269%E2%80%93285&rft.place=Singapore&rft.pub=Springer+Singapore&rft_id=info:doi\/10.1007%2F978-981-10-3084-0_10&rft.isbn=978-981-10-3083-3&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2F978-981-10-3084-0_10&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:12-30\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:12_30-0\">30.0<\/a><\/sup> <sup><a href=\"#cite_ref-:12_30-1\">30.1<\/a><\/sup> <sup><a href=\"#cite_ref-:12_30-2\">30.2<\/a><\/sup> <sup><a href=\"#cite_ref-:12_30-3\">30.3<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ait Ali, Nadia; Bernal, M. Pilar; Ater, Mohammed (2002). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1023\/A:1014995321560\" target=\"_blank\">\"[No title found<\/a>\"]. <i>Plant and Soil<\/i> <b>239<\/b> (1): 103\u2013111. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1023%2FA%3A1014995321560\" target=\"_blank\">10.1023\/A:1014995321560<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1023\/A:1014995321560\" target=\"_blank\">http:\/\/link.springer.com\/10.1023\/A:1014995321560<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=%5BNo+title+found%5D&rft.jtitle=Plant+and+Soil&rft.aulast=Ait+Ali&rft.aufirst=Nadia&rft.au=Ait+Ali%2C%26%2332%3BNadia&rft.au=Bernal%2C%26%2332%3BM.+Pilar&rft.au=Ater%2C%26%2332%3BMohammed&rft.date=2002&rft.volume=239&rft.issue=1&rft.pages=103%E2%80%93111&rft_id=info:doi\/10.1023%2FA%3A1014995321560&rft_id=http%3A%2F%2Flink.springer.com%2F10.1023%2FA%3A1014995321560&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:13-31\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:13_31-0\">31.0<\/a><\/sup> <sup><a href=\"#cite_ref-:13_31-1\">31.1<\/a><\/sup> <sup><a href=\"#cite_ref-:13_31-2\">31.2<\/a><\/sup> <sup><a href=\"#cite_ref-:13_31-3\">31.3<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Liu, W.-X.; Liu, J.-W.; Wu, M.-Z.; Li, Y.; Zhao, Y.; Li, S.-R. (1 March 2009). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/s00128-008-9575-6\" target=\"_blank\">\"Accumulation and Translocation of Toxic Heavy Metals in Winter Wheat (Triticum aestivum L.) Growing in Agricultural Soil of Zhengzhou, China\"<\/a> (in en). <i>Bulletin of Environmental Contamination and Toxicology<\/i> <b>82<\/b> (3): 343\u2013347. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2Fs00128-008-9575-6\" target=\"_blank\">10.1007\/s00128-008-9575-6<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0007-4861\" target=\"_blank\">0007-4861<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/s00128-008-9575-6\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/s00128-008-9575-6<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Accumulation+and+Translocation+of+Toxic+Heavy+Metals+in+Winter+Wheat+%28Triticum+aestivum+L.%29+Growing+in+Agricultural+Soil+of+Zhengzhou%2C+China&rft.jtitle=Bulletin+of+Environmental+Contamination+and+Toxicology&rft.aulast=Liu&rft.aufirst=W.-X.&rft.au=Liu%2C%26%2332%3BW.-X.&rft.au=Liu%2C%26%2332%3BJ.-W.&rft.au=Wu%2C%26%2332%3BM.-Z.&rft.au=Li%2C%26%2332%3BY.&rft.au=Zhao%2C%26%2332%3BY.&rft.au=Li%2C%26%2332%3BS.-R.&rft.date=1+March+2009&rft.volume=82&rft.issue=3&rft.pages=343%E2%80%93347&rft_id=info:doi\/10.1007%2Fs00128-008-9575-6&rft.issn=0007-4861&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2Fs00128-008-9575-6&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:14-32\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:14_32-0\">32.0<\/a><\/sup> <sup><a href=\"#cite_ref-:14_32-1\">32.1<\/a><\/sup> <sup><a href=\"#cite_ref-:14_32-2\">32.2<\/a><\/sup> <sup><a href=\"#cite_ref-:14_32-3\">32.3<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Pachura, Piotr; Ociepa-Kubicka, Agnieszka; Skowron-Grabowska, Beata (14 January 2016). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.tandfonline.com\/doi\/full\/10.1080\/19443994.2015.1017330\" target=\"_blank\">\"Assessment of the availability of heavy metals to plants based on the translocation index and the bioaccumulation factor\"<\/a> (in en). <i>Desalination and Water Treatment<\/i> <b>57<\/b> (3): 1469\u20131477. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1080%2F19443994.2015.1017330\" target=\"_blank\">10.1080\/19443994.2015.1017330<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1944-3994\" target=\"_blank\">1944-3994<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.tandfonline.com\/doi\/full\/10.1080\/19443994.2015.1017330\" target=\"_blank\">http:\/\/www.tandfonline.com\/doi\/full\/10.1080\/19443994.2015.1017330<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Assessment+of+the+availability+of+heavy+metals+to+plants+based+on+the+translocation+index+and+the+bioaccumulation+factor&rft.jtitle=Desalination+and+Water+Treatment&rft.aulast=Pachura&rft.aufirst=Piotr&rft.au=Pachura%2C%26%2332%3BPiotr&rft.au=Ociepa-Kubicka%2C%26%2332%3BAgnieszka&rft.au=Skowron-Grabowska%2C%26%2332%3BBeata&rft.date=14+January+2016&rft.volume=57&rft.issue=3&rft.pages=1469%E2%80%931477&rft_id=info:doi\/10.1080%2F19443994.2015.1017330&rft.issn=1944-3994&rft_id=http%3A%2F%2Fwww.tandfonline.com%2Fdoi%2Ffull%2F10.1080%2F19443994.2015.1017330&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:15-33\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:15_33-0\">33.0<\/a><\/sup> <sup><a href=\"#cite_ref-:15_33-1\">33.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Zayed, Adel; Gowthaman, Suvarnalatha; Terry, Norman (1 May 1998). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/acsess.onlinelibrary.wiley.com\/doi\/10.2134\/jeq1998.00472425002700030032x\" target=\"_blank\">\"Phytoaccumulation of Trace Elements by Wetland Plants: I. Duckweed\"<\/a> (in en). <i>Journal of Environmental Quality<\/i> <b>27<\/b> (3): 715\u2013721. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.2134%2Fjeq1998.00472425002700030032x\" target=\"_blank\">10.2134\/jeq1998.00472425002700030032x<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0047-2425\" target=\"_blank\">0047-2425<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/acsess.onlinelibrary.wiley.com\/doi\/10.2134\/jeq1998.00472425002700030032x\" target=\"_blank\">https:\/\/acsess.onlinelibrary.wiley.com\/doi\/10.2134\/jeq1998.00472425002700030032x<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Phytoaccumulation+of+Trace+Elements+by+Wetland+Plants%3A+I.+Duckweed&rft.jtitle=Journal+of+Environmental+Quality&rft.aulast=Zayed&rft.aufirst=Adel&rft.au=Zayed%2C%26%2332%3BAdel&rft.au=Gowthaman%2C%26%2332%3BSuvarnalatha&rft.au=Terry%2C%26%2332%3BNorman&rft.date=1+May+1998&rft.volume=27&rft.issue=3&rft.pages=715%E2%80%93721&rft_id=info:doi\/10.2134%2Fjeq1998.00472425002700030032x&rft.issn=0047-2425&rft_id=https%3A%2F%2Facsess.onlinelibrary.wiley.com%2Fdoi%2F10.2134%2Fjeq1998.00472425002700030032x&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:16-34\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:16_34-0\">34.0<\/a><\/sup> <sup><a href=\"#cite_ref-:16_34-1\">34.1<\/a><\/sup> <sup><a href=\"#cite_ref-:16_34-2\">34.2<\/a><\/sup> <sup><a href=\"#cite_ref-:16_34-3\">34.3<\/a><\/sup> <sup><a href=\"#cite_ref-:16_34-4\">34.4<\/a><\/sup> <sup><a href=\"#cite_ref-:16_34-5\">34.5<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Luyckx, Marie; Hausman, Jean-Fran\u00e7ois; Blanquet, Mathilde; Guerriero, Gea; Lutts, Stanley (1 July 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/link.springer.com\/10.1007\/s11356-021-12912-y\" target=\"_blank\">\"Silicon reduces cadmium absorption and increases root-to-shoot translocation without impacting growth in young plants of hemp (Cannabis sativa L.) on a short-term basis\"<\/a> (in en). <i>Environmental Science and Pollution Research<\/i> <b>28<\/b> (28): 37963\u201337977. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2Fs11356-021-12912-y\" target=\"_blank\">10.1007\/s11356-021-12912-y<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0944-1344\" target=\"_blank\">0944-1344<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/link.springer.com\/10.1007\/s11356-021-12912-y\" target=\"_blank\">https:\/\/link.springer.com\/10.1007\/s11356-021-12912-y<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Silicon+reduces+cadmium+absorption+and+increases+root-to-shoot+translocation+without+impacting+growth+in+young+plants+of+hemp+%28Cannabis+sativa+L.%29+on+a+short-term+basis&rft.jtitle=Environmental+Science+and+Pollution+Research&rft.aulast=Luyckx&rft.aufirst=Marie&rft.au=Luyckx%2C%26%2332%3BMarie&rft.au=Hausman%2C%26%2332%3BJean-Fran%C3%A7ois&rft.au=Blanquet%2C%26%2332%3BMathilde&rft.au=Guerriero%2C%26%2332%3BGea&rft.au=Lutts%2C%26%2332%3BStanley&rft.date=1+July+2021&rft.volume=28&rft.issue=28&rft.pages=37963%E2%80%9337977&rft_id=info:doi\/10.1007%2Fs11356-021-12912-y&rft.issn=0944-1344&rft_id=https%3A%2F%2Flink.springer.com%2F10.1007%2Fs11356-021-12912-y&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:17-35\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:17_35-0\">35.0<\/a><\/sup> <sup><a href=\"#cite_ref-:17_35-1\">35.1<\/a><\/sup> <sup><a href=\"#cite_ref-:17_35-2\">35.2<\/a><\/sup> <sup><a href=\"#cite_ref-:17_35-3\">35.3<\/a><\/sup> <sup><a href=\"#cite_ref-:17_35-4\">35.4<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Marabesi, Amanda O.; Nambeesan, Savithri U.; van Iersel, Marc W.; Lessl, Jason T.; Coolong, Timothy W. (30 May 2023). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpls.2023.1183249\/full\" target=\"_blank\">\"Cadmium exposure is associated with increased transcript abundance of multiple heavy metal associated transporter genes in roots of hemp (Cannabis sativa L.)\"<\/a>. <i>Frontiers in Plant Science<\/i> <b>14<\/b>: 1183249. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3389%2Ffpls.2023.1183249\" target=\"_blank\">10.3389\/fpls.2023.1183249<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1664-462X\" target=\"_blank\">1664-462X<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.pubmedcentral.gov\/articlerender.fcgi?tool=pmcentrez&artid=PMC10265645\" target=\"_blank\">PMC10265645<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/37324677\" target=\"_blank\">37324677<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpls.2023.1183249\/full\" target=\"_blank\">https:\/\/www.frontiersin.org\/articles\/10.3389\/fpls.2023.1183249\/full<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Cadmium+exposure+is+associated+with+increased+transcript+abundance+of+multiple+heavy+metal+associated+transporter+genes+in+roots+of+hemp+%28Cannabis+sativa+L.%29&rft.jtitle=Frontiers+in+Plant+Science&rft.aulast=Marabesi&rft.aufirst=Amanda+O.&rft.au=Marabesi%2C%26%2332%3BAmanda+O.&rft.au=Nambeesan%2C%26%2332%3BSavithri+U.&rft.au=van+Iersel%2C%26%2332%3BMarc+W.&rft.au=Lessl%2C%26%2332%3BJason+T.&rft.au=Coolong%2C%26%2332%3BTimothy+W.&rft.date=30+May+2023&rft.volume=14&rft.pages=1183249&rft_id=info:doi\/10.3389%2Ffpls.2023.1183249&rft.issn=1664-462X&rft_id=info:pmc\/PMC10265645&rft_id=info:pmid\/37324677&rft_id=https%3A%2F%2Fwww.frontiersin.org%2Farticles%2F10.3389%2Ffpls.2023.1183249%2Ffull&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-36\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-36\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Hoagland, D.R.; Arnon, D.I. (1950). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/openlibrary.org\/books\/OL25240089M\/The_water-culture_method_for_growing_plants_without_soil\" target=\"_blank\">\"The water-culture method for growing plants without soil\"<\/a>. <i>California Agricultural Experiment Station, Circular<\/i>. University of California<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/openlibrary.org\/books\/OL25240089M\/The_water-culture_method_for_growing_plants_without_soil\" target=\"_blank\">https:\/\/openlibrary.org\/books\/OL25240089M\/The_water-culture_method_for_growing_plants_without_soil<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=The+water-culture+method+for+growing+plants+without+soil&rft.atitle=California+Agricultural+Experiment+Station%2C+Circular&rft.aulast=Hoagland%2C+D.R.%3B+Arnon%2C+D.I.&rft.au=Hoagland%2C+D.R.%3B+Arnon%2C+D.I.&rft.date=1950&rft.pub=University+of+California&rft_id=https%3A%2F%2Fopenlibrary.org%2Fbooks%2FOL25240089M%2FThe_water-culture_method_for_growing_plants_without_soil&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-37\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-37\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">U.S. EPA (December 1996). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.epa.gov\/sites\/default\/files\/2015-12\/documents\/3052.pdf\" target=\"_blank\">\"Method 3052. Microwave assisted acid digestion of siliceous and organically based matrices\"<\/a> (PDF). <i>Test methods for evaluating solid waste<\/i><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.epa.gov\/sites\/default\/files\/2015-12\/documents\/3052.pdf\" target=\"_blank\">https:\/\/www.epa.gov\/sites\/default\/files\/2015-12\/documents\/3052.pdf<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Method+3052.+Microwave+assisted+acid+digestion+of+siliceous+and+organically+based+matrices&rft.atitle=Test+methods+for+evaluating+solid+waste&rft.aulast=U.S.+EPA&rft.au=U.S.+EPA&rft.date=December+1996&rft_id=https%3A%2F%2Fwww.epa.gov%2Fsites%2Fdefault%2Ffiles%2F2015-12%2Fdocuments%2F3052.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-38\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-38\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">U.S. EPA (1994). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.epa.gov\/esam\/epa-method-2008-determination-trace-elements-waters-and-wastes-inductively-coupled-plasma-mass\" target=\"_blank\">\"EPA Method 200.8: Determination of Trace Elements in Waters and Wastes by Inductively Coupled Plasma-Mass Spectrometry\"<\/a>. <i>Selected Analytical Methods for Environmental Remediation and Recovery (SAM)<\/i><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.epa.gov\/esam\/epa-method-2008-determination-trace-elements-waters-and-wastes-inductively-coupled-plasma-mass\" target=\"_blank\">https:\/\/www.epa.gov\/esam\/epa-method-2008-determination-trace-elements-waters-and-wastes-inductively-coupled-plasma-mass<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=EPA+Method+200.8%3A+Determination+of+Trace+Elements+in+Waters+and+Wastes+by+Inductively+Coupled+Plasma-Mass+Spectrometry&rft.atitle=Selected+Analytical+Methods+for+Environmental+Remediation+and+Recovery+%28SAM%29&rft.aulast=U.S.+EPA&rft.au=U.S.+EPA&rft.date=1994&rft_id=https%3A%2F%2Fwww.epa.gov%2Fesam%2Fepa-method-2008-determination-trace-elements-waters-and-wastes-inductively-coupled-plasma-mass&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:18-39\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:18_39-0\">39.0<\/a><\/sup> <sup><a href=\"#cite_ref-:18_39-1\">39.1<\/a><\/sup> <sup><a href=\"#cite_ref-:18_39-2\">39.2<\/a><\/sup> <sup><a href=\"#cite_ref-:18_39-3\">39.3<\/a><\/sup> <sup><a href=\"#cite_ref-:18_39-4\">39.4<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Shi, Gangrong; Cai, Qingsheng (1 September 2009). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0734975009000573\" target=\"_blank\">\"Cadmium tolerance and accumulation in eight potential energy crops\"<\/a> (in en). <i>Biotechnology Advances<\/i> <b>27<\/b> (5): 555\u2013561. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.biotechadv.2009.04.006\" target=\"_blank\">10.1016\/j.biotechadv.2009.04.006<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0734975009000573\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0734975009000573<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Cadmium+tolerance+and+accumulation+in+eight+potential+energy+crops&rft.jtitle=Biotechnology+Advances&rft.aulast=Shi&rft.aufirst=Gangrong&rft.au=Shi%2C%26%2332%3BGangrong&rft.au=Cai%2C%26%2332%3BQingsheng&rft.date=1+September+2009&rft.volume=27&rft.issue=5&rft.pages=555%E2%80%93561&rft_id=info:doi\/10.1016%2Fj.biotechadv.2009.04.006&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0734975009000573&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-40\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-40\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">\u0106a\u0107i\u0107, Marija; Per\u010din, Aleksandra; Zgorelec, \u017deljka; Kisi\u0107, Ivica (2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/jcea.agr.hr\/en\/issues\/article\/2201\" target=\"_blank\">\"Evaluation of heavy metals accumulation potential of hemp (Cannabis sativa L.)\"<\/a> (in en). <i>Journal of Central European Agriculture<\/i> <b>20<\/b> (2): 700\u2013711. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.5513%2FJCEA01%2F20.2.2201\" target=\"_blank\">10.5513\/JCEA01\/20.2.2201<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1332-9049\" target=\"_blank\">1332-9049<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/jcea.agr.hr\/en\/issues\/article\/2201\" target=\"_blank\">https:\/\/jcea.agr.hr\/en\/issues\/article\/2201<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Evaluation+of+heavy+metals+accumulation+potential+of+hemp+%28Cannabis+sativa+L.%29&rft.jtitle=Journal+of+Central+European+Agriculture&rft.aulast=%C4%86a%C4%87i%C4%87&rft.aufirst=Marija&rft.au=%C4%86a%C4%87i%C4%87%2C%26%2332%3BMarija&rft.au=Per%C4%8Din%2C%26%2332%3BAleksandra&rft.au=Zgorelec%2C%26%2332%3B%C5%BDeljka&rft.au=Kisi%C4%87%2C%26%2332%3BIvica&rft.date=2019&rft.volume=20&rft.issue=2&rft.pages=700%E2%80%93711&rft_id=info:doi\/10.5513%2FJCEA01%2F20.2.2201&rft.issn=1332-9049&rft_id=https%3A%2F%2Fjcea.agr.hr%2Fen%2Fissues%2Farticle%2F2201&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:19-41\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:19_41-0\">41.0<\/a><\/sup> <sup><a href=\"#cite_ref-:19_41-1\">41.1<\/a><\/sup> <sup><a href=\"#cite_ref-:19_41-2\">41.2<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Shi, Gangrong; Liu, Caifeng; Cui, Meicheng; Ma, Yuhua; Cai, Qingsheng (1 September 2012). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/s12010-011-9382-0\" target=\"_blank\">\"Cadmium Tolerance and Bioaccumulation of 18 Hemp Accessions\"<\/a> (in en). <i>Applied Biochemistry and Biotechnology<\/i> <b>168<\/b> (1): 163\u2013173. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2Fs12010-011-9382-0\" target=\"_blank\">10.1007\/s12010-011-9382-0<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0273-2289\" target=\"_blank\">0273-2289<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/s12010-011-9382-0\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/s12010-011-9382-0<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Cadmium+Tolerance+and+Bioaccumulation+of+18+Hemp+Accessions&rft.jtitle=Applied+Biochemistry+and+Biotechnology&rft.aulast=Shi&rft.aufirst=Gangrong&rft.au=Shi%2C%26%2332%3BGangrong&rft.au=Liu%2C%26%2332%3BCaifeng&rft.au=Cui%2C%26%2332%3BMeicheng&rft.au=Ma%2C%26%2332%3BYuhua&rft.au=Cai%2C%26%2332%3BQingsheng&rft.date=1+September+2012&rft.volume=168&rft.issue=1&rft.pages=163%E2%80%93173&rft_id=info:doi\/10.1007%2Fs12010-011-9382-0&rft.issn=0273-2289&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2Fs12010-011-9382-0&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-42\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-42\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">University of Sulaimani; Majid, Salih N.; Khwakaram, Ahmed I.; University of Sulaimani; Rasul, Ghafoor A. Mam; University of Sulaimani (16 October 2014). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/jzs.univsul.edu.iq\/index.php\/jzs\/article\/view\/jzs-10350\" target=\"_blank\">\"Bioaccumulation, Enrichment and Translocation Factors of some Heavy Metals in Typha Angustifolia and Phragmites Australis Species Growing along Qalyasan Stream in Sulaimani City \/IKR\"<\/a>. <i>Journal of Zankoy Sulaimani - Part A<\/i> <b>16<\/b> (4): 93\u2013109. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.17656%2Fjzs.10350\" target=\"_blank\">10.17656\/jzs.10350<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/jzs.univsul.edu.iq\/index.php\/jzs\/article\/view\/jzs-10350\" target=\"_blank\">https:\/\/jzs.univsul.edu.iq\/index.php\/jzs\/article\/view\/jzs-10350<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Bioaccumulation%2C+Enrichment+and+Translocation+Factors+of+some+Heavy+Metals+in+Typha+Angustifolia+and+Phragmites+Australis+Species+Growing+along+Qalyasan+Stream+in+Sulaimani+City+%2FIKR&rft.jtitle=Journal+of+Zankoy+Sulaimani+-+Part+A&rft.aulast=University+of+Sulaimani&rft.au=University+of+Sulaimani&rft.au=Majid%2C%26%2332%3BSalih+N.&rft.au=Khwakaram%2C%26%2332%3BAhmed+I.&rft.au=University+of+Sulaimani&rft.au=Rasul%2C%26%2332%3BGhafoor++A.+Mam&rft.au=University+of+Sulaimani&rft.date=16+October+2014&rft.volume=16&rft.issue=4&rft.pages=93%E2%80%93109&rft_id=info:doi\/10.17656%2Fjzs.10350&rft_id=https%3A%2F%2Fjzs.univsul.edu.iq%2Findex.php%2Fjzs%2Farticle%2Fview%2Fjzs-10350&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-43\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-43\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Bernstein, Nirit; Gorelick, Jonathan; Zerahia, Roei; Koch, Sraya (17 June 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.frontiersin.org\/article\/10.3389\/fpls.2019.00736\/full\" target=\"_blank\">\"Impact of N, P, K, and Humic Acid Supplementation on the Chemical Profile of Medical Cannabis (Cannabis sativa L)\"<\/a>. <i>Frontiers in Plant Science<\/i> <b>10<\/b>: 736. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3389%2Ffpls.2019.00736\" target=\"_blank\">10.3389\/fpls.2019.00736<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1664-462X\" target=\"_blank\">1664-462X<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.pubmedcentral.gov\/articlerender.fcgi?tool=pmcentrez&artid=PMC6589925\" target=\"_blank\">PMC6589925<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/31263470\" target=\"_blank\">31263470<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.frontiersin.org\/article\/10.3389\/fpls.2019.00736\/full\" target=\"_blank\">https:\/\/www.frontiersin.org\/article\/10.3389\/fpls.2019.00736\/full<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Impact+of+N%2C+P%2C+K%2C+and+Humic+Acid+Supplementation+on+the+Chemical+Profile+of+Medical+Cannabis+%28Cannabis+sativa+L%29&rft.jtitle=Frontiers+in+Plant+Science&rft.aulast=Bernstein&rft.aufirst=Nirit&rft.au=Bernstein%2C%26%2332%3BNirit&rft.au=Gorelick%2C%26%2332%3BJonathan&rft.au=Zerahia%2C%26%2332%3BRoei&rft.au=Koch%2C%26%2332%3BSraya&rft.date=17+June+2019&rft.volume=10&rft.pages=736&rft_id=info:doi\/10.3389%2Ffpls.2019.00736&rft.issn=1664-462X&rft_id=info:pmc\/PMC6589925&rft_id=info:pmid\/31263470&rft_id=https%3A%2F%2Fwww.frontiersin.org%2Farticle%2F10.3389%2Ffpls.2019.00736%2Ffull&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-44\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-44\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Khan, Anwarzeb; Khan, Sardar; Alam, Mehboob; Khan, Muhammad Amjad; Aamir, Muhammad; Qamar, Zahir; Rehman, Zahir Ur; Perveen, Sajida (1 March 2016). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0045653515304616\" target=\"_blank\">\"Toxic metal interactions affect the bioaccumulation and dietary intake of macro- and micro-nutrients\"<\/a> (in en). <i>Chemosphere<\/i> <b>146<\/b>: 121\u2013128. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.chemosphere.2015.12.014\" target=\"_blank\">10.1016\/j.chemosphere.2015.12.014<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0045653515304616\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0045653515304616<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Toxic+metal+interactions+affect+the+bioaccumulation+and+dietary+intake+of+macro-+and+micro-nutrients&rft.jtitle=Chemosphere&rft.aulast=Khan&rft.aufirst=Anwarzeb&rft.au=Khan%2C%26%2332%3BAnwarzeb&rft.au=Khan%2C%26%2332%3BSardar&rft.au=Alam%2C%26%2332%3BMehboob&rft.au=Khan%2C%26%2332%3BMuhammad+Amjad&rft.au=Aamir%2C%26%2332%3BMuhammad&rft.au=Qamar%2C%26%2332%3BZahir&rft.au=Rehman%2C%26%2332%3BZahir+Ur&rft.au=Perveen%2C%26%2332%3BSajida&rft.date=1+March+2016&rft.volume=146&rft.pages=121%E2%80%93128&rft_id=info:doi\/10.1016%2Fj.chemosphere.2015.12.014&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0045653515304616&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-45\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-45\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Ahmad, Parvaiz; Abdel Latef, Arafat A.; Abd_Allah, Elsayed F.; Hashem, Abeer; Sarwat, Maryam; Anjum, Naser A.; Gucel, Salih (27 April 2016). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/journal.frontiersin.org\/Article\/10.3389\/fpls.2016.00513\/abstract\" target=\"_blank\">\"Calcium and Potassium Supplementation Enhanced Growth, Osmolyte Secondary Metabolite Production, and Enzymatic Antioxidant Machinery in Cadmium-Exposed Chickpea (Cicer arietinum L.)\"<\/a>. <i>Frontiers in Plant Science<\/i> <b>7<\/b>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3389%2Ffpls.2016.00513\" target=\"_blank\">10.3389\/fpls.2016.00513<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1664-462X\" target=\"_blank\">1664-462X<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.pubmedcentral.gov\/articlerender.fcgi?tool=pmcentrez&artid=PMC4847423\" target=\"_blank\">PMC4847423<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/27200003\" target=\"_blank\">27200003<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/journal.frontiersin.org\/Article\/10.3389\/fpls.2016.00513\/abstract\" target=\"_blank\">http:\/\/journal.frontiersin.org\/Article\/10.3389\/fpls.2016.00513\/abstract<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Calcium+and+Potassium+Supplementation+Enhanced+Growth%2C+Osmolyte+Secondary+Metabolite+Production%2C+and+Enzymatic+Antioxidant+Machinery+in+Cadmium-Exposed+Chickpea+%28Cicer+arietinum+L.%29&rft.jtitle=Frontiers+in+Plant+Science&rft.aulast=Ahmad&rft.aufirst=Parvaiz&rft.au=Ahmad%2C%26%2332%3BParvaiz&rft.au=Abdel+Latef%2C%26%2332%3BArafat+A.&rft.au=Abd_Allah%2C%26%2332%3BElsayed+F.&rft.au=Hashem%2C%26%2332%3BAbeer&rft.au=Sarwat%2C%26%2332%3BMaryam&rft.au=Anjum%2C%26%2332%3BNaser+A.&rft.au=Gucel%2C%26%2332%3BSalih&rft.date=27+April+2016&rft.volume=7&rft_id=info:doi\/10.3389%2Ffpls.2016.00513&rft.issn=1664-462X&rft_id=info:pmc\/PMC4847423&rft_id=info:pmid\/27200003&rft_id=http%3A%2F%2Fjournal.frontiersin.org%2FArticle%2F10.3389%2Ffpls.2016.00513%2Fabstract&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:20-46\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:20_46-0\">46.0<\/a><\/sup> <sup><a href=\"#cite_ref-:20_46-1\">46.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Gomes, Marcelo Pedrosa; Marques, Teresa Cristina Lara Lanza S\u00e1 e Mel; Soares, Angela Maria (1 April 2013). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.2478\/s11756-013-0005-9\" target=\"_blank\">\"Cadmium effects on mineral nutrition of the Cd-hyperaccumulator Pfaffia glomerata\"<\/a> (in en). <i>Biologia<\/i> <b>68<\/b> (2): 223\u2013230. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.2478%2Fs11756-013-0005-9\" target=\"_blank\">10.2478\/s11756-013-0005-9<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0006-3088\" target=\"_blank\">0006-3088<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.2478\/s11756-013-0005-9\" target=\"_blank\">http:\/\/link.springer.com\/10.2478\/s11756-013-0005-9<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Cadmium+effects+on+mineral+nutrition+of+the+Cd-hyperaccumulator+Pfaffia+glomerata&rft.jtitle=Biologia&rft.aulast=Gomes&rft.aufirst=Marcelo+Pedrosa&rft.au=Gomes%2C%26%2332%3BMarcelo+Pedrosa&rft.au=Marques%2C%26%2332%3BTeresa+Cristina+Lara+Lanza+S%C3%A1+e+Mel&rft.au=Soares%2C%26%2332%3BAngela+Maria&rft.date=1+April+2013&rft.volume=68&rft.issue=2&rft.pages=223%E2%80%93230&rft_id=info:doi\/10.2478%2Fs11756-013-0005-9&rft.issn=0006-3088&rft_id=http%3A%2F%2Flink.springer.com%2F10.2478%2Fs11756-013-0005-9&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-47\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-47\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Sarwar, Nadeem; Saifullah; Malhi, Sukhdev S; Zia, Munir Hussain; Naeem, Asif; Bibi, Sadia; Farid, Ghulam (30 April 2010). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/jsfa.3916\" target=\"_blank\">\"Role of mineral nutrition in minimizing cadmium accumulation by plants: Mineral nutrition for minimizing cadmium accumulation\"<\/a> (in en). <i>Journal of the Science of Food and Agriculture<\/i> <b>90<\/b> (6): 925\u2013937. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1002%2Fjsfa.3916\" target=\"_blank\">10.1002\/jsfa.3916<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/jsfa.3916\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/jsfa.3916<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Role+of+mineral+nutrition+in+minimizing+cadmium+accumulation+by+plants%3A+Mineral+nutrition+for+minimizing+cadmium+accumulation&rft.jtitle=Journal+of+the+Science+of+Food+and+Agriculture&rft.aulast=Sarwar&rft.aufirst=Nadeem&rft.au=Sarwar%2C%26%2332%3BNadeem&rft.au=Saifullah&rft.au=Malhi%2C%26%2332%3BSukhdev+S&rft.au=Zia%2C%26%2332%3BMunir+Hussain&rft.au=Naeem%2C%26%2332%3BAsif&rft.au=Bibi%2C%26%2332%3BSadia&rft.au=Farid%2C%26%2332%3BGhulam&rft.date=30+April+2010&rft.volume=90&rft.issue=6&rft.pages=925%E2%80%93937&rft_id=info:doi\/10.1002%2Fjsfa.3916&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1002%2Fjsfa.3916&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-48\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-48\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Husain, Rabab; Weeden, Hannah; Bogush, Daniel; Deguchi, Michihito; Soliman, Mario; Potlakayala, Shobha; Katam, Ramesh; Goldman, Stephen <i>et al.<\/i> (29 August 2019). Thavamani, Palanisami. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/dx.plos.org\/10.1371\/journal.pone.0221570\" target=\"_blank\">\"Enhanced tolerance of industrial hemp (Cannabis sativa L.) plants on abandoned mine land soil leads to overexpression of cannabinoids\"<\/a> (in en). <i>PLOS ONE<\/i> <b>14<\/b> (8): e0221570. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1371%2Fjournal.pone.0221570\" target=\"_blank\">10.1371\/journal.pone.0221570<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1932-6203\" target=\"_blank\">1932-6203<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.pubmedcentral.gov\/articlerender.fcgi?tool=pmcentrez&artid=PMC6715179\" target=\"_blank\">PMC6715179<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/31465423\" target=\"_blank\">31465423<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/dx.plos.org\/10.1371\/journal.pone.0221570\" target=\"_blank\">https:\/\/dx.plos.org\/10.1371\/journal.pone.0221570<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Enhanced+tolerance+of+industrial+hemp+%28Cannabis+sativa+L.%29+plants+on+abandoned+mine+land+soil+leads+to+overexpression+of+cannabinoids&rft.jtitle=PLOS+ONE&rft.aulast=Husain&rft.aufirst=Rabab&rft.au=Husain%2C%26%2332%3BRabab&rft.au=Weeden%2C%26%2332%3BHannah&rft.au=Bogush%2C%26%2332%3BDaniel&rft.au=Deguchi%2C%26%2332%3BMichihito&rft.au=Soliman%2C%26%2332%3BMario&rft.au=Potlakayala%2C%26%2332%3BShobha&rft.au=Katam%2C%26%2332%3BRamesh&rft.au=Goldman%2C%26%2332%3BStephen&rft.au=Rudrabhatla%2C%26%2332%3BSairam&rft.date=29+August+2019&rft.volume=14&rft.issue=8&rft.pages=e0221570&rft_id=info:doi\/10.1371%2Fjournal.pone.0221570&rft.issn=1932-6203&rft_id=info:pmc\/PMC6715179&rft_id=info:pmid\/31465423&rft_id=https%3A%2F%2Fdx.plos.org%2F10.1371%2Fjournal.pone.0221570&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-49\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-49\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Yang, Rui; Berthold, Erin C.; McCurdy, Christopher R.; da Silva Benevenute, Sarah; Brym, Zachary T.; Freeman, Joshua H. (3 June 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acs.jafc.0c01211\" target=\"_blank\">\"Development of Cannabinoids in Flowers of Industrial Hemp ( Cannabis sativa L.): A Pilot Study\"<\/a> (in en). <i>Journal of Agricultural and Food Chemistry<\/i> <b>68<\/b> (22): 6058\u20136064. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1021%2Facs.jafc.0c01211\" target=\"_blank\">10.1021\/acs.jafc.0c01211<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0021-8561\" target=\"_blank\">0021-8561<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acs.jafc.0c01211\" target=\"_blank\">https:\/\/pubs.acs.org\/doi\/10.1021\/acs.jafc.0c01211<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Development+of+Cannabinoids+in+Flowers+of+Industrial+Hemp+%28+Cannabis+sativa+L.%29%3A+A+Pilot+Study&rft.jtitle=Journal+of+Agricultural+and+Food+Chemistry&rft.aulast=Yang&rft.aufirst=Rui&rft.au=Yang%2C%26%2332%3BRui&rft.au=Berthold%2C%26%2332%3BErin+C.&rft.au=McCurdy%2C%26%2332%3BChristopher+R.&rft.au=da+Silva+Benevenute%2C%26%2332%3BSarah&rft.au=Brym%2C%26%2332%3BZachary+T.&rft.au=Freeman%2C%26%2332%3BJoshua+H.&rft.date=3+June+2020&rft.volume=68&rft.issue=22&rft.pages=6058%E2%80%936064&rft_id=info:doi\/10.1021%2Facs.jafc.0c01211&rft.issn=0021-8561&rft_id=https%3A%2F%2Fpubs.acs.org%2Fdoi%2F10.1021%2Facs.jafc.0c01211&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-50\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-50\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Trancoso, Ingrid; de Souza, Guilherme A. R.; dos Santos, Paulo Ricardo; dos Santos, K\u00e9sia Dias; de Miranda, Rosana Maria dos Santos Nani; da Silva, Amanda L\u00facia Pereira Machado; Santos, Dennys Zsolt; Garc\u00eda-Tejero, Ivan F. <i>et al.<\/i> (22 June 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.mdpi.com\/2073-4395\/12\/7\/1492\" target=\"_blank\">\"Cannabis sativa L.: Crop Management and Abiotic Factors That Affect Phytocannabinoid Production\"<\/a> (in en). <i>Agronomy<\/i> <b>12<\/b> (7): 1492. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3390%2Fagronomy12071492\" target=\"_blank\">10.3390\/agronomy12071492<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2073-4395\" target=\"_blank\">2073-4395<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.mdpi.com\/2073-4395\/12\/7\/1492\" target=\"_blank\">https:\/\/www.mdpi.com\/2073-4395\/12\/7\/1492<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Cannabis+sativa+L.%3A+Crop+Management+and+Abiotic+Factors+That+Affect+Phytocannabinoid+Production&rft.jtitle=Agronomy&rft.aulast=Trancoso&rft.aufirst=Ingrid&rft.au=Trancoso%2C%26%2332%3BIngrid&rft.au=de+Souza%2C%26%2332%3BGuilherme+A.+R.&rft.au=dos+Santos%2C%26%2332%3BPaulo+Ricardo&rft.au=dos+Santos%2C%26%2332%3BK%C3%A9sia+Dias&rft.au=de+Miranda%2C%26%2332%3BRosana+Maria+dos+Santos+Nani&rft.au=da+Silva%2C%26%2332%3BAmanda+L%C3%BAcia+Pereira+Machado&rft.au=Santos%2C%26%2332%3BDennys+Zsolt&rft.au=Garc%C3%ADa-Tejero%2C%26%2332%3BIvan+F.&rft.au=Campostrini%2C%26%2332%3BEliemar&rft.date=22+June+2022&rft.volume=12&rft.issue=7&rft.pages=1492&rft_id=info:doi\/10.3390%2Fagronomy12071492&rft.issn=2073-4395&rft_id=https%3A%2F%2Fwww.mdpi.com%2F2073-4395%2F12%2F7%2F1492&rfr_id=info:sid\/en.wikipedia.org:Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<\/ol><\/div><\/div>\n<h2><span class=\"mw-headline\" id=\"Notes\">Notes<\/span><\/h2>\n<p>This presentation is faithful to the original, with only a few minor changes to presentation. Some grammar and punctuation was cleaned up to improve readability. In some cases important information was missing from the references, and that information was added.\n<\/p>\n<!-- \nNewPP limit report\nCached time: 20231215231611\nCache expiry: 86400\nDynamic content: false\nComplications: []\nCPU time usage: 0.697 seconds\nReal time usage: 0.724 seconds\nPreprocessor visited node count: 49860\/1000000\nPost\u2010expand include size: 442720\/2097152 bytes\nTemplate argument size: 133038\/2097152 bytes\nHighest expansion depth: 25\/40\nExpensive parser function count: 0\/100\nUnstrip recursion depth: 0\/20\nUnstrip post\u2010expand size: 127895\/5000000 bytes\n-->\n<!--\nTransclusion expansion time report (%,ms,calls,template)\n100.00% 643.727 1 -total\n 82.84% 533.265 1 Template:Reflist\n 66.69% 429.322 50 Template:Citation\/core\n 48.88% 314.678 35 Template:Cite_journal\n 14.07% 90.598 7 Template:Cite_book\n 12.78% 82.283 48 Template:Date\n 8.24% 53.061 1 Template:Infobox_journal_article\n 7.78% 50.095 87 Template:Citation\/identifier\n 7.50% 48.260 1 Template:Infobox\n 6.35% 40.886 4 Template:Citation\n-->\n\n<!-- Saved in parser cache with key cannaqa_wiki:pcache:idhash:6012-0!canonical and timestamp 20231215231610 and revision id 18437. Serialized with JSON.\n -->\n<\/div><\/div><div class=\"printfooter\">Source: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market\">https:\/\/www.cannaqa.wiki\/index.php?title=Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market<\/a><\/div>\n<!-- end content --><div class=\"visualClear\"><\/div><\/div><\/div><div class=\"visualClear\"><\/div><\/div><!-- end of the left (by default at least) column --><div class=\"visualClear\"><\/div><\/div>\n<\/body>","34fb9a0e0648cc8855dc5ea15c3c3a92_images":["https:\/\/www.cannaqa.wiki\/images\/8\/86\/Fig1_Marebesi_Water23_15-12.png","https:\/\/www.cannaqa.wiki\/images\/2\/20\/Fig2_Marebesi_Water23_15-12.png"],"34fb9a0e0648cc8855dc5ea15c3c3a92_timestamp":1702682170,"147ac42afe5c27b45d4ed44a531a6754_type":"article","147ac42afe5c27b45d4ed44a531a6754_title":"High levels of pesticides found in illicit cannabis inflorescence compared to licensed samples in Canadian study using expanded 327 pesticides multiresidue method (Gagnon et al. 2023)","147ac42afe5c27b45d4ed44a531a6754_url":"https:\/\/www.cannaqa.wiki\/index.php?title=Journal:High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method","147ac42afe5c27b45d4ed44a531a6754_plaintext":"\n\nJournal:High levels of pesticides found in illicit cannabis inflorescence compared to licensed samples in Canadian study using expanded 327 pesticides multiresidue methodFrom CannaQAWikiJump to navigationJump to searchFull article title\n \nHigh levels of pesticides found in illicit cannabis inflorescence compared to licensed samples in Canadian study using expanded 327 pesticides multiresidue methodJournal\n \nJournal of Cannabis ResearchAuthor(s)\n \nGagnon, Mathieu; McRitchie, Tyler; Montsion, Kim; Tilly, Jos\u00e9e; Blais, Michel; Snider, Neil; Blais, David R.Author affiliation(s)\n \nHealth CanadaPrimary contact\n \nEmail: David dot Blais at hc dash sc dot gc dot caYear published\n \n2023Volume and issue\n \n5Article #\n \n34DOI\n \n10.1186\/s42238-023-00200-0ISSN\n \n2522-5782Distribution license\n \nCreative Commons Attribution 4.0 InternationalWebsite\n \nhttps:\/\/jcannabisresearch.biomedcentral.com\/articles\/10.1186\/s42238-023-00200-0Download\n \nhttps:\/\/jcannabisresearch.biomedcentral.com\/counter\/pdf\/10.1186\/s42238-023-00200-0.pdf (PDF)\n\nContents \n\n1 Abstract \n2 Introduction \n3 Methods \n\n3.1 Sampling \n3.2 Standards and reagents \n3.3 Apparatus \n3.4 Standard solution preparation \n3.5 Sample preparation of dried cannabis flowers \n3.6 Instrument conditions \n\n3.6.1 LC\u2013MS\/MS \n3.6.2 GC\u2013MS\/MS \n\n\n3.7 Validation criteria \n3.8 Quality control \n\n\n4 Results \n\n4.1 Validation data \n4.2 Application of method to real-world samples \n\n\n5 Discussion \n6 Conclusion \n7 Abbreviations, acronyms, and initialisms \n8 Acknowledgements \n\n8.1 Author contributions \n8.2 Funding \n8.3 Availability of data and materials \n8.4 Competing interests \n\n\n9 References \n10 Notes \n\n\n\nAbstract \nBackground: As Cannabis was legalized in Canada for recreational use in 2018 with the implementation of the Cannabis Act, regulations were put in place to ensure safety and consistency across the cannabis industry. This includes the requirement for licence holders to demonstrate that no unauthorized pesticides are used to treat cannabis or have contaminated it. In this study, we describe an expanded 327 multi-residue pesticide analysis in cannabis inflorescence to confirm if the implementation of the Cannabis Act is providing safer licensed products to Canadians in comparison to those of the illicit market.\nMethods: An extensive multi-residue method was developed using a modified quick, easy, cheap, effective, rugged, and safe (QuEChERS) sample preparation method using a combination of gas chromatography\u2014triple quadrupole mass spectrometry (GC\u2013MS\/MS) and liquid chromatography\u2014triple quadrupole mass spectrometry (LC\u2013MS\/MS) for the simultaneous quantification of 327 active pesticide ingredients in cannabis inflorescence.\nResults: Application of this method to Canadian licensed inflorescence samples revealed a six percent sample positivity rate, with only two pesticide residues detected\u2014myclobutanil and dichlobenil\u2014at the method\u2019s lowest calibrated level (LCL) of 0.01 \u03bcg\/g. Canadian illicit cannabis inflorescence samples that were analyzed showed a striking contrast with a 92 percent sample positivity rate, covering 23 unique pesticide active ingredients with 3.7 different pesticides identified on average per sample. Chlorpyrifos, imidacloprid, and myclobutanil were measured in illicit samples at concentrations up to three orders of magnitude above the method LCL of 0.01 \u03bcg\/g.\nConclusion: These results demonstrate the need of an extensive multi-residue method capable of analyzing hundreds of pesticides simultaneously, to generate data for future policy and regulatory decision-making, and to enable Canadians to make safe cannabis choices.\n\nIntroduction \nIn 2018, Canada legalized the recreational usage of Cannabis, supplementing the framework for cannabis for medical purposes, which had been in place since 2001. With the Cannabis Act[1] and its associated regulations[2] coming into force in 2018, Canada sought to standardize and enforce consistency, health, and safety across Canada\u2019s legal cannabis industry. To ensure safe cannabis products to Canadians, Health Canada regulates microbial and chemical contaminants, including pesticides. In addition to the existing analytical testing requirements under Canada's cannabis regulations, since January 2019, the industry must also follow the mandatory requirements for testing cannabis for pesticide active ingredients[3], where license holders must demonstrate that none of the 96 unauthorized pesticide active ingredients are used to treat cannabis or have contaminated it.\nPrior to the regulations going into effect in January 2019, some 18 percent of licensed cannabis products contained unregistered pesticides, with myclobutanil, bifenazate, boscalid, and fludioxonil pesticides most commonly present[4], and myclobutanil notably being classified as moderately hazardous by the World Health Organization (WHO).[5] This study aims to determine if unregistered pesticides are still prevalent in the licensed market. To gain a broader view of pesticide usage during cannabis production, we streamlined, expanded, and validated a single method using a combination of gas chromatography\u2014triple quadrupole mass spectrometry (GC\u2013MS\/MS) and liquid chromatography\u2014triple quadrupole mass spectrometry (LC\u2013MS\/MS) for the simultaneous quantification of 327 pesticide active ingredients in cannabis inflorescence, going well beyond the mandatory testing of 96 pesticide active ingredients.[3] Although use of the licensed, legal cannabis market has been gaining ground in Canada since legalization, up to 13 percent of Canadians still report consuming illicit cannabis almost exclusively.[6] As such, illicit cannabis samples were also analyzed for pesticides in this study to determine how they compare to the Canadian licensed cannabis market.\n\nMethods \nSampling \nTo reflect as realistically as possible the sources of cannabis inflorescence available to Canadians across the country, 36 licensed samples were purchased in 2021 from the Ontario Cannabis Store (Ontario, Canada) from license holders located in all five Canadian regions (British Columbia, Prairies, Ontario, Quebec, and Atlantic) (Table 1). The 24 illicit cannabis samples were obtained from seizures by law enforcement officers across the country and submitted to Health Canada for laboratory testing in 2021.\n\n\n\n\n\n\n\nTable 1. Geographical distribution of cannabis (C. sativa) inflorescence samples obtained across Canada.\n\n\nRegion\n\nLicensed samples\n\nIllicit samples\n\n\nBritish Columbia\n\n9\n\n3\n\n\nPrairies\n\n5\n\n1\n\n\nOntario\n\n12\n\n5\n\n\nQu\u00e9bec\n\n6\n\n14\n\n\nAtlantic\n\n4\n\n1\n\n\nTotal\n\n36\n\n24\n\n\n\nStandards and reagents \nPesticide analytical standards were purchased from Chemservice (West Chester, PA) and Sigma-Aldrich Canada (Oakville, ON). Analytical grade acetone and toluene were purchased from EMD Millipore (Darmstadt, Germany). Analytical grade acetonitrile and Na2SO4 were purchased from Fisher Scientific (Fairlawn, NJ). Water was obtained from a Milli-Q\u00ae Plus Ultra Pure Water system (Millipore Corp., Burlington, MA). Sepra C18-E was obtained from Phenomenex (Torrance, CA). Supelclean ENVI-Carb SPE Tubes were obtained from Supelco (Bellefonte, PA). Sep-Pak Classic NH2 Cartridges were obtained from Waters Corp. (Milford, MA).\n\nApparatus \nFor sample preparation, a laboratory blender 51BL30 (Stamford, Connecticut), a high-speed shaker (Spex Sample Prep Geno-Grinder; Fisher Scientific, Fairlawn, NJ), a centrifuge (Allegra X15R 208v; Beckman Coulter Inc., Brea, CA), a solvent evaporator (Xcelvap; Horizon Technologies, Salem, NH), and a rotary evaporator (Rotavpor R-114, B\u00dcCHI Labortechnik AG, Flawil, Switzerland) were used. Sample analysis was carried out on a GC\u2013MS\/MS 7010B gas chromatograph quadrupole mass spectrometer\/mass spectrometer (Agilent Technologies, Santa Clara, CA) and LC\u2013MS\/MS Exion HPLC 6500 Q-Trap triple-quadrupole mass spectrometer (AB Sciex, Framingham, MA).\n\nStandard solution preparation \nHigh-concentration pesticide stock standard solutions were prepared from the purest analytical material commercially available, typically\u2009\u2265\u200995 percent. In general, stock standard solutions were prepared in the range of 1000\u20132500 \u03bcg\/mL in acetone for GC\u2013MS\/MS compounds, and in either 100 percent acetonitrile or 100 percent methanol for LC\u2013MS\/MS compounds. From these, intermediate and spiking standard solutions were prepared respectively at 50 \u03bcg\/mL and 1 \u03bcg\/mL. Calibration standards were prepared with each sample set at concentrations of 0.8\u2009\u00d7\u2009, 1\u2009\u00d7\u2009, 2\u2009\u00d7\u2009, 3\u2009\u00d7\u2009, and 5\u2009\u00d7\u2009the lowest calibrated level (LCL) in pesticide-free cannabis matrix extract to compensate for ion suppression\/enhancement effects.\n\nSample preparation of dried cannabis flowers \nCannabis inflorescence samples (5\u201320 g) were homogenized in a laboratory blender. Acetonitrile (20 mL) was added to 2 g ground cannabis inflorescence sample and the mixture was extracted with a Geno-Grinder at 1750 rpm for two minutes. The tube was centrifuged at 4500 rpm for five minutes. Exactly 4 mL of the extract was added to a tube containing 1 g of dispersive C18 and shaken by Geno-Grinder at 1200 rpm for one minute. Exactly 2 mL was transferred to an ENVI-Carb\/Aminopropyl SPE containing 1 cm of Na2SO4, and eluted with 25 mL of 3:1 ACN:Toluene. The sample\u2019s solvent was exchanged to acetone, blown down to less than 1 mL using a rotary evaporator, and 20 \u03bcL of 5 \u03bcg\/mL 2,4,6-tribromobiphenyl was added as an internal standard. The sample was diluted to 1 mL with acetone. Half of the extract was transferred to a vial for GC\u2013MS\/MS analysis. The remaining portion\u2019s solvent was exchanged to acetonitrile with solvent evaporator, brought to approximately 0.1 mL. Twenty microliters of isoprocarb 5 \u03bcg\/mL was added as an internal standard, which was then diluted to 0.5 mL with acetonitrile and brought to 1 mL with H2O. The sample was filtered using a 1-cc plastic syringe and a 0.2-\u00b5m filter and transferred to a vial for LC\u2013MS\/MS analysis.\n\nInstrument conditions \n LC\u2013MS\/MS \nSample analysis was carried out using a 6500 Q-Trap LC-MSMS (AB Sciex). Analyst version 1.6.3 (AB Sciex) and MultiQuant version 3.0.2 (AB Sciex) software were used for instrument control and data analysis, respectively. A Kinetex C18 column (2.1\u2009\u00d7\u200950 mm, 2.6 \u03bcm) was used and maintained at 30 \u00b0C. The source was maintained at 550 \u00b0C. The following gas parameters were used: curtain gas, 35 psi; collision gas, 9psi; ion spray voltage, 5500 V; ion source gas 1, 50 psi; ion source gas 2, 55 psi. The injection volume was 1 \u03bcL. The mobile phases were water methanol (95\u2009+\u20095)\u2009+\u200910 mM formic acid\u2009+\u200910 mM ammonium formate (A) and water\u2013methanol (5\u2009+\u200995)\u2009+\u200910 mM formic acid\u2009+\u200910 mM ammonium formate (B). The flow rate was 0.7 mL\/min. The following elution gradient was used: 0\u201320 minutes, 0 percent B increasing to 100 percent B; 20\u201324.50 minutes, 100 percent B; 24.50\u201324.60 decreasing to 0 percent B then held from 24.60 to 25 minutes. Analysis was carried out by positive electrospray ionization using retention time-scheduled multiple reaction monitoring (MRM) to acquire two transitions (quantitative and qualitative) for each analyte. A partial list of these transition masses for both the LC\u2212MS\/MS and GC\u2212MS\/MS methods can be found in Supplementary information, Table S1 and S2, respectively.\n\n GC\u2013MS\/MS \nAn Agilent 7010B GC\u2013MS\/MS carried out sample analysis. Mass Hunter software (Agilent) was used for instrument control and data analysis. The injection port was a multi mode injector (MMI) maintained at 250 \u00b0C. The liner was an inert double tapered splitless liner (Agilent # 5190\u20133983). The injection volume was 1 \u03bcL in splitless mode. Helium carrier gas was maintained at a constant flow of 1.0 mL\/min. ZB-Multiresidue-1 capillary columns were used (2 columns; each of 15 m\u2009\u00d7\u20090.25 mm\u2009\u00d7\u20090.25 \u03bcm) (Phenomenex # 7EG-G016-11-CI) with backflush procedure at mid-column. The front column was fitted with a 1-m retention gap of the same stationary phase. The oven temperature was maintained at 60 \u00b0C for one minute, ramped to 120 \u00b0C at 40 \u00b0C\/minute, then ramped to 310 \u00b0C at 5 \u00b0C\/min with a 11.5-minute hold (total run time: 52 minutes). The temperature of the MS source was maintained at 300 \u00b0C and the transfer line at 305 \u00b0C. Nitrogen was used as the collision gas at a flow of 1 mL\/minute. Analysis was carried out by electron impact ionization using dynamic MRM to acquire at least two transitions (quantitative and qualitative) for each analyte.\n\nValidation criteria \nQuantitative validation data must show that specific pesticide\/matrix combinations can be accurately quantitated at the LCL deemed fit for purpose, the lowest value for the method being 0.01 \u00b5g\/g. The LCL for each pesticide was determined by an injection of a series of matrix-matched standards. The LCL was deemed acceptable if the signal of the LCL peak height to the height of the surrounding noise was at a minimum of 5:1 ratio for two transitions for the GC\u2013MS\/MS and LC\u2013MS\/MS. This ratio is the relative intensity of the quantifying ion\u2019s response compared to the qualifying ion\u2019s response. Ion ratios must be within permitted tolerances to be acceptable (Table 2).\n\n\n\n\n\n\n\nTable 2. Permittable tolerance of quantifying ion responses compared to the qualifying ion relative intensity.\n\n\nRelative intensity (% of base peak)\n\nPermitted tolerance\n\n\n>\u200950%\n\n\u00b1\u200920%\n\n\n>\u200920 to 50%\n\n\u00b1\u200925%\n\n\n>\u200910 to 20%\n\n\u00b1\u200930%\n\n\n\u2264\u200910%\n\n\u00b1\u200950%\n\n\n\nIn addition, five replicate spikes at the LCL must meet method performance criteria of mean recoveries in the range of 70\u2013120 percent with an RSD\u2009\u2264\u200920%. Exceptionally, a mean recovery below 70 percent may be acceptable if the recovery is consistent with an RSD\u2009\u2264\u200920%.[7]\nThe accuracy and precision of the pesticide recoveries were measured by spiking blank cannabis inflorescence matrix at the LCL (n\u2009=\u20095), 3\u2009\u00d7\u2009LCL (n\u2009=\u20093) and 5\u2009\u00d7\u2009LCL (n\u2009=\u20092). Linearity was established based on matrix-matched standards in the concentration range of 0.005\u20130.04 \u03bcg\/mL for LC-MS\/MS, 0.010\u20130.080 \u03bcg\/mL for GC-MS\/MS. The calibration curve generated from the standards must have a correlation coefficient (R2) greater or equal to 0.99.\n\nQuality control \nAfter the method was validated, samples were analyzed with quality control (QC) measures in place for each sample set to ensure the integrity of the results. Each set of samples included a reagent blank, a matrix blank, and a representative matrix spike at the LCL for QC. A blank sample was spiked with 200 \u00b5L of 0.1 \u00b5g\/mL of GC\u2013MS\/MS and LC\u2013MS\/MS spiking solutions. The spike was allowed to stand for a minimum of 30 minutes. The blanks and spike were then processed the same way as the samples. To compensate for matrix effects on pesticides in plant material, all standards were made from pesticide-free cannabis inflorescence matrix extracts, with the addition of pesticides standards at various concentrations. Results were calculated using a six-point calibration curve (at concentrations of 0.8\u2009\u00d7\u2009, 1\u2009\u00d7\u2009, 2\u2009\u00d7\u2009, 3\u2009\u00d7\u2009, 5\u2009\u00d7\u2009, and 10\u2009\u00d7\u2009the LCL).\n\nResults \nValidation data \nTo meet the requirements of the mandatory cannabis testing for 96 pesticide active ingredients[3], the new method was validated using more sensitive GC\u2013MS\/MS and LC\u2013MS\/MS instruments with better selectivity to detect 327 pesticides. Although most pesticides met the validation requirements for a LCL of 0.010 \u00b5g\/g, 31 percent of pesticides (101 out of 327) did not meet the 0.010 \u00b5g\/g LCL target and were validated at higher levels. These higher adjusted LCLs range from 0.02 \u00b5g\/g to 0.4 \u00b5g\/g (Supplementary information, Table S3). Overall, 285 pesticides tested meet validation criteria and can confidently give a quantitative result (Supplementary information, Table S3). While the remaining 42 pesticides did not pass the stringent quantification validation, they still met the criteria for monitoring their presence in cannabis inflorescence. When qualitatively identified in a sample, the word \"monitored\" is added for these 42 pesticides.\nMean recoveries in the range of 70\u2009\u2212\u2009120 percent, with a relative standard deviation (RSD)\u2009\u2264\u200920 percent between the 10 spiked replicates, were achieved for over 68 percent of the pesticides validated (Supplementary information, Table S3). Mean recoveries below 70 percent were still accepted (in the range of 30 to 69 percent only; lower than 30 percent is considered not recovered) if RSD\u2009\u2264\u200920 percent for compound recoveries at that level. Of the 285 pesticides that passed the validation, 22 percent adhere to this exception for lower 30\u201369 percent recoveries with an RSD\u2009\u2264\u200920 percent (Supplementary information, Table S3). Overall, our method demonstrated good linearity for 83 percent of pesticides attempted as the calibration curves had a correlation coefficient greater than 0.99.\nIt is important to note that piperonyl butoxide did not meet the validation criteria due to a large interference present in the reference material. The samples found positive were quantitated with a more targeted method with enough resolution to provide separation of the piperonyl butoxide and interfering signals to gain a better performance for this compound. A summary of the 12 additional recoveries outside of the validation (Supplementary information, Table S4) shows piperonyl butoxide has a better average recovery of 40 percent at 0.01 ppm, with an RSD of 21 percent at the low level (n =\u20096). At a higher spike concentration of 0.25 ppm, an average recovery of 73 percent was observed, with an RSD of 23 percent (n\u2009=\u20096). The correlation coefficient value for the curve used to calibrate these recoveries was acceptable (R2\u2009=\u20090.9984). While the RSDs for these recoveries exceed the validation criteria, they provide more confidence in the ability of this method to qualitatively monitor piperonyl butoxide with an estimated concentration.\n\nApplication of method to real-world samples \nThis newly expanded method was applied to real-world cannabis inflorescence samples available to Canadians across the country and used to determine if unregistered pesticide use is still prevalent in both the licensed and illicit markets. In total, 36 licensed samples and 24 illicit cannabis samples (Table 1) were analyzed against the method\u2019s 327 pesticides. Of the 36 licensed samples analyzed, only two pesticide residues were quantified (Table 3), representing a six percent positivity rate, with the measured concentration at our method LCL of 0.01 \u03bcg\/g.\n\n\n\n\n\n\n\nTable 3. Pesticide quantified in licensed and illicit samples obtained across Canada. a = Monitored.\n\n\nSource\n\nSamples analyzed\n\nSample positive rate (%)\n\nPesticides detected\n\nSample detection frequency\n\nPesticide concentration range (\u00b5g\/g)\n\n\nLicensed\n\n36\n\n6\n\nDichlobenil\n\n1\n\n0.01\n\n\nMyclobutanil\n\n1\n\n0.01\n\n\nIllicit\n\n24\n\n92\n\nAbamectin\n\n2\n\n0.06 to 0.6\n\n\nAzoxystrobin\n\n1\n\n0.2\n\n\nBifenazate\n\n4\n\n0.009 to 0.1\n\n\nBoscalid\n\n1\n\n0.04\n\n\nCarbaryl\n\n2\n\n0.02 to 0.06\n\n\nChlorphenapyr\n\n2\n\n0.5 to 5\n\n\nChlorpyrifos\n\n4\n\n0.01 to 30\n\n\nDichlorvos\n\n2\n\n0.05 to 1\n\n\nFluopyram\n\n1\n\n0.03\n\n\nImidacloprid\n\n3\n\n0.1 to 60\n\n\nMalaoxon\n\n1\n\n0.009\n\n\nMalathion\n\n1\n\n0.2\n\n\nMyclobutanil\n\n17\n\n0.02 to 70\n\n\nPaclobutrazol\n\n10\n\n0.009 to 1\n\n\nPermethrin\n\n3\n\n0.1 to 0.7\n\n\nPiperonyl butoxide\n\n10\n\n0.01 to 2a\n\n\nPyrethrins\n\n8\n\n0.03 to 1\n\n\nPyridaben\n\n1\n\n0.03\n\n\nSpinosad\n\n1\n\n0.2a\n\n\nSpirodiclofen\n\n1\n\n0.3\n\n\nSpiromesifen\n\n2\n\n0.2 to 1\n\n\nSpirotetramat\n\n1\n\n0.1a\n\n\nTetramethrin\n\n1\n\n0.8\n\n\n\nPesticides were detected in 92 percent of Canadian illicit cannabis inflorescence samples, with 23 unique pesticide active ingredients quantified (Table 3). Four pesticides and synergists\u2014myclobutanil, paclobutrazol, piperonyl butoxide, and pyrethrins\u2014were detected at a high sample frequency rate, eight to 17 times in a total 24 illicit samples. One illicit sample alone contained nine different pesticide active ingredients. Illicit cannabis contained on average 3.7 different pesticides per sample, and 87 percent of positive samples contained more than one different pesticide. The pesticide concentrations quantified varied greatly, with chlorpyrifos, imidacloprid, and myclobutanil measured at 30, 60, and 70 \u03bcg\/g respectively, over three orders of magnitude higher that the method\u2019s LCLs of 0.01 \u03bcg\/g.\n\nDiscussion \nThe main objective of this study was to streamline and expand our existing cannabis inflorescence method[4], which was possible with more powerful instruments and enabled the addition of a GC\u2013MS\/MS quantification split. The existing modified quick, easy, cheap, effective, rugged, and safe (QuEChERS) extraction[4] was adapted by eliminating the addition of water, salting-out, and enhanced matrix removal (EMR) clean-up steps, while dispersive C-18 replaced the C-18 column SPE for a four-fold time efficiency gain.\nThe method validation of the streamlined extraction and new instruments shows that all 285 pesticides meet the validation criteria based on the LCL, accuracy, precision, and linearity using matrix-matched standards. The remaining 42 pesticides that did not pass the stringent quantification validation still met the criteria for monitoring their presence in cannabis inflorescence. Cannabis inflorescence is a challenging matrix with its complex composition of oils, resins, terpenes, and cannabinoids. The power and sensitivity of modern triple quadrupole mass spectrometry enable to reach most 0.010 \u00b5g\/g regulatory limits of quantification even while quantitatively detecting hundreds of pesticide and metabolite residues simultaneously. This validation data demonstrates that comprehensive testing of pesticides in cannabis inflorescence is achievable beyond the current 2019 testing requirements[3], enabling the provision of essential data for future policy and regulatory decision-making. These results are in-line with recent studies that successfully expanded their cannabis inflorescence pesticide method to several dozens[8][9] and even hundreds[10][11] of different pesticide residues analyzed simultaneously.\nApplication of this expanded method to licensed cannabis inflorescence found a six percent sample positivity rate, with measured concentrations at the method\u2019s LCL of 0.01 \u03bcg\/g. Although quantified in one licensed sample, dichlobenil is not part of the mandatory cannabis testing for pesticide active ingredients list[3], indicating the importance of expanded multi-residue methods to generate valuable data for informed decisions regarding regulatory policies aimed at Canadian cannabis users who themselves want to make informed choices. Despite a six percent positivity rate, the licensed Canadian cannabis sector has greatly improved with regards to presence of pesticides since testing requirements were enacted in 2019, given the sample positivity rate of 30 percent prior to 2019.[4]\nIn a striking contrast, Canadian illicit cannabis inflorescence samples show a 92 percent positivity rate, with 23 unique ctive pesticide ingredients quantified and at concentrations up to three orders of magnitude higher that the method\u2019s LCLs of 0.01 \u03bcg\/g. High illicit cannabis pesticide positivity rates have been also observed in other jurisdictions.[9][12][13][14] To the authors\u2019 knowledge, this study is the only extensive pesticide multi-residue analysis that compares pesticides in the licensed and illicit cannabis markets in a nationwide jurisdiction where cannabis has been legalized. Albeit being a small study, our results do support the Government of Canada messaging where \"consuming illegal products could lead to adverse effects and other serious harms,\" noting that \"testing of illegal cannabis has found contaminants like pesticides and unacceptable levels of bacteria, lead, and arsenic.\"[15]\n\nConclusion \nThis study demonstrates a new streamlined and expanded method for the detection of 327 pesticides in cannabis inflorescence via gas chromatography\u2014triple quadruple mass spectroscopy and liquid chromatography\u2014triple quadruple mass spectroscopy. The validation of this method determined 285 unique pesticides can be quantified at levels ranging from 0.01 to 0.4 \u00b5g\/g, and 42 pesticides analyzed qualitatively. This method was applied to real world samples from both licensed and illicit markets, revealing high presence and concentration of pesticides in illicit samples compared to samples from the licensed market. With a six percent sample positivity rete, the licensed Canadian cannabis sector has greatly improved with regards to presence of pesticides since the 2019 mandate on regulatory testing. As a first, this study demonstrates the importance of extensive pesticide multi-residue methods comparing pesticides in the licensed and illicit cannabis markets to generate valuable data for informed decisions regarding regulatory policies and for Canadian cannabis users making informed choices.\n\n Abbreviations, acronyms, and initialisms \nGC\u2013MS\/MS: Gas chromatography\u2013triple quadruple mass spectroscopy\nLC\u2013MS\/MS: Liquid chromatography\u2013triple quadruple mass spectroscopy\nLCL: Lowest calibrated level\nQC: Quality control\nQuEChERS: Quick, easy, cheap, effective, rugged, and safe\nRSD: Relative standard deviation\nWHO: World Health Organization\nAcknowledgements \nThe authors would like to thank Health Canada Cannabis Laboratory for their assistance in the procurement of illicit cannabis samples.\n\nAuthor contributions \nMG led the method development, acquisition of cannabis material, and sample analysis. TM contributed to the method development and drafted material and results of the manuscript. KM, JT, and MB contributed to the method development and sample analysis. NS peer reviewed the method validation data. DRB allocated the laboratory resources for this project and wrote the manuscript. All authors read and approved the final manuscript.\n\nFunding \nOpen Access funding provided by Health Canada. None other to declare.\n\nAvailability of data and materials \nThe data is available from the corresponding author on reasonable request.\n\nCompeting interests \nThe authors declare that they have no competing interests.\n\nReferences \n\n\n\u2191 \"Cannabis Act (S.C. 2018, c. 16)\". Justice Laws Website. Government of Canada. 27 April 2023. https:\/\/laws-lois.justice.gc.ca\/eng\/acts\/c-24.5\/FullText.html .   \n \n\n\u2191 \"Cannabis Regulations (SOR\/2018-144)\". Justice Laws Website. Government of Canada. 2 December 2022. https:\/\/laws-lois.justice.gc.ca\/eng\/regulations\/sor-2018-144\/FullText.html .   \n \n\n\u2191 3.0 3.1 3.2 3.3 3.4 Health Canada (30 August 2019). \"Mandatory cannabis testing for pesticide active ingredients - Requirements\". Government of Canada. https:\/\/www.canada.ca\/en\/public-health\/services\/publications\/drugs-health-products\/cannabis-testing-pesticide-requirements.html .   \n \n\n\u2191 4.0 4.1 4.2 4.3 Moulins, Jonathan R.; Blais, Michel; Montsion, Kim; Tully, Josee; Mohan, William; Gagnon, Mathieu; McRitchie, Tyler; Kwong, Keri et al. (1 November 2018). \"Multiresidue Method of Analysis of Pesticides in Medical Cannabis\". Journal of AOAC International 101 (6): 1948\u20131960. doi:10.5740\/jaoacint.17-0495. ISSN 1944-7922. PMID 29843862. https:\/\/pubmed.ncbi.nlm.nih.gov\/29843862 .   \n \n\n\u2191 World Health Organization (2020) (in en). The WHO recommended classification of pesticides by hazard and guidelines to classification 2019. WHO recommended classification of pesticides by hazard and guidelines to classification. Geneva: World Health Organization. ISBN 978-92-4-000566-2. https:\/\/apps.who.int\/iris\/handle\/10665\/332193 .   \n \n\n\u2191 Health Canada (23 December 2021). \"Canadian Cannabis Survey 2021: Summary\". Government of Canada. https:\/\/www.canada.ca\/en\/health-canada\/services\/drugs-medication\/cannabis\/research-data\/canadian-cannabis-survey-2021-summary.html .   \n \n\n\u2191 \"Analytical Quality Control and Method Validation Procedures for Pesticide Residues Analysis in Food and Feed - Document No. SANTE\/11312\/2021\". EURL Portal. EU Reference Laboratories for Residues of Pesticides. 16 March 2022. https:\/\/www.eurl-pesticides.eu\/docs\/public\/tmplt_article.asp?CntID=727 .   \n \n\n\u2191 Dalmia, Avinash; Cudjoe, Erasmus; Jalali, Jacob; Qin, Feng (1 December 2021). \"A LC-MS\/MS method with electrospray ionization and atmospheric pressure chemical ionization source for analysis of pesticides in hemp\" (in en). Journal of Cannabis Research 3 (1): 50. doi:10.1186\/s42238-021-00106-9. ISSN 2522-5782. PMC PMC8670113. PMID 34903307. https:\/\/jcannabisresearch.biomedcentral.com\/articles\/10.1186\/s42238-021-00106-9 .   \n \n\n\u2191 9.0 9.1 Daniel, Daniela; Lopes, Fernando Silva; do Lago, Claudimir Lucio (1 October 2019). \"A sensitive multiresidue method for the determination of pesticides in marijuana by liquid chromatography\u2013tandem mass spectrometry\" (in en). Journal of Chromatography A 1603: 231\u2013239. doi:10.1016\/j.chroma.2019.07.006. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S002196731930737X .   \n \n\n\u2191 Maguire, Wesley J.; Call, Cameron W.; Cerbu, Cornel; Jambor, Katie L.; Benavides-Montes, Victor E. (20 November 2019). \"Comprehensive Determination of Unregulated Pesticide Residues in Oregon Cannabis Flower by Liquid Chromatography Paired with Triple Quadrupole Mass Spectrometry and Gas Chromatography Paired with Triple Quadrupole Mass Spectrometry\". Journal of Agricultural and Food Chemistry 67 (46): 12670\u201312674. doi:10.1021\/acs.jafc.9b01559. ISSN 1520-5118. PMID 31398037. https:\/\/pubmed.ncbi.nlm.nih.gov\/31398037 .   \n \n\n\u2191 Wittayanan, Weerawut; Chaimongkol, Thoranit (2021). \"Determination of pesticides residue in cannabis, cannabis extract and cannabis oil by gas chromatography tandem mass spectrometry technique\". Pharmaceutical Sciences Asia 48 (4): 354\u2013366. doi:10.29090\/psa.2021.04.20.107. https:\/\/pharmacy.mahidol.ac.th\/journal\/journalabstract.php?jvol=48&jpart=4&jconnum=6 .   \n \n\n\u2191 Cuypers, Eva; Vanhove, Wouter; Gotink, Joachim; Bonneure, Arne; Van Damme, Patrick; Tytgat, Jan (1 August 2017). \"The use of pesticides in Belgian illicit indoor cannabis plantations\" (in en). Forensic Science International 277: 59\u201365. doi:10.1016\/j.forsciint.2017.05.016. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0379073817301871 .   \n \n\n\u2191 Schneider, Serge; Bebing, Roger; Dauberschmidt, Carole (2014). \"Detection of pesticides in seized illegal cannabis plants\" (in en). Anal. Methods 6 (2): 515\u2013520. doi:10.1039\/C3AY40930A. ISSN 1759-9660. http:\/\/xlink.rsc.org\/?DOI=C3AY40930A .   \n \n\n\u2191 Stempfer, Miriam; Reinstadler, Vera; Lang, Anna; Oberacher, Herbert (1 October 2021). \"Analysis of cannabis seizures by non-targeted liquid chromatography-tandem mass spectrometry\" (in en). Journal of Pharmaceutical and Biomedical Analysis 205: 114313. doi:10.1016\/j.jpba.2021.114313. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0731708521004246 .   \n \n\n\u2191 \"Buying Cannabis \u2013 What You Need To Know\". Public Safety Canada. Government of Canada. 2 August 2023. https:\/\/www.publicsafety.gc.ca\/cnt\/cntrng-crm\/llgl-drgs\/llgl-nln-sls-cnnbs-en.aspx .   \n \n\n\nNotes \nThis presentation is faithful to the original, with only a few minor changes to presentation. Some grammar and punctuation was cleaned up to improve readability. In some cases important information was missing from the references, and that information was added.\n\n\n\n\n\nSource: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Journal:High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method\">https:\/\/www.cannaqa.wiki\/index.php?title=Journal:High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method<\/a>\nCategories: CannaQAwiki journal articles (added in 2023)CannaQAwiki journal articles (all)CannaQAwiki journal articles on cannabis researchCannaQAwiki journal articles on cannabis testingNavigation menuPage actionsJournalDiscussionView sourceHistoryPage actionsJournalDiscussionMoreToolsIn other languagesPersonal toolsLog inNavigationMain pageList of articlesRandom pageRecent changesHelpSearch\u00a0 ToolsWhat links hereRelated changesSpecial pagesPrintable versionPermanent linkPage information This page was last edited on 31 August 2023, at 20:49.Content is available under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License unless otherwise noted.Privacy policyAbout CannaQAWikiDisclaimers\n","147ac42afe5c27b45d4ed44a531a6754_html":"<body class=\"mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-170 ns-subject page-Journal_High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method rootpage-Journal_High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method skin-monobook action-view skin--responsive\"><div id=\"rdp-ebb-globalWrapper\"><div id=\"rdp-ebb-column-content\"><div id=\"rdp-ebb-content\" class=\"mw-body\" role=\"main\"><a id=\"rdp-ebb-top\"><\/a>\n<h1 id=\"rdp-ebb-firstHeading\" class=\"firstHeading\" lang=\"en\">Journal:High levels of pesticides found in illicit cannabis inflorescence compared to licensed samples in Canadian study using expanded 327 pesticides multiresidue method<\/h1><div id=\"rdp-ebb-bodyContent\" class=\"mw-body-content\"><!-- start content --><div id=\"rdp-ebb-mw-content-text\" lang=\"en\" dir=\"ltr\" class=\"mw-content-ltr\"><div class=\"mw-parser-output\">\n\n\n<h2><span class=\"mw-headline\" id=\"Abstract\">Abstract<\/span><\/h2>\n<p><b>Background<\/b>: As <i><a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabis\" title=\"Cannabis\" class=\"wiki-link\" data-key=\"a70b76268930d795518ff1f98d7e500d\">Cannabis<\/a><\/i> was legalized in Canada for recreational use in 2018 with the implementation of the <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabis_Act\" title=\"Cannabis Act\" class=\"wiki-link\" data-key=\"acc321d5bb52ef6b9d355d82a1238aec\">Cannabis Act<\/a>, regulations were put in place to ensure safety and consistency across the cannabis industry. This includes the requirement for licence holders to demonstrate that no unauthorized <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Pesticide\" title=\"Pesticide\" class=\"wiki-link\" data-key=\"073c24701223ff2ad1d3e2cc0058b2de\">pesticides<\/a> are used to treat cannabis or have <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Contamination\" title=\"Contamination\" class=\"wiki-link\" data-key=\"0663203ff7531a97a6d414b3ecc2c94d\">contaminated<\/a> it. In this study, we describe an expanded 327 multi-residue pesticide analysis in cannabis <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Inflorescence\" title=\"Inflorescence\" class=\"wiki-link\" data-key=\"d8122354225ee6a932a086ca8a1b999f\">inflorescence<\/a> to confirm if the implementation of the Cannabis Act is providing safer licensed products to Canadians in comparison to those of the illicit market.\n<\/p><p><b>Methods<\/b>: An extensive multi-residue method was developed using a modified quick, easy, cheap, effective, rugged, and safe (<a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Quechers\" title=\"Quechers\" class=\"wiki-link\" data-key=\"af67a9ed87921d0a448df9cada475eb4\">QuEChERS<\/a>) <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Sample_(material)\" title=\"Sample (material)\" class=\"wiki-link\" data-key=\"2d7596b7c3616db8f56af698be478d3d\">sample<\/a> preparation method using a combination of <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Gas_chromatography\" title=\"Gas chromatography\" class=\"wiki-link\" data-key=\"5094f035cb8bead5003deb9181e398b8\">gas chromatography<\/a>\u2014<a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Triple_quadrupole_mass_spectrometer\" title=\"Triple quadrupole mass spectrometer\" class=\"wiki-link\" data-key=\"8e1c17c1001dfcc71142b3d94a5fa86e\">triple quadrupole mass spectrometry<\/a> (GC\u2013MS\/MS) and <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Liquid_chromatography%E2%80%93mass_spectrometry\" title=\"Liquid chromatography\u2013mass spectrometry\" class=\"wiki-link\" data-key=\"e2eb804872f9ac16965fce8e49cfa3b6\">liquid chromatography<\/a>\u2014triple quadrupole mass spectrometry (LC\u2013MS\/MS) for the simultaneous quantification of 327 active pesticide ingredients in cannabis inflorescence.\n<\/p><p><b>Results<\/b>: Application of this method to Canadian licensed inflorescence samples revealed a six percent sample positivity rate, with only two pesticide residues detected\u2014<a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Myclobutanil\" title=\"Myclobutanil\" class=\"wiki-link\" data-key=\"8123e3dee3f223682581d1d2214d5b5f\">myclobutanil<\/a> and <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Dichlobenil\" class=\"mw-redirect wiki-link\" title=\"Dichlobenil\" data-key=\"4fbab83372f13c19e20a7a7bf4c02071\">dichlobenil<\/a>\u2014at the method\u2019s lowest calibrated level (LCL) of 0.01 \u03bcg\/g. Canadian illicit cannabis inflorescence samples that were analyzed showed a striking contrast with a 92 percent sample positivity rate, covering 23 unique pesticide active ingredients with 3.7 different pesticides identified on average per sample. <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Chlorpyrifos\" title=\"Chlorpyrifos\" class=\"wiki-link\" data-key=\"3ff054630abcddac8fec8c168419321b\">Chlorpyrifos<\/a>, <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Imidacloprid\" title=\"Imidacloprid\" class=\"wiki-link\" data-key=\"61a73e2859b5d2d8040bd57cc7b44cef\">imidacloprid<\/a>, and <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Myclobutanil\" title=\"Myclobutanil\" class=\"wiki-link\" data-key=\"8123e3dee3f223682581d1d2214d5b5f\">myclobutanil<\/a> were measured in illicit samples at concentrations up to three orders of magnitude above the method LCL of 0.01 \u03bcg\/g.\n<\/p><p><b>Conclusion<\/b>: These results demonstrate the need of an extensive multi-residue method capable of analyzing hundreds of pesticides simultaneously, to generate data for future policy and <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Regulatory_compliance\" title=\"Regulatory compliance\" class=\"wiki-link\" data-key=\"96ef4e38e482cf2be714a7092a6f9caa\">regulatory decision-making<\/a>, and to enable Canadians to make safe cannabis choices.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Introduction\">Introduction<\/span><\/h2>\n<p>In 2018, Canada legalized the recreational usage of <i><a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabis\" title=\"Cannabis\" class=\"wiki-link\" data-key=\"a70b76268930d795518ff1f98d7e500d\">Cannabis<\/a><\/i>, supplementing the framework for cannabis for medical purposes, which had been in place since 2001. With the <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabis_Act\" title=\"Cannabis Act\" class=\"wiki-link\" data-key=\"acc321d5bb52ef6b9d355d82a1238aec\">Cannabis Act<\/a><sup id=\"rdp-ebb-cite_ref-1\" class=\"reference\"><a href=\"#cite_note-1\">[1]<\/a><\/sup> and its associated regulations<sup id=\"rdp-ebb-cite_ref-2\" class=\"reference\"><a href=\"#cite_note-2\">[2]<\/a><\/sup> coming into force in 2018, Canada sought to standardize and enforce consistency, health, and safety across Canada\u2019s legal cannabis industry. To ensure safe cannabis products to Canadians, Health Canada regulates <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Microorganism\" title=\"Microorganism\" class=\"wiki-link\" data-key=\"1a8a1e22218cbeacf9ab84615f8254fe\">microbial<\/a> and chemical <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Contamination\" title=\"Contamination\" class=\"wiki-link\" data-key=\"0663203ff7531a97a6d414b3ecc2c94d\">contaminants<\/a>, including <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Pesticide\" title=\"Pesticide\" class=\"wiki-link\" data-key=\"073c24701223ff2ad1d3e2cc0058b2de\">pesticides<\/a>. In addition to the existing analytical testing requirements under Canada's cannabis regulations, since January 2019, the industry must also follow the mandatory requirements for testing cannabis for pesticide active ingredients<sup id=\"rdp-ebb-cite_ref-:0_3-0\" class=\"reference\"><a href=\"#cite_note-:0-3\">[3]<\/a><\/sup>, where license holders must demonstrate that none of the 96 unauthorized pesticide active ingredients are used to treat cannabis or have contaminated it.\n<\/p><p>Prior to the regulations going into effect in January 2019, some 18 percent of licensed cannabis products contained unregistered pesticides, with <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Myclobutanil\" title=\"Myclobutanil\" class=\"wiki-link\" data-key=\"8123e3dee3f223682581d1d2214d5b5f\">myclobutanil<\/a>, bifenazate, <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Boscalid\" title=\"Boscalid\" class=\"wiki-link\" data-key=\"52eccfd8a8f95257a8bad5da4c6a34e5\">boscalid<\/a>, and <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Fludioxonil\" title=\"Fludioxonil\" class=\"wiki-link\" data-key=\"feaf8c1f71283087a010f959cfbebb88\">fludioxonil<\/a> pesticides most commonly present<sup id=\"rdp-ebb-cite_ref-:1_4-0\" class=\"reference\"><a href=\"#cite_note-:1-4\">[4]<\/a><\/sup>, and myclobutanil notably being classified as moderately hazardous by the World Health Organization (WHO).<sup id=\"rdp-ebb-cite_ref-5\" class=\"reference\"><a href=\"#cite_note-5\">[5]<\/a><\/sup> This study aims to determine if unregistered pesticides are still prevalent in the licensed market. To gain a broader view of pesticide usage during cannabis production, we streamlined, expanded, and validated a single method using a combination of <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Gas_chromatography\" title=\"Gas chromatography\" class=\"wiki-link\" data-key=\"5094f035cb8bead5003deb9181e398b8\">gas chromatography<\/a>\u2014<a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Triple_quadrupole_mass_spectrometer\" title=\"Triple quadrupole mass spectrometer\" class=\"wiki-link\" data-key=\"8e1c17c1001dfcc71142b3d94a5fa86e\">triple quadrupole mass spectrometry<\/a> (GC\u2013MS\/MS) and <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Liquid_chromatography%E2%80%93mass_spectrometry\" title=\"Liquid chromatography\u2013mass spectrometry\" class=\"wiki-link\" data-key=\"e2eb804872f9ac16965fce8e49cfa3b6\">liquid chromatography<\/a>\u2014triple quadrupole mass spectrometry (LC\u2013MS\/MS) for the simultaneous quantification of 327 pesticide active ingredients in cannabis <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Inflorescence\" title=\"Inflorescence\" class=\"wiki-link\" data-key=\"d8122354225ee6a932a086ca8a1b999f\">inflorescence<\/a>, going well beyond the mandatory testing of 96 pesticide active ingredients.<sup id=\"rdp-ebb-cite_ref-:0_3-1\" class=\"reference\"><a href=\"#cite_note-:0-3\">[3]<\/a><\/sup> Although use of the licensed, legal cannabis market has been gaining ground in Canada since legalization, up to 13 percent of Canadians still report consuming illicit cannabis almost exclusively.<sup id=\"rdp-ebb-cite_ref-6\" class=\"reference\"><a href=\"#cite_note-6\">[6]<\/a><\/sup> As such, illicit cannabis samples were also analyzed for pesticides in this study to determine how they compare to the Canadian licensed cannabis market.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Methods\">Methods<\/span><\/h2>\n<h3><span class=\"mw-headline\" id=\"Sampling\">Sampling<\/span><\/h3>\n<p>To reflect as realistically as possible the sources of cannabis inflorescence available to Canadians across the country, 36 licensed samples were purchased in 2021 from the Ontario Cannabis Store (Ontario, Canada) from license holders located in all five Canadian regions (British Columbia, Prairies, Ontario, Quebec, and Atlantic) (Table 1). The 24 illicit cannabis samples were obtained from seizures by law enforcement officers across the country and submitted to Health Canada for <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Laboratory\" title=\"Laboratory\" class=\"wiki-link\" data-key=\"ed14e6a67b4b14ad2c190c28455725f6\">laboratory<\/a> testing in 2021.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"3\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 1.<\/b> Geographical distribution of cannabis (<i><a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabis_sativa\" title=\"Cannabis sativa\" class=\"wiki-link\" data-key=\"e003358742012354d1ff6002bc5781de\">C. sativa<\/a><\/i>) inflorescence samples obtained across Canada.\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Region\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Licensed samples\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Illicit samples\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">British Columbia\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">9\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Prairies\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">5\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Ontario\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">12\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">5\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Qu\u00e9bec\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">6\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">14\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Atlantic\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">4\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Total<\/b>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>36<\/b>\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>24<\/b>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h3><span class=\"mw-headline\" id=\"Standards_and_reagents\">Standards and reagents<\/span><\/h3>\n<p>Pesticide analytical standards were purchased from Chemservice (West Chester, PA) and Sigma-Aldrich Canada (Oakville, ON). Analytical grade <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Acetone\" title=\"Acetone\" class=\"wiki-link\" data-key=\"89e520a43bf29eb1413f5e60c5ff30bc\">acetone<\/a> and toluene were purchased from EMD Millipore (Darmstadt, Germany). Analytical grade <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Acetonitrile\" title=\"Acetonitrile\" class=\"wiki-link\" data-key=\"582d25d2e9d0fbfc0c2dcc54938b6798\">acetonitrile<\/a> and Na<sub>2<\/sub>SO<sub>4<\/sub> were purchased from Fisher Scientific (Fairlawn, NJ). Water was obtained from a Milli-Q\u00ae Plus Ultra Pure Water system (Millipore Corp., Burlington, MA). Sepra C18-E was obtained from Phenomenex (Torrance, CA). Supelclean ENVI-Carb SPE Tubes were obtained from Supelco (Bellefonte, PA). Sep-Pak Classic NH2 Cartridges were obtained from Waters Corp. (Milford, MA).\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Apparatus\">Apparatus<\/span><\/h3>\n<p>For sample preparation, a laboratory blender 51BL30 (Stamford, Connecticut), a high-speed shaker (Spex Sample Prep Geno-Grinder; Fisher Scientific, Fairlawn, NJ), a centrifuge (Allegra X15R 208v; Beckman Coulter Inc., Brea, CA), a solvent evaporator (Xcelvap; Horizon Technologies, Salem, NH), and a rotary evaporator (Rotavpor R-114, B\u00dcCHI Labortechnik AG, Flawil, Switzerland) were used. Sample analysis was carried out on a GC\u2013MS\/MS 7010B gas chromatograph quadrupole mass spectrometer\/mass spectrometer (Agilent Technologies, Santa Clara, CA) and LC\u2013MS\/MS Exion HPLC 6500 Q-Trap triple-quadrupole mass spectrometer (AB Sciex, Framingham, MA).\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Standard_solution_preparation\">Standard solution preparation<\/span><\/h3>\n<p>High-concentration pesticide stock standard solutions were prepared from the purest analytical material commercially available, typically\u2009\u2265\u200995 percent. In general, stock standard solutions were prepared in the range of 1000\u20132500 \u03bcg\/mL in acetone for GC\u2013MS\/MS compounds, and in either 100 percent acetonitrile or 100 percent methanol for LC\u2013MS\/MS compounds. From these, intermediate and spiking standard solutions were prepared respectively at 50 \u03bcg\/mL and 1 \u03bcg\/mL. Calibration standards were prepared with each sample set at concentrations of 0.8\u2009\u00d7\u2009, 1\u2009\u00d7\u2009, 2\u2009\u00d7\u2009, 3\u2009\u00d7\u2009, and 5\u2009\u00d7\u2009the lowest calibrated level (LCL) in pesticide-free cannabis matrix extract to compensate for ion suppression\/enhancement effects.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Sample_preparation_of_dried_cannabis_flowers\">Sample preparation of dried cannabis flowers<\/span><\/h3>\n<p>Cannabis inflorescence samples (5\u201320 g) were homogenized in a laboratory blender. Acetonitrile (20 mL) was added to 2 g ground cannabis inflorescence sample and the mixture was extracted with a Geno-Grinder at 1750 rpm for two minutes. The tube was centrifuged at 4500 rpm for five minutes. Exactly 4 mL of the extract was added to a tube containing 1 g of dispersive C18 and shaken by Geno-Grinder at 1200 rpm for one minute. Exactly 2 mL was transferred to an ENVI-Carb\/Aminopropyl SPE containing 1 cm of Na<sub>2<\/sub>SO<sub>4<\/sub>, and eluted with 25 mL of 3:1 ACN:Toluene. The sample\u2019s solvent was exchanged to acetone, blown down to less than 1 mL using a rotary evaporator, and 20 \u03bcL of 5 \u03bcg\/mL 2,4,6-tribromobiphenyl was added as an internal standard. The sample was diluted to 1 mL with acetone. Half of the extract was transferred to a vial for GC\u2013MS\/MS analysis. The remaining portion\u2019s solvent was exchanged to acetonitrile with solvent evaporator, brought to approximately 0.1 mL. Twenty microliters of isoprocarb 5 \u03bcg\/mL was added as an internal standard, which was then diluted to 0.5 mL with acetonitrile and brought to 1 mL with H<sub>2<\/sub>O. The sample was filtered using a 1-cc plastic syringe and a 0.2-\u00b5m filter and transferred to a vial for LC\u2013MS\/MS analysis.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Instrument_conditions\">Instrument conditions<\/span><\/h3>\n<h4><span id=\"rdp-ebb-LC\u2013MS\/MS\"><\/span><span class=\"mw-headline\" id=\"LC.E2.80.93MS.2FMS\">LC\u2013MS\/MS<\/span><\/h4>\n<p>Sample analysis was carried out using a 6500 Q-Trap LC-MSMS (AB Sciex). Analyst version 1.6.3 (AB Sciex) and MultiQuant version 3.0.2 (AB Sciex) software were used for instrument control and data analysis, respectively. A Kinetex C18 column (2.1\u2009\u00d7\u200950 mm, 2.6 \u03bcm) was used and maintained at 30 \u00b0C. The source was maintained at 550 \u00b0C. The following gas parameters were used: curtain gas, 35 psi; collision gas, 9psi; ion spray voltage, 5500 V; ion source gas 1, 50 psi; ion source gas 2, 55 psi. The injection volume was 1 \u03bcL. The mobile phases were water methanol (95\u2009+\u20095)\u2009+\u200910 mM formic acid\u2009+\u200910 mM ammonium formate (A) and water\u2013methanol (5\u2009+\u200995)\u2009+\u200910 mM formic acid\u2009+\u200910 mM ammonium formate (B). The flow rate was 0.7 mL\/min. The following elution gradient was used: 0\u201320 minutes, 0 percent B increasing to 100 percent B; 20\u201324.50 minutes, 100 percent B; 24.50\u201324.60 decreasing to 0 percent B then held from 24.60 to 25 minutes. Analysis was carried out by positive electrospray ionization using retention time-scheduled multiple reaction monitoring (MRM) to acquire two transitions (quantitative and qualitative) for each analyte. A partial list of these transition masses for both the LC\u2212MS\/MS and GC\u2212MS\/MS methods can be found in Supplementary information, Table S1 and S2, respectively.\n<\/p>\n<h4><span id=\"rdp-ebb-GC\u2013MS\/MS\"><\/span><span class=\"mw-headline\" id=\"GC.E2.80.93MS.2FMS\">GC\u2013MS\/MS<\/span><\/h4>\n<p>An Agilent 7010B GC\u2013MS\/MS carried out sample analysis. Mass Hunter software (Agilent) was used for instrument control and data analysis. The injection port was a multi mode injector (MMI) maintained at 250 \u00b0C. The liner was an inert double tapered splitless liner (Agilent # 5190\u20133983). The injection volume was 1 \u03bcL in splitless mode. Helium carrier gas was maintained at a constant flow of 1.0 mL\/min. ZB-Multiresidue-1 capillary columns were used (2 columns; each of 15 m\u2009\u00d7\u20090.25 mm\u2009\u00d7\u20090.25 \u03bcm) (Phenomenex # 7EG-G016-11-CI) with backflush procedure at mid-column. The front column was fitted with a 1-m retention gap of the same stationary phase. The oven temperature was maintained at 60 \u00b0C for one minute, ramped to 120 \u00b0C at 40 \u00b0C\/minute, then ramped to 310 \u00b0C at 5 \u00b0C\/min with a 11.5-minute hold (total run time: 52 minutes). The temperature of the MS source was maintained at 300 \u00b0C and the transfer line at 305 \u00b0C. Nitrogen was used as the collision gas at a flow of 1 mL\/minute. Analysis was carried out by electron impact ionization using dynamic MRM to acquire at least two transitions (quantitative and qualitative) for each analyte.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Validation_criteria\">Validation criteria<\/span><\/h3>\n<p>Quantitative validation data must show that specific pesticide\/matrix combinations can be accurately quantitated at the LCL deemed fit for purpose, the lowest value for the method being 0.01 \u00b5g\/g. The LCL for each pesticide was determined by an injection of a series of matrix-matched standards. The LCL was deemed acceptable if the signal of the LCL peak height to the height of the surrounding noise was at a minimum of 5:1 ratio for two transitions for the GC\u2013MS\/MS and LC\u2013MS\/MS. This ratio is the relative intensity of the quantifying ion\u2019s response compared to the qualifying ion\u2019s response. Ion ratios must be within permitted tolerances to be acceptable (Table 2).\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"2\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 2.<\/b> Permittable tolerance of quantifying ion responses compared to the qualifying ion relative intensity.\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Relative intensity (% of base peak)\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Permitted tolerance\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">>\u200950%\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">\u00b1\u200920%\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">>\u200920 to 50%\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">\u00b1\u200925%\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">>\u200910 to 20%\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">\u00b1\u200930%\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">\u2264\u200910%\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">\u00b1\u200950%\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>In addition, five replicate spikes at the LCL must meet method performance criteria of mean recoveries in the range of 70\u2013120 percent with an RSD\u2009\u2264\u200920%. Exceptionally, a mean recovery below 70 percent may be acceptable if the recovery is consistent with an RSD\u2009\u2264\u200920%.<sup id=\"rdp-ebb-cite_ref-7\" class=\"reference\"><a href=\"#cite_note-7\">[7]<\/a><\/sup>\n<\/p><p>The accuracy and precision of the pesticide recoveries were measured by spiking blank cannabis inflorescence matrix at the LCL (<i>n<\/i>\u2009=\u20095), 3\u2009\u00d7\u2009LCL (<i>n<\/i>\u2009=\u20093) and 5\u2009\u00d7\u2009LCL (<i>n<\/i>\u2009=\u20092). Linearity was established based on matrix-matched standards in the concentration range of 0.005\u20130.04 \u03bcg\/mL for LC-MS\/MS, 0.010\u20130.080 \u03bcg\/mL for GC-MS\/MS. The calibration curve generated from the standards must have a correlation coefficient (<i>R<sup>2<\/sup><\/i>) greater or equal to 0.99.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Quality_control\">Quality control<\/span><\/h3>\n<p>After the method was validated, samples were analyzed with <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Quality_control\" title=\"Quality control\" class=\"wiki-link\" data-key=\"a50e0afa52af1b1741e5ca1c7d711282\">quality control<\/a> (QC) measures in place for each sample set to ensure the integrity of the results. Each set of samples included a reagent blank, a matrix blank, and a representative matrix spike at the LCL for QC. A blank sample was spiked with 200 \u00b5L of 0.1 \u00b5g\/mL of GC\u2013MS\/MS and LC\u2013MS\/MS spiking solutions. The spike was allowed to stand for a minimum of 30 minutes. The blanks and spike were then processed the same way as the samples. To compensate for matrix effects on pesticides in plant material, all standards were made from pesticide-free cannabis inflorescence matrix extracts, with the addition of pesticides standards at various concentrations. Results were calculated using a six-point calibration curve (at concentrations of 0.8\u2009\u00d7\u2009, 1\u2009\u00d7\u2009, 2\u2009\u00d7\u2009, 3\u2009\u00d7\u2009, 5\u2009\u00d7\u2009, and 10\u2009\u00d7\u2009the LCL).\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Results\">Results<\/span><\/h2>\n<h3><span class=\"mw-headline\" id=\"Validation_data\">Validation data<\/span><\/h3>\n<p>To meet the requirements of the mandatory cannabis testing for 96 pesticide active ingredients<sup id=\"rdp-ebb-cite_ref-:0_3-2\" class=\"reference\"><a href=\"#cite_note-:0-3\">[3]<\/a><\/sup>, the new method was validated using more sensitive GC\u2013MS\/MS and LC\u2013MS\/MS instruments with better selectivity to detect 327 pesticides. Although most pesticides met the validation requirements for a LCL of 0.010 \u00b5g\/g, 31 percent of pesticides (101 out of 327) did not meet the 0.010 \u00b5g\/g LCL target and were validated at higher levels. These higher adjusted LCLs range from 0.02 \u00b5g\/g to 0.4 \u00b5g\/g (Supplementary information, Table S3). Overall, 285 pesticides tested meet validation criteria and can confidently give a quantitative result (Supplementary information, Table S3). While the remaining 42 pesticides did not pass the stringent quantification validation, they still met the criteria for monitoring their presence in cannabis inflorescence. When qualitatively identified in a sample, the word \"monitored\" is added for these 42 pesticides.\n<\/p><p>Mean recoveries in the range of 70\u2009\u2212\u2009120 percent, with a relative standard deviation (RSD)\u2009\u2264\u200920 percent between the 10 spiked replicates, were achieved for over 68 percent of the pesticides validated (Supplementary information, Table S3). Mean recoveries below 70 percent were still accepted (in the range of 30 to 69 percent only; lower than 30 percent is considered not recovered) if RSD\u2009\u2264\u200920 percent for compound recoveries at that level. Of the 285 pesticides that passed the validation, 22 percent adhere to this exception for lower 30\u201369 percent recoveries with an RSD\u2009\u2264\u200920 percent (Supplementary information, Table S3). Overall, our method demonstrated good linearity for 83 percent of pesticides attempted as the calibration curves had a correlation coefficient greater than 0.99.\n<\/p><p>It is important to note that <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Piperonyl_butoxide\" title=\"Piperonyl butoxide\" class=\"wiki-link\" data-key=\"6f5f69371b84f714c19c19661590e45a\">piperonyl butoxide<\/a> did not meet the validation criteria due to a large interference present in the reference material. The samples found positive were quantitated with a more targeted method with enough resolution to provide separation of the piperonyl butoxide and interfering signals to gain a better performance for this compound. A summary of the 12 additional recoveries outside of the validation (Supplementary information, Table S4) shows piperonyl butoxide has a better average recovery of 40 percent at 0.01 ppm, with an RSD of 21 percent at the low level (<i>n<\/i> =\u20096). At a higher spike concentration of 0.25 ppm, an average recovery of 73 percent was observed, with an RSD of 23 percent (<i>n<\/i>\u2009=\u20096). The correlation coefficient value for the curve used to calibrate these recoveries was acceptable (<i>R<sup>2<\/sup><\/i>\u2009=\u20090.9984). While the RSDs for these recoveries exceed the validation criteria, they provide more confidence in the ability of this method to qualitatively monitor piperonyl butoxide with an estimated concentration.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Application_of_method_to_real-world_samples\">Application of method to real-world samples<\/span><\/h3>\n<p>This newly expanded method was applied to real-world cannabis inflorescence samples available to Canadians across the country and used to determine if unregistered pesticide use is still prevalent in both the licensed and illicit markets. In total, 36 licensed samples and 24 illicit cannabis samples (Table 1) were analyzed against the method\u2019s 327 pesticides. Of the 36 licensed samples analyzed, only two pesticide residues were quantified (Table 3), representing a six percent positivity rate, with the measured concentration at our method LCL of 0.01 \u03bcg\/g.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"6\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 3.<\/b> Pesticide quantified in licensed and illicit samples obtained across Canada. <sup>a<\/sup> = Monitored.\n<\/td><\/tr>\n<tr>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Source\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Samples analyzed\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Sample positive rate (%)\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Pesticides detected\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Sample detection frequency\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Pesticide concentration range (\u00b5g\/g)\n<\/th><\/tr>\n<tr>\n<td rowspan=\"2\" style=\"background-color:white; padding-left:10px; padding-right:10px;\">Licensed\n<\/td>\n<td rowspan=\"2\" style=\"background-color:white; padding-left:10px; padding-right:10px;\">36\n<\/td>\n<td rowspan=\"2\" style=\"background-color:white; padding-left:10px; padding-right:10px;\">6\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Dichlobenil\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.01\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Myclobutanil\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.01\n<\/td><\/tr>\n<tr>\n<td rowspan=\"23\" style=\"background-color:white; padding-left:10px; padding-right:10px;\">Illicit\n<\/td>\n<td rowspan=\"23\" style=\"background-color:white; padding-left:10px; padding-right:10px;\">24\n<\/td>\n<td rowspan=\"23\" style=\"background-color:white; padding-left:10px; padding-right:10px;\">92\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Abamectin\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.06 to 0.6\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Azoxystrobin\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.2\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Bifenazate\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">4\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.009 to 0.1\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Boscalid\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.04\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Carbaryl\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.02 to 0.06\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Chlorphenapyr\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.5 to 5\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Chlorpyrifos\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">4\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.01 to 30\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Dichlorvos\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.05 to 1\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Fluopyram\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.03\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Imidacloprid\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.1 to 60\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Malaoxon\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.009\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Malathion\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.2\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Myclobutanil\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">17\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.02 to 70\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Paclobutrazol\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">10\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.009 to 1\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Permethrin\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">3\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.1 to 0.7\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Piperonyl butoxide\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">10\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.01 to 2<sup>a<\/sup>\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Pyrethrins\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">8\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.03 to 1\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Pyridaben\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.03\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Spinosad\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.2<sup>a<\/sup>\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Spirodiclofen\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.3\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Spiromesifen\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">2\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.2 to 1\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Spirotetramat\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.1<sup>a<\/sup>\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Tetramethrin\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.8\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>Pesticides were detected in 92 percent of Canadian illicit cannabis inflorescence samples, with 23 unique pesticide active ingredients quantified (Table 3). Four pesticides and synergists\u2014myclobutanil, paclobutrazol, piperonyl butoxide, and pyrethrins\u2014were detected at a high sample frequency rate, eight to 17 times in a total 24 illicit samples. One illicit sample alone contained nine different pesticide active ingredients. Illicit cannabis contained on average 3.7 different pesticides per sample, and 87 percent of positive samples contained more than one different pesticide. The pesticide concentrations quantified varied greatly, with chlorpyrifos, imidacloprid, and myclobutanil measured at 30, 60, and 70 \u03bcg\/g respectively, over three orders of magnitude higher that the method\u2019s LCLs of 0.01 \u03bcg\/g.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Discussion\">Discussion<\/span><\/h2>\n<p>The main objective of this study was to streamline and expand our existing cannabis inflorescence method<sup id=\"rdp-ebb-cite_ref-:1_4-1\" class=\"reference\"><a href=\"#cite_note-:1-4\">[4]<\/a><\/sup>, which was possible with more powerful instruments and enabled the addition of a GC\u2013MS\/MS quantification split. The existing modified quick, easy, cheap, effective, rugged, and safe (QuEChERS) extraction<sup id=\"rdp-ebb-cite_ref-:1_4-2\" class=\"reference\"><a href=\"#cite_note-:1-4\">[4]<\/a><\/sup> was adapted by eliminating the addition of water, salting-out, and enhanced matrix removal (EMR) clean-up steps, while dispersive C-18 replaced the C-18 column SPE for a four-fold time efficiency gain.\n<\/p><p>The method validation of the streamlined extraction and new instruments shows that all 285 pesticides meet the validation criteria based on the LCL, accuracy, precision, and linearity using matrix-matched standards. The remaining 42 pesticides that did not pass the stringent quantification validation still met the criteria for monitoring their presence in cannabis inflorescence. Cannabis inflorescence is a challenging matrix with its complex composition of oils, resins, <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Terpene\" title=\"Terpene\" class=\"wiki-link\" data-key=\"6bcbb95d582c08073101e9c4cfc91931\">terpenes<\/a>, and <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabinoid\" title=\"Cannabinoid\" class=\"wiki-link\" data-key=\"c224c3041748677fcdce5b5209900b7b\">cannabinoids<\/a>. The power and sensitivity of modern triple quadrupole mass spectrometry enable to reach most 0.010 \u00b5g\/g regulatory limits of quantification even while quantitatively detecting hundreds of pesticide and metabolite residues simultaneously. This validation data demonstrates that comprehensive testing of pesticides in cannabis inflorescence is achievable beyond the current 2019 testing requirements<sup id=\"rdp-ebb-cite_ref-:0_3-3\" class=\"reference\"><a href=\"#cite_note-:0-3\">[3]<\/a><\/sup>, enabling the provision of essential data for future policy and regulatory decision-making. These results are in-line with recent studies that successfully expanded their cannabis inflorescence pesticide method to several dozens<sup id=\"rdp-ebb-cite_ref-8\" class=\"reference\"><a href=\"#cite_note-8\">[8]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:2_9-0\" class=\"reference\"><a href=\"#cite_note-:2-9\">[9]<\/a><\/sup> and even hundreds<sup id=\"rdp-ebb-cite_ref-10\" class=\"reference\"><a href=\"#cite_note-10\">[10]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-11\" class=\"reference\"><a href=\"#cite_note-11\">[11]<\/a><\/sup> of different pesticide residues analyzed simultaneously.\n<\/p><p>Application of this expanded method to licensed cannabis inflorescence found a six percent sample positivity rate, with measured concentrations at the method\u2019s LCL of 0.01 \u03bcg\/g. Although quantified in one licensed sample, dichlobenil is not part of the mandatory cannabis testing for pesticide active ingredients list<sup id=\"rdp-ebb-cite_ref-:0_3-4\" class=\"reference\"><a href=\"#cite_note-:0-3\">[3]<\/a><\/sup>, indicating the importance of expanded multi-residue methods to generate valuable data for informed decisions regarding regulatory policies aimed at Canadian cannabis users who themselves want to make informed choices. Despite a six percent positivity rate, the licensed Canadian cannabis sector has greatly improved with regards to presence of pesticides since testing requirements were enacted in 2019, given the sample positivity rate of 30 percent prior to 2019.<sup id=\"rdp-ebb-cite_ref-:1_4-3\" class=\"reference\"><a href=\"#cite_note-:1-4\">[4]<\/a><\/sup>\n<\/p><p>In a striking contrast, Canadian illicit cannabis inflorescence samples show a 92 percent positivity rate, with 23 unique ctive pesticide ingredients quantified and at concentrations up to three orders of magnitude higher that the method\u2019s LCLs of 0.01 \u03bcg\/g. High illicit cannabis pesticide positivity rates have been also observed in other jurisdictions.<sup id=\"rdp-ebb-cite_ref-:2_9-1\" class=\"reference\"><a href=\"#cite_note-:2-9\">[9]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-12\" class=\"reference\"><a href=\"#cite_note-12\">[12]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-13\" class=\"reference\"><a href=\"#cite_note-13\">[13]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-14\" class=\"reference\"><a href=\"#cite_note-14\">[14]<\/a><\/sup> To the authors\u2019 knowledge, this study is the only extensive pesticide multi-residue analysis that compares pesticides in the licensed and illicit cannabis markets in a nationwide jurisdiction where cannabis has been legalized. Albeit being a small study, our results do support the Government of Canada messaging where \"consuming illegal products could lead to adverse effects and other serious harms,\" noting that \"testing of illegal cannabis has found contaminants like pesticides and unacceptable levels of bacteria, lead, and arsenic.\"<sup id=\"rdp-ebb-cite_ref-15\" class=\"reference\"><a href=\"#cite_note-15\">[15]<\/a><\/sup>\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Conclusion\">Conclusion<\/span><\/h2>\n<p>This study demonstrates a new streamlined and expanded method for the detection of 327 pesticides in cannabis inflorescence via gas chromatography\u2014triple quadruple mass spectroscopy and liquid chromatography\u2014triple quadruple mass spectroscopy. The validation of this method determined 285 unique pesticides can be quantified at levels ranging from 0.01 to 0.4 \u00b5g\/g, and 42 pesticides analyzed qualitatively. This method was applied to real world samples from both licensed and illicit markets, revealing high presence and concentration of pesticides in illicit samples compared to samples from the licensed market. With a six percent sample positivity rete, the licensed Canadian cannabis sector has greatly improved with regards to presence of pesticides since the 2019 mandate on regulatory testing. As a first, this study demonstrates the importance of extensive pesticide multi-residue methods comparing pesticides in the licensed and illicit cannabis markets to generate valuable data for informed decisions regarding regulatory policies and for Canadian cannabis users making informed choices.\n<\/p>\n<h2><span id=\"rdp-ebb-Abbreviations,_acronyms,_and_initialisms\"><\/span><span class=\"mw-headline\" id=\"Abbreviations.2C_acronyms.2C_and_initialisms\">Abbreviations, acronyms, and initialisms<\/span><\/h2>\n<ul><li><b>GC\u2013MS\/MS<\/b>: Gas chromatography\u2013triple quadruple mass spectroscopy<\/li>\n<li><b>LC\u2013MS\/MS<\/b>: Liquid chromatography\u2013triple quadruple mass spectroscopy<\/li>\n<li><b>LCL<\/b>: Lowest calibrated level<\/li>\n<li><b>QC<\/b>: Quality control<\/li>\n<li><b>QuEChERS<\/b>: Quick, easy, cheap, effective, rugged, and safe<\/li>\n<li><b>RSD<\/b>: Relative standard deviation<\/li>\n<li><b>WHO<\/b>: World Health Organization<\/li><\/ul>\n<h2><span class=\"mw-headline\" id=\"Acknowledgements\">Acknowledgements<\/span><\/h2>\n<p>The authors would like to thank Health Canada Cannabis Laboratory for their assistance in the procurement of illicit cannabis samples.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Author_contributions\">Author contributions<\/span><\/h3>\n<p>MG led the method development, acquisition of cannabis material, and sample analysis. TM contributed to the method development and drafted material and results of the manuscript. KM, JT, and MB contributed to the method development and sample analysis. NS peer reviewed the method validation data. DRB allocated the laboratory resources for this project and wrote the manuscript. All authors read and approved the final manuscript.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Funding\">Funding<\/span><\/h3>\n<p>Open Access funding provided by Health Canada. None other to declare.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Availability_of_data_and_materials\">Availability of data and materials<\/span><\/h3>\n<p>The data is available from the corresponding author on reasonable request.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Competing_interests\">Competing interests<\/span><\/h3>\n<p>The authors declare that they have no competing interests.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"References\">References<\/span><\/h2>\n<div class=\"reflist references-column-width\" style=\"-moz-column-width: 30em; -webkit-column-width: 30em; column-width: 30em; list-style-type: decimal;\">\n<div class=\"mw-references-wrap mw-references-columns\"><ol class=\"references\">\n<li id=\"cite_note-1\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-1\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/laws-lois.justice.gc.ca\/eng\/acts\/c-24.5\/FullText.html\" target=\"_blank\">\"Cannabis Act (S.C. 2018, c. 16)\"<\/a>. <i>Justice Laws Website<\/i>. Government of Canada. 27 April 2023<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/laws-lois.justice.gc.ca\/eng\/acts\/c-24.5\/FullText.html\" target=\"_blank\">https:\/\/laws-lois.justice.gc.ca\/eng\/acts\/c-24.5\/FullText.html<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Cannabis+Act+%28S.C.+2018%2C+c.+16%29&rft.atitle=Justice+Laws+Website&rft.date=27+April+2023&rft.pub=Government+of+Canada&rft_id=https%3A%2F%2Flaws-lois.justice.gc.ca%2Feng%2Facts%2Fc-24.5%2FFullText.html&rfr_id=info:sid\/en.wikipedia.org:Journal:High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-2\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-2\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/laws-lois.justice.gc.ca\/eng\/regulations\/sor-2018-144\/FullText.html\" target=\"_blank\">\"Cannabis Regulations (SOR\/2018-144)\"<\/a>. <i>Justice Laws Website<\/i>. Government of Canada. 2 December 2022<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/laws-lois.justice.gc.ca\/eng\/regulations\/sor-2018-144\/FullText.html\" target=\"_blank\">https:\/\/laws-lois.justice.gc.ca\/eng\/regulations\/sor-2018-144\/FullText.html<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Cannabis+Regulations+%28SOR%2F2018-144%29&rft.atitle=Justice+Laws+Website&rft.date=2+December+2022&rft.pub=Government+of+Canada&rft_id=https%3A%2F%2Flaws-lois.justice.gc.ca%2Feng%2Fregulations%2Fsor-2018-144%2FFullText.html&rfr_id=info:sid\/en.wikipedia.org:Journal:High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:0-3\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:0_3-0\">3.0<\/a><\/sup> <sup><a href=\"#cite_ref-:0_3-1\">3.1<\/a><\/sup> <sup><a href=\"#cite_ref-:0_3-2\">3.2<\/a><\/sup> <sup><a href=\"#cite_ref-:0_3-3\">3.3<\/a><\/sup> <sup><a href=\"#cite_ref-:0_3-4\">3.4<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation web\">Health Canada (30 August 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.canada.ca\/en\/public-health\/services\/publications\/drugs-health-products\/cannabis-testing-pesticide-requirements.html\" target=\"_blank\">\"Mandatory cannabis testing for pesticide active ingredients - Requirements\"<\/a>. Government of Canada<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.canada.ca\/en\/public-health\/services\/publications\/drugs-health-products\/cannabis-testing-pesticide-requirements.html\" target=\"_blank\">https:\/\/www.canada.ca\/en\/public-health\/services\/publications\/drugs-health-products\/cannabis-testing-pesticide-requirements.html<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Mandatory+cannabis+testing+for+pesticide+active+ingredients+-+Requirements&rft.atitle=&rft.aulast=Health+Canada&rft.au=Health+Canada&rft.date=30+August+2019&rft.pub=Government+of+Canada&rft_id=https%3A%2F%2Fwww.canada.ca%2Fen%2Fpublic-health%2Fservices%2Fpublications%2Fdrugs-health-products%2Fcannabis-testing-pesticide-requirements.html&rfr_id=info:sid\/en.wikipedia.org:Journal:High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:1-4\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:1_4-0\">4.0<\/a><\/sup> <sup><a href=\"#cite_ref-:1_4-1\">4.1<\/a><\/sup> <sup><a href=\"#cite_ref-:1_4-2\">4.2<\/a><\/sup> <sup><a href=\"#cite_ref-:1_4-3\">4.3<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Moulins, Jonathan R.; Blais, Michel; Montsion, Kim; Tully, Josee; Mohan, William; Gagnon, Mathieu; McRitchie, Tyler; Kwong, Keri <i>et al.<\/i> (1 November 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/29843862\" target=\"_blank\">\"Multiresidue Method of Analysis of Pesticides in Medical Cannabis\"<\/a>. <i>Journal of AOAC International<\/i> <b>101<\/b> (6): 1948\u20131960. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.5740%2Fjaoacint.17-0495\" target=\"_blank\">10.5740\/jaoacint.17-0495<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1944-7922\" target=\"_blank\">1944-7922<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/29843862\" target=\"_blank\">29843862<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/29843862\" target=\"_blank\">https:\/\/pubmed.ncbi.nlm.nih.gov\/29843862<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Multiresidue+Method+of+Analysis+of+Pesticides+in+Medical+Cannabis&rft.jtitle=Journal+of+AOAC+International&rft.aulast=Moulins&rft.aufirst=Jonathan+R.&rft.au=Moulins%2C%26%2332%3BJonathan+R.&rft.au=Blais%2C%26%2332%3BMichel&rft.au=Montsion%2C%26%2332%3BKim&rft.au=Tully%2C%26%2332%3BJosee&rft.au=Mohan%2C%26%2332%3BWilliam&rft.au=Gagnon%2C%26%2332%3BMathieu&rft.au=McRitchie%2C%26%2332%3BTyler&rft.au=Kwong%2C%26%2332%3BKeri&rft.au=Snider%2C%26%2332%3BNeil&rft.date=1+November+2018&rft.volume=101&rft.issue=6&rft.pages=1948%E2%80%931960&rft_id=info:doi\/10.5740%2Fjaoacint.17-0495&rft.issn=1944-7922&rft_id=info:pmid\/29843862&rft_id=https%3A%2F%2Fpubmed.ncbi.nlm.nih.gov%2F29843862&rfr_id=info:sid\/en.wikipedia.org:Journal:High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-5\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-5\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation book\">World Health Organization (2020) (in en). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/apps.who.int\/iris\/handle\/10665\/332193\" target=\"_blank\"><i>The WHO recommended classification of pesticides by hazard and guidelines to classification 2019<\/i><\/a>. WHO recommended classification of pesticides by hazard and guidelines to classification. Geneva: World Health Organization. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-92-4-000566-2<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/apps.who.int\/iris\/handle\/10665\/332193\" target=\"_blank\">https:\/\/apps.who.int\/iris\/handle\/10665\/332193<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=The+WHO+recommended+classification+of+pesticides+by+hazard+and+guidelines+to+classification+2019&rft.aulast=World+Health+Organization&rft.au=World+Health+Organization&rft.date=2020&rft.series=WHO+recommended+classification+of+pesticides+by+hazard+and+guidelines+to+classification&rft.place=Geneva&rft.pub=World+Health+Organization&rft.isbn=978-92-4-000566-2&rft_id=https%3A%2F%2Fapps.who.int%2Firis%2Fhandle%2F10665%2F332193&rfr_id=info:sid\/en.wikipedia.org:Journal:High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-6\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-6\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Health Canada (23 December 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.canada.ca\/en\/health-canada\/services\/drugs-medication\/cannabis\/research-data\/canadian-cannabis-survey-2021-summary.html\" target=\"_blank\">\"Canadian Cannabis Survey 2021: Summary\"<\/a>. Government of Canada<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.canada.ca\/en\/health-canada\/services\/drugs-medication\/cannabis\/research-data\/canadian-cannabis-survey-2021-summary.html\" target=\"_blank\">https:\/\/www.canada.ca\/en\/health-canada\/services\/drugs-medication\/cannabis\/research-data\/canadian-cannabis-survey-2021-summary.html<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Canadian+Cannabis+Survey+2021%3A+Summary&rft.atitle=&rft.aulast=Health+Canada&rft.au=Health+Canada&rft.date=23+December+2021&rft.pub=Government+of+Canada&rft_id=https%3A%2F%2Fwww.canada.ca%2Fen%2Fhealth-canada%2Fservices%2Fdrugs-medication%2Fcannabis%2Fresearch-data%2Fcanadian-cannabis-survey-2021-summary.html&rfr_id=info:sid\/en.wikipedia.org:Journal:High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-7\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-7\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.eurl-pesticides.eu\/docs\/public\/tmplt_article.asp?CntID=727\" target=\"_blank\">\"Analytical Quality Control and Method Validation Procedures for Pesticide Residues Analysis in Food and Feed - Document No. SANTE\/11312\/2021\"<\/a>. <i>EURL Portal<\/i>. EU Reference Laboratories for Residues of Pesticides. 16 March 2022<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.eurl-pesticides.eu\/docs\/public\/tmplt_article.asp?CntID=727\" target=\"_blank\">https:\/\/www.eurl-pesticides.eu\/docs\/public\/tmplt_article.asp?CntID=727<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Analytical+Quality+Control+and+Method+Validation+Procedures+for+Pesticide+Residues+Analysis+in+Food+and+Feed+-+Document+No.+SANTE%2F11312%2F2021&rft.atitle=EURL+Portal&rft.date=16+March+2022&rft.pub=EU+Reference+Laboratories+for+Residues+of+Pesticides&rft_id=https%3A%2F%2Fwww.eurl-pesticides.eu%2Fdocs%2Fpublic%2Ftmplt_article.asp%3FCntID%3D727&rfr_id=info:sid\/en.wikipedia.org:Journal:High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-8\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-8\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Dalmia, Avinash; Cudjoe, Erasmus; Jalali, Jacob; Qin, Feng (1 December 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/jcannabisresearch.biomedcentral.com\/articles\/10.1186\/s42238-021-00106-9\" target=\"_blank\">\"A LC-MS\/MS method with electrospray ionization and atmospheric pressure chemical ionization source for analysis of pesticides in hemp\"<\/a> (in en). <i>Journal of Cannabis Research<\/i> <b>3<\/b> (1): 50. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1186%2Fs42238-021-00106-9\" target=\"_blank\">10.1186\/s42238-021-00106-9<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2522-5782\" target=\"_blank\">2522-5782<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.pubmedcentral.gov\/articlerender.fcgi?tool=pmcentrez&artid=PMC8670113\" target=\"_blank\">PMC8670113<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/34903307\" target=\"_blank\">34903307<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/jcannabisresearch.biomedcentral.com\/articles\/10.1186\/s42238-021-00106-9\" target=\"_blank\">https:\/\/jcannabisresearch.biomedcentral.com\/articles\/10.1186\/s42238-021-00106-9<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+LC-MS%2FMS+method+with+electrospray+ionization+and+atmospheric+pressure+chemical+ionization+source+for+analysis+of+pesticides+in+hemp&rft.jtitle=Journal+of+Cannabis+Research&rft.aulast=Dalmia&rft.aufirst=Avinash&rft.au=Dalmia%2C%26%2332%3BAvinash&rft.au=Cudjoe%2C%26%2332%3BErasmus&rft.au=Jalali%2C%26%2332%3BJacob&rft.au=Qin%2C%26%2332%3BFeng&rft.date=1+December+2021&rft.volume=3&rft.issue=1&rft.pages=50&rft_id=info:doi\/10.1186%2Fs42238-021-00106-9&rft.issn=2522-5782&rft_id=info:pmc\/PMC8670113&rft_id=info:pmid\/34903307&rft_id=https%3A%2F%2Fjcannabisresearch.biomedcentral.com%2Farticles%2F10.1186%2Fs42238-021-00106-9&rfr_id=info:sid\/en.wikipedia.org:Journal:High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:2-9\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:2_9-0\">9.0<\/a><\/sup> <sup><a href=\"#cite_ref-:2_9-1\">9.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Daniel, Daniela; Lopes, Fernando Silva; do Lago, Claudimir Lucio (1 October 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S002196731930737X\" target=\"_blank\">\"A sensitive multiresidue method for the determination of pesticides in marijuana by liquid chromatography\u2013tandem mass spectrometry\"<\/a> (in en). <i>Journal of Chromatography A<\/i> <b>1603<\/b>: 231\u2013239. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.chroma.2019.07.006\" target=\"_blank\">10.1016\/j.chroma.2019.07.006<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S002196731930737X\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S002196731930737X<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+sensitive+multiresidue+method+for+the+determination+of+pesticides+in+marijuana+by+liquid+chromatography%E2%80%93tandem+mass+spectrometry&rft.jtitle=Journal+of+Chromatography+A&rft.aulast=Daniel&rft.aufirst=Daniela&rft.au=Daniel%2C%26%2332%3BDaniela&rft.au=Lopes%2C%26%2332%3BFernando+Silva&rft.au=do+Lago%2C%26%2332%3BClaudimir+Lucio&rft.date=1+October+2019&rft.volume=1603&rft.pages=231%E2%80%93239&rft_id=info:doi\/10.1016%2Fj.chroma.2019.07.006&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS002196731930737X&rfr_id=info:sid\/en.wikipedia.org:Journal:High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-10\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-10\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Maguire, Wesley J.; Call, Cameron W.; Cerbu, Cornel; Jambor, Katie L.; Benavides-Montes, Victor E. (20 November 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/31398037\" target=\"_blank\">\"Comprehensive Determination of Unregulated Pesticide Residues in Oregon Cannabis Flower by Liquid Chromatography Paired with Triple Quadrupole Mass Spectrometry and Gas Chromatography Paired with Triple Quadrupole Mass Spectrometry\"<\/a>. <i>Journal of Agricultural and Food Chemistry<\/i> <b>67<\/b> (46): 12670\u201312674. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1021%2Facs.jafc.9b01559\" target=\"_blank\">10.1021\/acs.jafc.9b01559<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1520-5118\" target=\"_blank\">1520-5118<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/31398037\" target=\"_blank\">31398037<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/31398037\" target=\"_blank\">https:\/\/pubmed.ncbi.nlm.nih.gov\/31398037<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Comprehensive+Determination+of+Unregulated+Pesticide+Residues+in+Oregon+Cannabis+Flower+by+Liquid+Chromatography+Paired+with+Triple+Quadrupole+Mass+Spectrometry+and+Gas+Chromatography+Paired+with+Triple+Quadrupole+Mass+Spectrometry&rft.jtitle=Journal+of+Agricultural+and+Food+Chemistry&rft.aulast=Maguire&rft.aufirst=Wesley+J.&rft.au=Maguire%2C%26%2332%3BWesley+J.&rft.au=Call%2C%26%2332%3BCameron+W.&rft.au=Cerbu%2C%26%2332%3BCornel&rft.au=Jambor%2C%26%2332%3BKatie+L.&rft.au=Benavides-Montes%2C%26%2332%3BVictor+E.&rft.date=20+November+2019&rft.volume=67&rft.issue=46&rft.pages=12670%E2%80%9312674&rft_id=info:doi\/10.1021%2Facs.jafc.9b01559&rft.issn=1520-5118&rft_id=info:pmid\/31398037&rft_id=https%3A%2F%2Fpubmed.ncbi.nlm.nih.gov%2F31398037&rfr_id=info:sid\/en.wikipedia.org:Journal:High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-11\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-11\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Wittayanan, Weerawut; Chaimongkol, Thoranit (2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pharmacy.mahidol.ac.th\/journal\/journalabstract.php?jvol=48&jpart=4&jconnum=6\" target=\"_blank\">\"Determination of pesticides residue in cannabis, cannabis extract and cannabis oil by gas chromatography tandem mass spectrometry technique\"<\/a>. <i>Pharmaceutical Sciences Asia<\/i> <b>48<\/b> (4): 354\u2013366. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.29090%2Fpsa.2021.04.20.107\" target=\"_blank\">10.29090\/psa.2021.04.20.107<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pharmacy.mahidol.ac.th\/journal\/journalabstract.php?jvol=48&jpart=4&jconnum=6\" target=\"_blank\">https:\/\/pharmacy.mahidol.ac.th\/journal\/journalabstract.php?jvol=48&jpart=4&jconnum=6<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Determination+of+pesticides+residue+in+cannabis%2C+cannabis+extract+and+cannabis+oil+by+gas+chromatography+tandem+mass+spectrometry+technique&rft.jtitle=Pharmaceutical+Sciences+Asia&rft.aulast=Wittayanan&rft.aufirst=Weerawut&rft.au=Wittayanan%2C%26%2332%3BWeerawut&rft.au=Chaimongkol%2C%26%2332%3BThoranit&rft.date=2021&rft.volume=48&rft.issue=4&rft.pages=354%E2%80%93366&rft_id=info:doi\/10.29090%2Fpsa.2021.04.20.107&rft_id=https%3A%2F%2Fpharmacy.mahidol.ac.th%2Fjournal%2Fjournalabstract.php%3Fjvol%3D48%26jpart%3D4%26jconnum%3D6&rfr_id=info:sid\/en.wikipedia.org:Journal:High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-12\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-12\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Cuypers, Eva; Vanhove, Wouter; Gotink, Joachim; Bonneure, Arne; Van Damme, Patrick; Tytgat, Jan (1 August 2017). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0379073817301871\" target=\"_blank\">\"The use of pesticides in Belgian illicit indoor cannabis plantations\"<\/a> (in en). <i>Forensic Science International<\/i> <b>277<\/b>: 59\u201365. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.forsciint.2017.05.016\" target=\"_blank\">10.1016\/j.forsciint.2017.05.016<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0379073817301871\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0379073817301871<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+use+of+pesticides+in+Belgian+illicit+indoor+cannabis+plantations&rft.jtitle=Forensic+Science+International&rft.aulast=Cuypers&rft.aufirst=Eva&rft.au=Cuypers%2C%26%2332%3BEva&rft.au=Vanhove%2C%26%2332%3BWouter&rft.au=Gotink%2C%26%2332%3BJoachim&rft.au=Bonneure%2C%26%2332%3BArne&rft.au=Van+Damme%2C%26%2332%3BPatrick&rft.au=Tytgat%2C%26%2332%3BJan&rft.date=1+August+2017&rft.volume=277&rft.pages=59%E2%80%9365&rft_id=info:doi\/10.1016%2Fj.forsciint.2017.05.016&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0379073817301871&rfr_id=info:sid\/en.wikipedia.org:Journal:High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-13\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-13\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Schneider, Serge; Bebing, Roger; Dauberschmidt, Carole (2014). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/xlink.rsc.org\/?DOI=C3AY40930A\" target=\"_blank\">\"Detection of pesticides in seized illegal cannabis plants\"<\/a> (in en). <i>Anal. Methods<\/i> <b>6<\/b> (2): 515\u2013520. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1039%2FC3AY40930A\" target=\"_blank\">10.1039\/C3AY40930A<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1759-9660\" target=\"_blank\">1759-9660<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/xlink.rsc.org\/?DOI=C3AY40930A\" target=\"_blank\">http:\/\/xlink.rsc.org\/?DOI=C3AY40930A<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Detection+of+pesticides+in+seized+illegal+cannabis+plants&rft.jtitle=Anal.+Methods&rft.aulast=Schneider&rft.aufirst=Serge&rft.au=Schneider%2C%26%2332%3BSerge&rft.au=Bebing%2C%26%2332%3BRoger&rft.au=Dauberschmidt%2C%26%2332%3BCarole&rft.date=2014&rft.volume=6&rft.issue=2&rft.pages=515%E2%80%93520&rft_id=info:doi\/10.1039%2FC3AY40930A&rft.issn=1759-9660&rft_id=http%3A%2F%2Fxlink.rsc.org%2F%3FDOI%3DC3AY40930A&rfr_id=info:sid\/en.wikipedia.org:Journal:High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-14\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-14\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Stempfer, Miriam; Reinstadler, Vera; Lang, Anna; Oberacher, Herbert (1 October 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0731708521004246\" target=\"_blank\">\"Analysis of cannabis seizures by non-targeted liquid chromatography-tandem mass spectrometry\"<\/a> (in en). <i>Journal of Pharmaceutical and Biomedical Analysis<\/i> <b>205<\/b>: 114313. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.jpba.2021.114313\" target=\"_blank\">10.1016\/j.jpba.2021.114313<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0731708521004246\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0731708521004246<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Analysis+of+cannabis+seizures+by+non-targeted+liquid+chromatography-tandem+mass+spectrometry&rft.jtitle=Journal+of+Pharmaceutical+and+Biomedical+Analysis&rft.aulast=Stempfer&rft.aufirst=Miriam&rft.au=Stempfer%2C%26%2332%3BMiriam&rft.au=Reinstadler%2C%26%2332%3BVera&rft.au=Lang%2C%26%2332%3BAnna&rft.au=Oberacher%2C%26%2332%3BHerbert&rft.date=1+October+2021&rft.volume=205&rft.pages=114313&rft_id=info:doi\/10.1016%2Fj.jpba.2021.114313&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0731708521004246&rfr_id=info:sid\/en.wikipedia.org:Journal:High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-15\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-15\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\"><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.publicsafety.gc.ca\/cnt\/cntrng-crm\/llgl-drgs\/llgl-nln-sls-cnnbs-en.aspx\" target=\"_blank\">\"Buying Cannabis \u2013 What You Need To Know\"<\/a>. <i>Public Safety Canada<\/i>. Government of Canada. 2 August 2023<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.publicsafety.gc.ca\/cnt\/cntrng-crm\/llgl-drgs\/llgl-nln-sls-cnnbs-en.aspx\" target=\"_blank\">https:\/\/www.publicsafety.gc.ca\/cnt\/cntrng-crm\/llgl-drgs\/llgl-nln-sls-cnnbs-en.aspx<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Buying+Cannabis+%E2%80%93+What+You+Need+To+Know&rft.atitle=Public+Safety+Canada&rft.date=2+August+2023&rft.pub=Government+of+Canada&rft_id=https%3A%2F%2Fwww.publicsafety.gc.ca%2Fcnt%2Fcntrng-crm%2Fllgl-drgs%2Fllgl-nln-sls-cnnbs-en.aspx&rfr_id=info:sid\/en.wikipedia.org:Journal:High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<\/ol><\/div><\/div>\n<h2><span class=\"mw-headline\" id=\"Notes\">Notes<\/span><\/h2>\n<p>This presentation is faithful to the original, with only a few minor changes to presentation. Some grammar and punctuation was cleaned up to improve readability. In some cases important information was missing from the references, and that information was added.\n<\/p>\n<!-- \nNewPP limit report\nCached time: 20231215215511\nCache expiry: 86400\nDynamic content: false\nComplications: []\nCPU time usage: 0.260 seconds\nReal time usage: 0.282 seconds\nPreprocessor visited node count: 15279\/1000000\nPost\u2010expand include size: 125740\/2097152 bytes\nTemplate argument size: 42782\/2097152 bytes\nHighest expansion depth: 25\/40\nExpensive parser function count: 0\/100\nUnstrip recursion depth: 0\/20\nUnstrip post\u2010expand size: 33828\/5000000 bytes\n-->\n<!--\nTransclusion expansion time report (%,ms,calls,template)\n100.00% 243.390 1 -total\n 77.47% 188.548 1 Template:Reflist\n 59.26% 144.230 15 Template:Citation\/core\n 36.73% 89.397 8 Template:Cite_journal\n 26.90% 65.471 6 Template:Cite_web\n 17.67% 43.011 1 Template:Infobox_journal_article\n 16.02% 38.997 1 Template:Infobox\n 11.26% 27.395 15 Template:Date\n 8.63% 21.016 17 Template:Citation\/identifier\n 6.80% 16.544 80 Template:Infobox\/row\n-->\n\n<!-- Saved in parser cache with key cannaqa_wiki:pcache:idhash:6008-0!canonical and timestamp 20231215215511 and revision id 18418. Serialized with JSON.\n -->\n<\/div><\/div><div class=\"printfooter\">Source: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Journal:High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method\">https:\/\/www.cannaqa.wiki\/index.php?title=Journal:High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method<\/a><\/div>\n<!-- end content --><div class=\"visualClear\"><\/div><\/div><\/div><div class=\"visualClear\"><\/div><\/div><!-- end of the left (by default at least) column --><div class=\"visualClear\"><\/div><\/div>\n<\/body>","147ac42afe5c27b45d4ed44a531a6754_images":[],"147ac42afe5c27b45d4ed44a531a6754_timestamp":1702682170,"e023a20c9343c9211ed31c45f339ab22_type":"article","e023a20c9343c9211ed31c45f339ab22_title":"Combined ambient ionization mass spectrometric and chemometric approach for the differentiation of hemp and marijuana varieties of Cannabis sativa (Chambers et al. 2023)","e023a20c9343c9211ed31c45f339ab22_url":"https:\/\/www.cannaqa.wiki\/index.php?title=Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa","e023a20c9343c9211ed31c45f339ab22_plaintext":"\n\nJournal:Combined ambient ionization mass spectrometric and chemometric approach for the differentiation of hemp and marijuana varieties of Cannabis sativaFrom CannaQAWikiJump to navigationJump to searchFull article title\n \nCombined ambient ionization mass spectrometric and chemometric approach for the differentiation of hemp and marijuana varieties of Cannabis sativaJournal\n \nJournal of Cannabis ResearchAuthor(s)\n \nChambers, Megan I.; Beyramysoltan, Samira; Garosi, Benedetta; Musah, Rabi A.Author affiliation(s)\n \nState University of New YorkPrimary contact\n \nEmail: rmusah at albany dot eduYear published\n \n2023Volume and issue\n \n5Article #\n \n5DOI\n \n10.1186\/s42238-023-00173-0ISSN\n \n2522-5782Distribution license\n \nCreative Commons Attribution 4.0 InternationalWebsite\n \nhttps:\/\/jcannabisresearch.biomedcentral.com\/articles\/10.1186\/s42238-023-00173-0Download\n \nhttps:\/\/jcannabisresearch.biomedcentral.com\/counter\/pdf\/10.1186\/s42238-023-00173-0.pdf (PDF)\n\nContents \n\n1 Abstract \n2 Background \n3 Materials and methods \n\n3.1 Cannabis sativa plant materials \n3.2 Mass spectral acquisition and analysis of DART-HRMS-derived data \n3.3 Multivariate data analysis \n\n\n4 Results \n\n4.1 DART-HRMS analysis of Cannabis sativa plant material \n4.2 Differentiation of hemp and marijuana varieties of C. sativa \n4.3 Classification of external C. sativa plant materials \n\n\n5 Discussion \n6 Conclusions \n7 Supplementary information \n8 Abbreviations, acronyms, and initialisms \n9 Acknowledgements \n\n9.1 Author contributions \n9.2 Funding \n9.3 Availability of data and materials \n9.4 Competing interests \n\n\n10 References \n11 Notes \n\n\n\nAbstract \nBackground: Hemp and marijuana are the two major varieties of Cannabis sativa. While both contain \u03949-tetrahydrocannabinol (THC), the primary psychoactive component of C. sativa, they differ in the amount of THC that they contain. Presently, U.S. federal laws stipulate that C. sativa containing greater than 0.3% THC is classified as marijuana, while plant material that contains less than or equal to 0.3% THC is hemp. Current methods to determine THC content are chromatography-based, which requires extensive sample preparation to render the materials into extracts suitable for sample injection, for complete separation and differentiation of THC from all other analytes present. This can create problems for forensic laboratories due to the increased workload associated with the need to analyze and quantify THC in all C. sativa materials.\nMethod: The work presented herein combines direct analysis in real time high-resolution mass spectrometry (DART-HRMS) and advanced chemometrics to differentiate hemp and marijuana plant materials. Samples were obtained from several sources (e.g., commercial vendors, DEA-registered suppliers, and the recreational Cannabis market). DART-HRMS enabled the interrogation of plant materials with no sample pretreatment. Advanced multivariate data analysis approaches, including random forest and principal component analysis (PCA), were used to optimally differentiate these two varieties with a high level of accuracy.\nResults: When PCA was applied to the hemp and marijuana data, distinct clustering that enabled their differentiation was observed. Furthermore, within the marijuana class, subclusters between recreational and DEA-supplied marijuana samples were observed. A separate investigation using the silhouette width index to determine the optimal number of clusters for the marijuana and hemp data revealed this number to be two. Internal validation of the model using random forest demonstrated an accuracy of 98%, while external validation samples were classified with 100% accuracy.\nDiscussion: The results show that the developed approach would significantly aid in the analysis and differentiation of C. sativa plant materials prior to launching painstaking confirmatory testing using chromatography. However, to maintain and\/or enhance the accuracy of the prediction model and keep it from becoming outdated, it will be necessary to continue to expand it to include mass spectral data representative of emerging hemp and marijuana strains\/cultivars.\nKeywords: Cannabis sativa, ambient ionization mass spectrometry, direct analysis in real time\u2014high-resolution mass spectrometry, multivariate data analysis, random forest, principal component analysis\n\nBackground \nAmong the greatest challenges to emerge for U.S. forensic laboratories in recent years are those attributed to the increased legalization and decriminalization of marijuana at the state level, in addition to the permitted production of hemp. The 2019 National Institute of Justice (NIJ) Report to Congress: Needs Assessment of Forensic Laboratories and Medical Examiner\/Coroner Offices identified this area as requiring focused attention towards improving criminal justice practices in the USA.[1] The challenge that hemp and marijuana present is as follows: both are major varieties of the same species Cannabis sativa, often referred to as Cannabis. While they each contain \u03949-tetrahydrocannabinol (THC), which is the primary psychoactive component of C. sativa, marijuana and hemp differ in the amount of this molecule that is present. In 2018, the U.S. federal guidelines stipulated that C. sativa which contains greater than 0.3% THC is a scheduled controlled substance (i.e., marijuana), while plant material that contains less than or equal to 0.3% is a legal agricultural commodity (i.e., hemp).[2] This definition has imposed severe challenges on crime labs. Among them is the dramatic increase in workload that results from the need to analyze and quantify the THC content of all C. sativa samples so that seized material can be appropriately designated. This is a time-consuming and resource-intensive enterprise that to greater and greater extents is consuming even larger forensic lab resources. Furthermore, defining the error cutoff for the 0.3% designation presents a challenge for the analysis of samples whose THC level is at the threshold.\nTraditionally, hemp and marijuana plant materials are differentiated by determining the THC content through chromatography-based approaches such as gas chromatography-flame ionization detection (GC-FID) and gas chromatography-mass spectrometry (GC\u2013MS)[3], in addition to high-performance liquid chromatography (HPLC) coupled to ultraviolet (UV) detection.[4] However, to accurately determine the THC content with these approaches, THC must be separated from all other components in the material (i.e., cannabinoids, terpenes, etc.) prior to quantification. One way to achieve this is to extend run times to allow for baseline separation between cannabinoids and other analytes present. Another option is to introduce a chemical derivatization step into the sample preparation protocol (which can be time-consuming), to differentiate between cannabinoids and their corresponding cannabinoid acids (e.g., THC and tetrahydrocannabinolic acid [THCA]). Although many investigations have been successful at differentiating between hemp and marijuana varieties or strains[5][6][7][8], the methods are reliant upon chromatography and are therefore susceptible to the aforementioned delineated challenges that can arise using this technique (i.e., lengthy run times, column contamination, etc.). Research towards developing, optimizing, and validating methods suitable for field testing of Cannabis materials has also been investigated.\nColorimetric tests represent a large percentage of these methods, which yield a presumptive result (by producing a color change)[9] when Cannabis-related substances are present, without the need for additional instrumentation (i.e., it is visible to the naked eye). Some examples include the 4-aminophenol test[10][11], Fast Blue BB test[11][12], and Duquenois-Levine test.[13] Similar to chromatography-based methods, these tests all rely upon the detection of THC specifically, which can complicate analyses because both marijuana and hemp contain this compound. Thus, while the distinction between marijuana and hemp has been defined based on THC levels, this is accompanied by several analytical challenges (i.e., baseline separation of molecules by chromatography-based methods, lengthy sample preparation protocols, and presumptive tests that can yield false positives[14], etc.).\nAn alternative less arbitrary approach is to base the distinction between them on the genome-defined differences in their metabolome signatures (i.e., small-molecule profiles). Studies utilizing the genetic profiles of Cannabis, such as genotyping-by-sequencing (GBS) and single-nucleotide polymorphisms (SNPs), have shown that, although they represent the same species, hemp and marijuana differ at the genome-wide level.[15][16][17] However, in addition to the fact that many crime laboratories are not positioned to integrate these types of analyses into current workflows, one of the bottlenecks to the routine use of the genome-defined small-molecule profiles for species attribution is the challenge of accessing this information quickly and reliably. One way to rapidly reveal this information, and subsequently distinguish between hemp and marijuana, is to combine an ambient ionization mass spectrometric technique\u2014e.g., direct analysis in real time high-resolution mass spectrometry (DART-HRMS)[18]\u2014with advanced statistical analysis. Ambient ionization methods (e.g., DART-HRMS, desorption electrospray ionization [DESI-MS]) have proven successful at screening for cannabinoids in Cannabis plant materials[19][20][21] and Cannabis-derived products (e.g., edibles, personal-care products, vape products, concentrates).[19][21] The unique capabilities of DART-HRMS are well-suited for the analysis of complex plant materials; the results are characterized by having high chemical information content, and little to no sample preparation prior to interrogating the materials is required. When applied to DART-HRMS-derived spectra, statistical data processing has enabled the successful differentiation of psychoactive plant species[22] and their headspace chemical signatures.[23] A modified version of DART-MS analysis introduced thermal desorption (TD) into the methodology (TD-DART-MS). One study utilized TD-DART-MS data to differentiate four hemp cultivars using PCA and partial least squares discriminant analysis (PLS-DA).[24] Another found that the application of statistical analysis to DART-MS data derived from methanolic extracts of hemp and marijuana samples revealed the potential for utilizing this method for optimally differentiating hemp and marijuana varieties.[25]\nThe study presented here, which is summarized in the scheme presented in Fig. 1, utilized DART-HRMS, for the first time, to investigate the complex genome-defined chemical fingerprints of hemp and marijuana (with no sample pretreatment) for the purpose of distinguishing between these two C. sativa varieties using multivariate statistical approaches. Advanced chemometrics was applied to the DART-HRMS data derived from commercial hemp, recreational marijuana, and marijuana samples from Drug Enforcement Administration (DEA)-registered suppliers to develop a robust model by which they (i.e., hemp and marijuana) could be readily differentiated. The success rate of the developed model\u2019s ability to predict external validation samples was 100%, indicating a high level of certainty. Importantly, the developed method circumvents the need to separate and differentiate cannabinoids by chromatography techniques (i.e., the traditional forensic approach for determining the THC concentration in a sample and which is used for differentiating between hemp and marijuana), in addition to bypassing all sample pretreatment steps.\n\r\n\n\n\n\n\n\n\n\n\n\nFig. 1. Workflow for discrimination of hemp and marijuana samples.\n\n\n\nMaterials and methods \nCannabis sativa plant materials \nTwenty-nine C. sativa flower samples of the hemp variety were purchased from three online vendors: (1) CBD Hemp Direct (Las Vegas, Nevada, USA), (2) Berkshire CBD (Brattleboro, Vermont, USA), and (3) Plain Jane (Berkeley, California, USA). These samples were used to build the model (i.e., training set). An additional 12 samples of hemp plant material were purchased from Plain Jane (Medford, Oregon, USA) at a later date to test the model (i.e., they were used for external validation). Additional information (e.g., cultivar\/strain, vendor, batch number) for these hemp materials is provided (see Additional file 1).\nC. sativa plant material of the marijuana variety was obtained from two DEA-registered sources. The National Institute on Drug Abuse (NIDA) (Research Triangle Park (RTP), North Carolina, USA) Drug Supply Program, which is part of the National Institutes of Health (NIH), provided the following four samples (i.e., cultivars) with varying levels of THC and cannabidiol (CBD) (the major non-psychoactive constituent in C. sativa): 1 g low THC cultivar (low THC\/very high CBD), 1 g medium THC cultivar (medium THC\/medium CBD), 1 g high THC cultivar (high THC\/low CBD), and 1 g very high THC cultivar (very high THC\/low CBD). The National Institute of Standards and Technology (NIST) (Gaithersburg, Maryland, USA) provided eight 0.5 g samples of marijuana that were confiscated by local law enforcement at different times over the past few years. Twenty-one strains of recreational marijuana were purchased from Garden Remedies Marijuana Dispensary (Melrose, Massachusetts, USA). Ten of the recreational samples were randomly selected for use in the development of the training model, while the remaining 11 samples were used to test the model (i.e., for external validation). Information for all marijuana samples (e.g., sample name, brand, supplier\/vendor, batch number, etc.) is available (see Additional file 1).\n\nMass spectral acquisition and analysis of DART-HRMS-derived data \nThe collection of mass spectral data was achieved by employing DART-HRMS. Two DART-HRMS instruments were used: (1) mass spectral data collected for all hemp products and the marijuana samples from DEA-registered suppliers were analyzed using the DART-HRMS instrument at the University at Albany (UAlbany) (Albany, New York, USA) and were translated and calibrated prior to data processing; and (2) all recreational marijuana flower samples were analyzed at IonSense Inc. (Saugus, Massachusetts, USA), with the raw data files calibrated, processed, and evaluated at UAlbany. The DART SVP (simplified voltage and pressure) ion source at IonSense was coupled to a JEOL AccuTOF high-resolution time-of-flight (TOF) mass spectrometer (Peabody, Massachusetts, USA) with a resolving power of 6000 full width at half maximum (FWHM) and mass accuracy of 5 millimass units (mmu). Data were collected in positive-ion mode using a DART ion source grid voltage of 300 V with the following mass spectrometer settings: ring lens, 5 V; orifice 1, 20 V; orifice 2 voltage, 5 V; peak voltage, 600 V; and detector voltage, 2000 V. The DART SVP ion source at UAlbany was also coupled to a JEOL AccuTOF high-resolution TOF mass spectrometer. The only difference between the DART ion source settings used at the two facilities was that the grid voltage at UAlbany was 250 V instead of 300 V. All mass spectral data were collected at a DART gas temperature of 350 \u00b0C using ultra-high purity helium gas at a flow rate of 2 L\/min. Mass spectra were collected at a rate of 1 spectrum per second over a mass range of m\/z 60\u20131000. TSSPro 3.0 software from Shrader Software Solutions (Grosse Pointe, Michigan, USA) was used for the calibration, spectral averaging, background subtraction, and peak centroiding of mass spectral data. Polyethylene glycol (PEG 600) (Sigma Aldrich, St. Louis, Missouri, USA) was used as the mass calibrant for all samples. Processing of the mass spectra of hemp and marijuana samples was performed with the Mass Mountaineer software suite from RBC Software (Portsmouth, New Hampshire, USA).\n\nMultivariate data analysis \nThe workflow which extended from DART-HRMS data collection to multivariate data analysis is displayed in Fig. 1. In Step 1, DART mass spectra of the C. sativa samples representing hemp and marijuana varieties were acquired. The spectra in the form of text files were imported into MATLAB 9.9.0, R2020b Software (The MathWorks, Inc., Natick, Massachusetts, USA) and R 3.5.1 (R Core Team 2018) for analysis. Each text file was comprised of a two-column matrix of m\/z values and their corresponding abundances (i.e., ion counts). In Step 2, peaks were aligned along common m\/z values by histogram estimation and nearest-neighbor correction methods using the \u201cmspalign\u201d function in MATLAB. The generated matrix contained the aligned spectra for the replicates of hemp and marijuana samples. The replicates for each sample were averaged, normalized, transformed (with log 10), and subjected to unsupervised (Step 3) and supervised analyses (Step 4). As shown in Step 3, PCA[26] and k-means[27][28] were used to recognize the similarity and dissimilarity patterns of the samples and to reveal possible clusters, respectively. Silhouette width indexes were calculated to indicate the optimal number of clusters characterized by k-means and to validate the goodness of the clustering results. The data matrix was analyzed using supervised random forest (RF)[29][30] (Step 4) to create a model for differentiating hemp and marijuana plant materials. RF is an ensemble of individual tree predictors, in which each tree in the forest is grown based on the independent replicas of training samples and variables. The samples not included in the replicates for a given tree (1\/3 of the original dataset) are termed \u201cout-of-bag\u201d (OOB) for that tree. The overall accuracy and performance characteristics of the discrimination model were estimated based on the predictions of OOB observations and external validation samples.\n\nResults \nDART-HRMS analysis of Cannabis sativa plant material \nInitial investigations of C. sativa plant material focused on obtaining the DART-HRMS chemical profiles for both hemp and marijuana flower samples. Detailed information about the samples, including variety, cultivar\/strain, vendor, and the batch number (when available) is provided (see Additional file 1). All samples were analyzed by inserting the closed end of a glass melting point capillary tube into the material and presenting the coated surface into the DART gas stream for approximately five seconds. A total of 29 hemp strains (i.e., cultivars) were purchased from three vendors at the beginning of this study, which included 27 CBD flower products and two cannabigerol (CBG) flower products. CBD flower contains high levels of CBD and cannabidiolic acid (CBDA), while CBG flower contains high levels of CBG and cannabigerolic acid (CBGA). An additional 12 hemp samples were purchased at a later date to test the developed model. Utilizing DART-HRMS is optimal for analyzing hemp and marijuana samples in their native forms (i.e., with no sample pretreatment, such as a decarboxylation step) to rapidly obtain the small-molecule profiles (i.e., in under one minute). The DART-HR mass spectra of all hemp flower samples (training-set hemp and test-set hemp) collected in positive-ion mode under soft ionization conditions (20 V) are available (see Additional file 2). \nFigure 2 shows representative DART-HR mass spectra acquired in positive-ion mode from analysis of C. sativa plant materials, including CBD (panel A) and CBG (panel D) hemp flower samples. The DART-HR mass spectra of all CBD hemp flower samples are very similar to one another; protonated masses consistent with CBD and CBDA were detected at m\/z 315 and 359, respectively, in all samples. DART-HRMS analysis of the two CBG hemp flower samples also yielded these peaks, in addition to peaks at nominal m\/z 317 and 361, which are consistent with the protonated masses of CBG and CBGA, respectively. The DART-HR mass spectra of the CBG hemp flower samples retained similarities with the CBD hemp flower profiles. However, indicative of the high CBG levels reported in the CBG flower samples, the relative intensities of the peaks attributed to CBG and CBGA were much higher in the DART-HR mass spectra of the CBG flower products.\n\r\n\n\n\n\n\n\n\n\n\n\nFig. 2. Representative DART-HR mass spectra of commercial hemp flower (panels A and D), marijuana samples supplied by NIST (panel B) and NIDA (panel E), and recreational marijuana flower products (panels C and F). Peaks consistent with the protonated masses of THC\/CBD, CBG, THCA\/CBDA, and CBGA at nominal m\/z 315, 317, 359, and 361, respectively, were detected in the various samples.\n\n\n\nC. sativa plant material of the marijuana variety was acquired from two U.S. DEA-registered sources: (1) NIDA supplied four marijuana samples (approximately 1 g each) through the NIDA\/NIH Drug Supply Program; and (2) NIST provided eight marijuana samples (0.5 g each). All 12 marijuana samples were received in powdered form and were analyzed by DART-HRMS in positive-ion mode using the capillary tube sampling technique. Figure 2 presents two spectra of representative NIST (panel B) and NIDA (panel E) marijuana materials. Commercially available recreational marijuana samples were also analyzed. The DART-HR mass spectra for all marijuana samples from these suppliers are available (see Additional file 2). In total, 21 recreational marijuana samples were purchased from the Garden Remedies Marijuana Dispensary Adult-Use Menu. These products spanned the various marijuana strain types available (i.e., indica-dominant, sativa-dominant, hybrid), which represent C. sativa subspecies. Figure 2 presents two representative DART-HR mass spectra for indica (panel C) and sativa (panel F) dominant flower samples. The mass spectral profiles of all recreational marijuana flower products are available (see Additional file 2). Ten of the samples were randomly selected for inclusion in the training model. The remaining 11 recreational flower samples were used to test the prediction ability of the model (i.e., for external validation).\n\nDifferentiation of hemp and marijuana varieties of C. sativa \nThe aim of this work was to accomplish the following: (1) develop a rapid, easy-to-use, and efficient means by which to differentiate hemp and marijuana varieties of C. sativa, and by extension, a method to identify C. sativa unknowns; and (2) circumvent some of the challenges typically encountered during the analysis of C. sativa materials when using chromatography-based methods. The approach is founded on the hypothesis that inherent in the small-molecule profiles of hemp and marijuana is the necessary information for the differentiation of these Cannabis varieties. Prior to the application of multivariate analysis methods to the features of the DART-HRMS-derived chemical profiles of hemp and marijuana, the spectra of all samples were binned to create a common m\/z reference vector to ease their comparison. Accordingly, the \u201cmspalign\u201d function in MATLAB was performed with a hist resolution parameter of 0.01, while the peak relative abundance cutoff threshold was set to 0.1% of the maximum intensity to detect all potentially significant peaks. The marijuana samples provided by NIDA and NIST were packaged in plastic bags, the composition of which contributed to the DART-HRMS profiles of the samples. Thus, the m\/z values derived from the packaging (e.g., nominal m\/z 59, 75, 89, 107, 127) were removed from the data. Another m\/z value that was removed was nominal m\/z 371, which has been previously shown to be a plasticizer present on the capillary tubes used for sampling.[31] The resulting matrix had dimensions of 430\u2009\u00d7\u2009390 and contained the aligned spectra for the five replicates of each of the 41 hemp samples, the five replicates of each of the 21 recreational marijuana samples, and the 10 replicates of each of the 12 marijuana samples supplied by NIDA and NIST. The results of the preliminary PCA analysis were examined by Q residuals and Hotelling\u2019s T2 statistic to detect any outliers, and this resulted in three spectra being removed from the data. Outlier spectra included those whose acquisition was accompanied by poor mass calibration or those that were not representative of a typical chemical profile. The averaging of sample replicates resulted in a matrix with dimensions of 74\u2009\u00d7\u2009390. Following logarithm transformation, the matrix was subjected to further analysis. Figure 3 panel A presents the PCA results as a 2-dimensional (2D) score plot, where the color-coded classes appear in the coordinate space represented by the first two principal components (PCs), which cover 41% of the data variance. While the recreational marijuana samples (cyan triangles) are located in close proximity to the NIDA-supplied marijuana sample that was reported to contain medium levels of both THC and CBD, they were distant from the other NIDA and NIST samples. These results support previous studies that indicated differences between marijuana sold at dispensaries, and that provided for research purposes by DEA-registered suppliers.[17][32] Clustering by k-means using one minus correlation metrics resulted in the categorization of the hemp samples into one cluster (magenta circles) and the marijuana samples into the other cluster (cyan circles).\n\r\n\n\n\n\n\n\n\n\n\n\nFig. 3. 2D score plot resulting from PCA of hemp and marijuana sample spectra (panel A); 2D score plot of multidimensional scaling (MDS) analysis of the proximity matrix resulting from the application of supervised random forest (panel B). The magenta and cyan colors represent hemp and marijuana, respectively. The cyan triangles show the subset of recreational marijuana samples.\n\n\n\nEven though the DART-HR mass spectra of hemp and marijuana plant materials are readily visually apparent, a more objective approach to the assessment of the identity of C. sativa material was devised, using the random forest algorithm. This was applied to the 74\u2009\u00d7\u2009390 matrix. A total of 33 flower samples (12 hemp and 11 marijuana) of the 74 total C. sativa samples were randomly selected for external validation to examine the ability of the model to accurately predict the class assignments for new sample unknowns. The number of variables (which were randomly sampled as candidates at each split), and the number of trees found to be optimal were 20 and 500, respectively. Figure 3, panel B displays the proximity matrix generated from using supervised RF with a multidimensional scaling (MDS) method to show the pairwise similarities in a 2D Cartesian space, with the magenta and cyan points corresponding to the hemp and marijuana samples, respectively. It demonstrates the number of times that observations ended up in the same leaf node. According to Figure 3, panel B, although the NIDA marijuana sample reported as low THC\/very high CBD is located between the two groups, the samples belonging to each group are close together and separated from the samples of the other group.\nThe optimal number of clusters was estimated by computing the average silhouette (which measures the quality of the clustering) of observations for different numbers of clusters. Figure 4, panel A displays the average silhouette width over a range of the possible number of clusters. The optimal number of clusters is the one that maximizes the average silhouette width. Based on the information provided in Figure 4, panel A, the optimal number of clusters is two. The silhouette plot in Figure 4, panel B displays silhouette coefficients for each sample when the data are split into two clusters. The silhouette width of each sample is a measure of how similar each sample is to its respective cluster in comparison to the other cluster. As shown in Figure 4, the optimum number of clusters is two: cluster 1 (magenta) has 40 members with a mean width of 0.23, and cluster 2 (cyan) has 34 members with a mean width of 0.45. Cluster 1 and cluster 2 members correspond to the samples of hemp and marijuana, respectively. One hemp sample was falsely clustered with the marijuana samples. The average silhouette width for the cluster of marijuana samples is higher than the average silhouette width for the hemp samples. This demonstrates that the cluster of marijuana samples is denser and that the samples are more similar to one another.\n\r\n\n\n\n\n\n\n\n\n\n\nFig. 4. The average silhouette width over a range of cluster numbers (2\u20136) reveals that the optimum number of clusters is 2 (panel A). A silhouette plot (i.e., the visualization of the silhouette width for each sample) reveals the results with two clusters (panel B). Cluster 1 contains 40 members and cluster 2 contains 34 members. Hemp samples are shown in magenta, while marijuana samples are shown in cyan.\n\n\n\nTo reveal the model\u2019s ability to distinguish between hemp and marijuana samples, Table 1 presents the confusion matrix for the prediction of OOB samples, while Table 2 contains the performance characteristics of the model (accuracy, sensitivity, specificity, and precision) for predicting the OOB samples. According to this table, the model performed well and the accuracy for predicting OOB samples is 98%.\n\n\n\n\n\n\n\nTable 1. Confusion matrix associated with the prediction of \u201cout-of-bag\u201d samples in the random forest model.\n\n\nConfusion matrix\n\nPrediction\n\n\nHemp\n\nMarijuana\n\n\nTrue\n\nHemp (29)\n\n1.00\n\n0.00\n\n\nMarijuana (22)\n\n0.04\n\n0.96\n\n\n\n\n\n\n\n\n\nTable 2. Performance results of the random forest model for prediction of \u201cout-of-bag\u201d and external validation samples.\n\n\n\n\nOut-of-bag samples\n\n\nAccuracy: 0.98 (98%)\n\n\nSensitivity\n\nSpecificity\n\nPrecision\n\n\nHemp (29)\n\n1.00\n\n0.96\n\n0.97\n\n\nMarijuana (22)\n\n0.96\n\n1.00\n\n1.00\n\n\n\n\nExternal C. sativa plant materials\n\n\nAccuracy: 1.00 (100%)\n\n\nSensitivity\n\nSpecificity\n\nPrecision\n\n\nHemp (12)\n\n1.00\n\n1.00\n\n1.00\n\n\nMarijuana (11)\n\n1.00\n\n1.00\n\n1.00\n\n\n\nClassification of external C. sativa plant materials \nThe remaining 11 recreational marijuana flower products that were not included in the training set, in addition to the 12 hemp products purchased after the model had been developed, were screened against the model to test its ability to classify samples that were unknown to the model. Table 3 shows the confusion matrix results for the prediction of the test samples (i.e., for external validation). In addition, Table 2 shows the performance characteristics of the model for predicting the external C. sativa samples, with all performance merits equal to 1 for both test sample sets (i.e., hemp and marijuana). The information presented in Tables 1, 2, and 3 reveal that the model is well-fitted for discriminating the two C. sativa varieties.\n\n\n\n\n\n\n\nTable 3. Confusion matrix associated with the prediction of external validation samples using a random forest model.\n\n\nConfusion matrix\n\nPrediction\n\n\nHemp\n\nMarijuana\n\n\nTrue\n\nHemp (12)\n\n1.00\n\n0.00\n\n\nMarijuana (11)\n\n0.00\n\n1.00\n\n\n\nDiscussion \nThe most common methods for differentiating hemp and marijuana plant materials are chromatography-based approaches (e.g., GC-FID, GC\u2013MS, HPLC\u2013UV)[3][4], with the categorization based upon THC content. Several reports have emphasized the use of GC-FID[8][33][34][35][36][37] and GC\u2013MS[33][36][38][39][40][41][42] methods for detection of natural cannabinoids (among other Cannabis-derived molecules) in various Cannabis plant materials. Modifications to standard GC-FID and GC\u2013MS protocols include GC-vacuum UV (VUV) spectroscopy[43], two-dimensional GC-FID (GCxGC-FID)[44], and GCxGC-MS with multivariate curve resolution-alternating least squares (MCR-ALS).[45] However, these methods rely upon the quantification of THC, which can be plagued with a number of analytical challenges, such as baseline separation of peaks and lengthy sample preparation protocols.\nIn an effort to circumvent the need to extend run times or incorporate extra sample preparation steps, several studies have investigated alternative sample collection techniques coupled with chromatography-based methods to differentiate C. sativa varieties. One study demonstrated the use of capillary microextraction of volatiles (CMV) coupled with GC\u2013MS to distinguish the headspace volatiles of marijuana and hemp products based on their apparently distinct volatile organic compound (VOC) profiles.[5] However, this report revealed that potential adulterants and inconsistent packaging of samples may have contributed to the observed distinctions.[5] Another study utilized GC\u2013MS coupled with dispersive pipette extraction (DPX) to investigate forensic casework marijuana and donated hemp samples.[6] Although the approach was successful at differentiating the two varieties with greater than 98% accuracy, a significant reduction of THC stability after 48 hours indicated that the samples would need to be reanalyzed if there was a delay between sample preparation and instrumental analysis.[6] Another GC-based study sought to differentiate hemp and marijuana through their cannabinoid and terpene profiles using GC-FID and principal component analysis (PCA).[7] This study, which included two recreational cultivars and three pharmacy Cannabis samples, successfully distinguished between the two C. sativa varieties.[7] In this case, expanding the sample source diversity could strengthen the ability of the model to classify a wider range of Cannabis samples. Another study applied PCA algorithms to quantitative data acquired from high-performance liquid chromatography-mass spectrometry (HPLC\u2013MS) analysis of Cannabis plant materials.[8] This study identified several cannabinoids essential for differentiating between Cannabis strain types[8] (i.e., strains within the marijuana variety) as opposed to specifically targeting the cannabinoids essential to differentiating C. sativa varieties (i.e., hemp and marijuana), which would be important for criminal justice purposes in the U.S. Although many of these investigations were successful at differentiating between hemp and marijuana varieties or strains, the methods are reliant upon chromatography and are therefore susceptible to the aforementioned delineated challenges that can arise using this technique (i.e., lengthy run times, column contamination, etc.).\nNon-chromatographic approaches that circumvent the requirement to separate and\/or differentiate between cannabinoids have also been investigated for distinguishing hemp and marijuana. A hand-held Raman spectrometer coupled with orthogonal partial least squares-discriminant analysis (OPLS-DA) tools proved successful in differentiating between the two C. sativa varieties.[46] However, \u201creal\u201d forensic casework samples are rarely received in pristine form, and as such, the Raman approach is susceptible to interferences from various components that may be associated with the complex matrix and interfere with the Raman signal. Another study utilized advanced statistical modeling of nuclear magnetic resonance (NMR) spectroscopy and mass spectral data of C. sativa extracts[47], which is unique in that it is typically difficult to utilize NMR for the analysis of complex matrices and mixtures. Although effective, this instrumentation is not commonly found in forensic or other Cannabis analysis laboratories due to expensive start-up and maintenance costs.\nColorimetric tests are also commonly used for differentiating between hemp and marijuana varieties of Cannabis, especially in forensic fieldwork, and these do not generally require instrumental analysis to arrive at a presumptive identification. A validated method utilizing the 4-aminophenol color test to differentiate hemp and marijuana revealed some degree of success.[10] However, this test can yield inconclusive results with samples that have THC and CBD levels that are within a factor of three of one another.[10] Another common color test for the identification of marijuana samples is the Fast Blue BB (FBBB) colorimetric test, which reacts with the cannabinoids present in Cannabis (primarily THC). A study utilizing this test found that hemp and marijuana plant materials could be classified correctly when linear discriminant analysis (LDA) was used to develop a model based on RGB (red, green, blue) numerical codes from both fluorescence and color images that resulted from the application of the FBBB color test.[12] Positive-ion mode electrospray ionization Fourier transform-ion cyclotron resonance mass spectrometry (ESI(\u2009+)FT-ICR MS, ESI(\u2009+)MS\/MS, ultraviolet\u2013visible (UV\u2013Vis) spectroscopy, and thin-layer chromatography (TLC) techniques have been used to investigate the products (i.e., chromophores) resulting from the application of the FBBB test to marijuana samples.[48] In addition, direct analysis in real time-mass spectrometry (DART-MS) and 1H NMR techniques were coupled to identify the chromophores produced when various cannabinoids react with the FBBB reagent.[49] A third color test to identify marijuana through the presence of THC is the Duquenois-Levine test. Research has been conducted to characterize (by mass spectrometry) the chromophores formed when cannabinoids react with the Duquenois reagents.[13][50][51] Similar to the chromatography-based methods described, these tests all rely upon detection of THC specifically, which can complicate analyses because both marijuana and hemp contain this compound. Thus, while the distinction between marijuana and hemp has been defined based on THC levels, this is accompanied by the several aforementioned analytical challenges. By using the entire metabolomic profiles of hemp and marijuana acquired through ambient ionization mass spectrometry, the method presented here does not rely solely on the presence of any one molecule (or set of molecules), ratios of molecules to one another, or the ability to differentiate between cannabinoid isomers (i.e., THC and CBD).\nThe overall results of this study reveal that DART-HRMS yields consistent and unique chemical profiles for analyzed Cannabis materials that enable hemp and marijuana samples to be accurately differentiated, while circumventing challenges typically encountered with traditional chromatography methods (difficulties with cannabinoid separation and extensive sample preparation) and presumptive color tests (inconclusive or false positive results). Furthermore, this study utilized a sample set that demonstrates a balance between the total number of samples included, the number of replicates obtained, and a diversity in sources from which the C. sativa materials were acquired. This research provides a strong foundation upon which to develop a comprehensive mass spectral database for identifying unknown C. sativa variants through the acquisition of their DART-HR mass spectra. While the approach does not aim to replace confirmatory testing for THC concentrations, the model accomplishes the following: (1) bypasses the typical sample preparation steps required for analyzing materials by chromatography-based methods that seek to differentiate the samples through separation of their constituent cannabinoids; (2) reduces the chances for false positives that can result from presumptive color tests; and (3) serves as a supplementary tool for forensic investigators that enables more targeted confirmatory testing. \nThis is timely and highly relevant, given the introduction in the U.S. House of Representatives of the \u201cH.R.6645 \u2013 Hemp Advancement Act of 2022\u201d bill.[52] This act aims to amend the current federal ruling regarding hemp by: (1) changing the 0.3% [THC] designation to 1% and (2) replacing the word \u201cdelta-9\u201d with the word \u201ctotal\u201d to include the various isomers of THC that have emerged in recent years.[52] The introduction of this bill underscores some of the disadvantages of utilizing THC cutoffs in particular as the sole means by which to identify hemp and marijuana. Among other issues, it upends well-established and long-standing practices in criminalistics in a fashion that is expensive to address, since it will require the development of an entirely new set of protocols and data processing steps. Furthermore, it may not stand the test of time, as the cutoff thresholds are subject to change in the future. A method such as the one presented here, and which does not solely rely upon a 0.3% THC cutoff, is not at risk of becoming outdated upon further advancements of this bill or others in the U.S. House and Senate.\n\nConclusions \nA combined ambient ionization mass spectrometric (i.e., DART-HRMS) and chemometric approach was successfully used to create a prediction model that facilitated rapid high-accuracy differentiation of C. sativa hemp and marijuana plant materials obtained from multiple sources (i.e., commercial, DEA-registered, recreational). This method, which circumvents sample pretreatment steps (i.e., solvent extractions), addresses some of the difficulties encountered when analyzing samples using more conventional forensic analysis methodologies. A primary example of this is eliminating the need to separate and differentiate cannabinoids by chromatography techniques in order to determine the sample\u2019s THC content, which is the primary basis for distinguishing between hemp and marijuana varieties of Cannabis for most methods. When new hemp and recreational marijuana flower products were screened against the model developed in this study, 100% accuracy in prediction was observed. The identities of m\/z values that were determined to be important for the optimal differentiation of hemp and marijuana are the subject of continuing investigations. In addition, it is possible that C. sativa materials (of either the hemp or marijuana variety) with atypical levels of minor cannabinoids (such as CBN or isomers of THC) may respond differently in the DART gas stream and that this, in turn, may influence the results predicted by the model. Therefore, samples such as these will be investigated (as was done with the analysis of the two CBG hemp flower samples), along with new samples\/strains from commercial and DEA-registered suppliers as they become available so that the model reflects ongoing changes in the chemical profiles of Cannabis products on the market.\n\nSupplementary information \nAdditional file 1 (.docx): Supplementary Mass Spectral Data and Sample Information. (1) Information about C. sativa plant materials analyzed in this study.\nAdditional file 2 (.docx): Supplementary Mass Spectral Data for C. sativa Materials. (1) DART-HR mass spectra for hemp and marijuana materials.\n Abbreviations, acronyms, and initialisms \n2D: two-dimensional\nCBD: cannabidiol\nCBDA: cannabidiolic acid\nCBG: cannabigerol\nCBGA: cannabigerolic acid\nCMV: capillary microextraction of volatiles\nDART: direct analysis in real-time\nDEA: U.S. Drug Enforcement Administration\nDESI: desorption electrospray ionization\nDPX: dispersive pipette extraction\nESI: electrospray ionization\nFBBB: Fast Blue BB\nFID: flame ionization detection\nFT: Fourier transform\nFWHM: full width at half maximum\nGBS: genotyping-by-sequencing\nGC: gas chromatography\nGCxGC: two-dimensional gas chromatography\nHR: high-resolution\nHRMS: high-resolution mass spectrometry\nHPLC: high-performance liquid chromatography\nICR: ion cyclotron resonance\nLDA: linear discriminant analysis\nMCD-ALS: multivariate curve resolution-alternating least squares\nMDS: multidimensional scaling\nmmu: millimass unit\nMS: mass spectrometry\nNIDA: ational Institute on Drug Abuse\nNIH: ational Institutes of Health\nNIJ: ational Institute of Justice\nNIST: ational Institute of Standards and Technology\nNMR: uclear magnetic resonance\nOOB: ut-of-bag\nOPLS-DA: rthogonal partial least squares-discriminant analysis\nPC: rincipal component\nPCA: rincipal component analysis\nPEG: olyethylene glycol\nPLS-DA: artial least squares-discriminant analysis\nRF: andom forest\nRGB: ed, green, blue\nRTP: Research Triangle Institute\nSNP: single-nucleotide polymorphism\nSUNY: State University of New York\nSVP: simplified voltage and pressure\nTD: thermal desorption\nTHCA: delta-9-tetrahydrocannabinolic acid\nTLC: thin-layer chromatography\nTOF: time-of-flight\nUAlbany: The University at Albany\nUV: ultraviolet\nVis: visible\nVOC: volatile organic compounds\nVUV: vacuum UV\n\u22069-THC or THC: delta-9-tetrahydrocannabinol\nAcknowledgements \nThanks are extended to the National Institute on Drug Abuse\/National Institutes of Health (NIDA\/NIH) and the National Institute of Standards and Technology (NIST) for supplying Cannabis sativa marijuana samples analyzed in this study. Thanks are extended to IonSense, Inc. for the analysis of recreational Cannabis flower products and to Dr. Brent Wilson (NIST) for helpful assistance.\n\nAuthor contributions \nRAM conceived of the project, data analysis, project design, and project management and drafted the manuscript; MIC contributed to the experimental work and data analysis and drafted the manuscript; SB contributed to the data processing and data analysis and drafted the manuscript; BG contributed to the experimental work and data analysis. The authors read and approved the final manuscript.\n\nFunding \nThe financial support of the National Institute of Justice (NIJ), Office of Justice programs, U.S. Department of Justice (DOJ) under Grant Nos. 2015-DN-BX-K057, 2017-R2-CX-0020 and 2019-BU-DX-0026 to RAM; the U.S. National Science Foundation (NSF) under Grant No. 1429329 to RAM; the 2020 Northeastern Association of Forensic Scientists (NEAFS) Carol De Forest Research Grant to MIC; the Initiatives for Women Foundation (IFW) Karen R. Hitchcock New Frontiers award to MIC; and the Research Foundation of SUNY are gratefully acknowledged. The opinions, findings, and conclusions or recommendations expressed in this publication are those of the authors and do not necessarily reflect those of the DOJ and\/or the NSF.\n\nAvailability of data and materials \nThe datasets analyzed in the current study are available upon request at the discretion of the corresponding author.\n\nCompeting interests \nThe authors declare that they have no competing interests.\n\r\n\n\nReferences \n\n\n\u2191 National Institute of Justice (April 2019). \"Report to Congress: Needs Assessment of Forensic Laboratories and Medical Examiner\/Coroner Offices\" (PDF). U.S. Department of Justice. pp. 86\u201397. https:\/\/www.ojp.gov\/pdffiles1\/nij\/253626.pdf .   \n \n\n\u2191 Conaway, K.M. (20 December 2018). \"H.R.2 - Agriculture Improvement Act of 2018\". Congress.gov. Library of Congress. https:\/\/www.congress.gov\/bill\/115th-congress\/house-bill\/112 .   \n \n\n\u2191 3.0 3.1 Pourseyed Lazarjani, Masoumeh; Torres, Stephanie; Hooker, Thom; Fowlie, Chris; Young, Owen; Seyfoddin, Ali (1 December 2020). \"Methods for quantification of cannabinoids: a narrative review\" (in en). Journal of Cannabis Research 2 (1): 35. doi:10.1186\/s42238-020-00040-2. ISSN 2522-5782. PMC PMC7819317. PMID 33526084. https:\/\/jcannabisresearch.biomedcentral.com\/articles\/10.1186\/s42238-020-00040-2 .   \n \n\n\u2191 4.0 4.1 Laboratory and Scientific Section, United Nations Office on Drugs and Crime (2009). \"Recommended methods for the identification and analysis of cannabis and cannabis products\" (PDF). ISBN 978-92-1-148242-3. https:\/\/www.unodc.org\/documents\/scientific\/ST-NAR-40-Ebook_1.pdf .   \n \n\n\u2191 5.0 5.1 5.2 Wiebelhaus, Nancy; Hamblin, D\u2019Nisha; Kreitals, Natasha M.; Almirall, Jose R. (1 November 2016). \"Differentiation of marijuana headspace volatiles from other plants and hemp products using capillary microextraction of volatiles (CMV) coupled to gas-chromatography\u2013mass spectrometry (GC\u2013MS)\" (in en). Forensic Chemistry 2: 1\u20138. doi:10.1016\/j.forc.2016.08.004. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170916300285 .   \n \n\n\u2191 6.0 6.1 6.2 Horne, Melissa; Mastrianni, Kaylee R.; Amick, Gray; Hardy, Rachel; Renneker, Elissa; Miller, Kevin W.P. (1 September 2020). \"Fast Discrimination of Marijuana using Automated High\u2010throughput Cannabis Sample Preparation and Analysis by Gas Chromatography\u2013Mass Spectrometry\" (in en). Journal of Forensic Sciences 65 (5): 1709\u20131715. doi:10.1111\/1556-4029.14525. ISSN 0022-1198. https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/1556-4029.14525 .   \n \n\n\u2191 7.0 7.1 7.2 Pacula, Rosalie Liccardo; Jacobson, Mireille; Maksabedian, Ervant J. (1 June 2016). \"In the weeds: a baseline view of cannabis use among legalizing states and their neighbours: In the weeds: a baseline view of cannabis use among legalizing states and their neighbours\" (in en). Addiction 111 (6): 973\u2013980. doi:10.1111\/add.13282. PMC PMC5216038. PMID 26687431. https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/add.13282 .   \n \n\n\u2191 8.0 8.1 8.2 8.3 Fischedick, Justin Thomas; Hazekamp, Arno; Erkelens, Tjalling; Choi, Young Hae; Verpoorte, Rob (1 December 2010). \"Metabolic fingerprinting of Cannabis sativa L., cannabinoids and terpenoids for chemotaxonomic and drug standardization purposes\" (in en). Phytochemistry 71 (17-18): 2058\u20132073. doi:10.1016\/j.phytochem.2010.10.001. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S003194221000381X .   \n \n\n\u2191 Philp, Morgan; Shimmon, Ronald; Tahtouh, Mark; Fu, Shanlin (5 February 2018). \"Color Spot Test As a Presumptive Tool for the Rapid Detection of Synthetic Cathinones\" (in en). Journal of Visualized Experiments (132): 57045. doi:10.3791\/57045. ISSN 1940-087X. PMC PMC5912360. PMID 29443096. https:\/\/www.jove.com\/t\/57045\/color-spot-test-as-a-presumptive-tool-for-the-rapid-detection-of-synthetic-cathinones .   \n \n\n\u2191 10.0 10.1 10.2 Lewis, Kenna; Wagner, Rebecca; Rodriguez\u2010Cruz, Sandra E.; Weaver, Michael J.; Dumke, Jonathan C. (1 January 2021). \"Validation of the 4\u2010aminophenol color test for the differentiation of marijuana\u2010type and hemp\u2010type cannabis\" (in en). Journal of Forensic Sciences 66 (1): 285\u2013294. doi:10.1111\/1556-4029.14562. ISSN 0022-1198. https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/1556-4029.14562 .   \n \n\n\u2191 11.0 11.1 Acosta, Alexander; Li, Li; Weaver, Mike; Capote, Ryan; Perr, Jeannette; Almirall, Jos\u00e9 (1 December 2022). \"Validation of a combined Fast blue BB and 4-Aminophenol colorimetric test for indication of Hemp-type and Marijuana-type cannabis\" (in en). Forensic Chemistry 31: 100448. doi:10.1016\/j.forc.2022.100448. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170922000510 .   \n \n\n\u2191 12.0 12.1 Acosta, Alexander; Almirall, Jos\u00e9 (1 December 2021). \"Differentiation between hemp-type and marijuana-type cannabis using the Fast Blue BB colorimetric test\" (in en). Forensic Chemistry 26: 100376. doi:10.1016\/j.forc.2021.100376. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170921000722 .   \n \n\n\u2191 13.0 13.1 Forrester, D.E. (15 April 1997). \"The Duquenois Color Test for Marijuana: Spectroscopic and Chemical Studies\". Georgetown University. https:\/\/www.proquest.com\/openview\/b2bc339ba2dc1b9f55ab83bb9a112ec9\/1?pq-origsite=gscholar&cbl=18750&diss=y .   \n \n\n\u2191 Gabrielson, R.; Sanders, T. (7 July 2016). \"Busted: Tens of thousands of people every year are sent to jail based on the results of a $2 roadside drug test. Widespread evidence shows that these tests routinely produce false positives. Why are police departments and prosecutors still using them?\". ProPublica. https:\/\/www.propublica.org\/article\/common-roadside-drug-test-routinely-produces-false-positives .   \n \n\n\u2191 Sawler, Jason; Stout, Jake M.; Gardner, Kyle M.; Hudson, Darryl; Vidmar, John; Butler, Laura; Page, Jonathan E.; Myles, Sean (26 August 2015). Tinker, Nicholas A.. ed. \"The Genetic Structure of Marijuana and Hemp\" (in en). PLOS ONE 10 (8): e0133292. doi:10.1371\/journal.pone.0133292. ISSN 1932-6203. PMC PMC4550350. PMID 26308334. https:\/\/dx.plos.org\/10.1371\/journal.pone.0133292 .   \n \n\n\u2191 Roman, Madeline G.; Houston, Rachel (1 November 2020). \"Investigation of chloroplast regions rps16 and clpP for determination of Cannabis sativa crop type and biogeographical origin\" (in en). Legal Medicine 47: 101759. doi:10.1016\/j.legalmed.2020.101759. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1344622320300936 .   \n \n\n\u2191 17.0 17.1 Schwabe, Anna L.; Hansen, Connor J.; Hyslop, Richard M.; McGlaughlin, Mitchell E. (29 September 2021). \"Comparative Genetic Structure of Cannabis sativa Including Federally Produced, Wild Collected, and Cultivated Samples\". Frontiers in Plant Science 12: 675770. doi:10.3389\/fpls.2021.675770. ISSN 1664-462X. PMC PMC8544287. PMID 34707624. https:\/\/www.frontiersin.org\/articles\/10.3389\/fpls.2021.675770\/full .   \n \n\n\u2191 Cody, Robert B.; Laram\u00e9e, James A.; Durst, H. Dupont (1 April 2005). \"Versatile New Ion Source for the Analysis of Materials in Open Air under Ambient Conditions\" (in en). Analytical Chemistry 77 (8): 2297\u20132302. doi:10.1021\/ac050162j. ISSN 0003-2700. https:\/\/pubs.acs.org\/doi\/10.1021\/ac050162j .   \n \n\n\u2191 19.0 19.1 Chambers, Megan I.; Musah, Rabi A. (1 March 2022). \"DART-HRMS as a triage approach for the rapid analysis of cannabinoid-infused edible matrices, personal-care products and Cannabis sativa hemp plant material\" (in en). Forensic Chemistry 27: 100382. doi:10.1016\/j.forc.2021.100382. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170921000783 .   \n \n\n\u2191 Rodriguez-Cruz, Sandra E. (15 January 2006). \"Rapid analysis of controlled substances using desorption electrospray ionization mass spectrometry\" (in en). Rapid Communications in Mass Spectrometry 20 (1): 53\u201360. doi:10.1002\/rcm.2267. ISSN 0951-4198. https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/rcm.2267 .   \n \n\n\u2191 21.0 21.1 Chambers, Megan I.; Musah, Rabi A. (1 May 2023). \"DART-HRMS triage approach part 2 \u2013 Application to the detection of cannabinoids and terpenes in recreational Cannabis products\" (in en). Forensic Chemistry 33: 100469. doi:10.1016\/j.forc.2023.100469. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S246817092300005X .   \n \n\n\u2191 Beyramysoltan, Samira; Abdul-Rahman, Nana-Hawwa; Musah, Rabi A. (1 November 2019). \"Call it a \u201cnightshade\u201d\u2014A hierarchical classification approach to identification of hallucinogenic Solanaceae spp. using DART-HRMS-derived chemical signatures\" (in en). Talanta 204: 739\u2013746. doi:10.1016\/j.talanta.2019.06.010. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0039914019306290 .   \n \n\n\u2191 Appley, Meghan Grace; Beyramysoltan, Samira; Musah, Rabi Ann (24 September 2019). \"Random Forest Processing of Direct Analysis in Real-Time Mass Spectrometric Data Enables Species Identification of Psychoactive Plants from Their Headspace Chemical Signatures\" (in en). ACS Omega 4 (13): 15636\u201315644. doi:10.1021\/acsomega.9b02145. ISSN 2470-1343. PMC PMC6761758. PMID 31572865. https:\/\/pubs.acs.org\/doi\/10.1021\/acsomega.9b02145 .   \n \n\n\u2191 Dong, Wen; Liang, Jian; Barnett, Isabella; Kline, Paul C.; Altman, Elliot; Zhang, Mengliang (1 December 2019). \"The classification of Cannabis hemp cultivars by thermal desorption direct analysis in real time mass spectrometry (TD-DART-MS) with chemometrics\" (in en). Analytical and Bioanalytical Chemistry 411 (30): 8133\u20138142. doi:10.1007\/s00216-019-02200-7. ISSN 1618-2642. http:\/\/link.springer.com\/10.1007\/s00216-019-02200-7 .   \n \n\n\u2191 Pieslak, J.R. (2021). \"Analytical techniques for the differentiation of hemp and marijuana\". OpenBU. Boston University Libraries. https:\/\/hdl.handle.net\/2144\/43518 .   \n \n\n\u2191 Jolliffe, Ian T.; Cadima, Jorge (13 April 2016). \"Principal component analysis: a review and recent developments\" (in en). Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 374 (2065): 20150202. doi:10.1098\/rsta.2015.0202. ISSN 1364-503X. PMC PMC4792409. PMID 26953178. https:\/\/royalsocietypublishing.org\/doi\/10.1098\/rsta.2015.0202 .   \n \n\n\u2191 Sammut, Claude; Webb, Geoffrey I., eds. (2010) (in en). Encyclopedia of Machine Learning. Boston, MA: Springer US. doi:10.1007\/978-0-387-30164-8. ISBN 978-0-387-30768-8. http:\/\/link.springer.com\/10.1007\/978-0-387-30164-8 .   \n \n\n\u2191 Lloyd, S. (1 March 1982). \"Least squares quantization in PCM\" (in en). IEEE Transactions on Information Theory 28 (2): 129\u2013137. doi:10.1109\/TIT.1982.1056489. ISSN 0018-9448. http:\/\/ieeexplore.ieee.org\/document\/1056489\/ .   \n \n\n\u2191 Liaw, A.; Wiener, M. (2002). \"Classification and Regression by randomForest\" (PDF). R News 2 (3): 18\u201322. ISSN 1609-3631. https:\/\/cogns.northwestern.edu\/cbmg\/LiawAndWiener2002.pdf .   \n \n\n\u2191 Breiman, Leo (2001). \"Random Forests\". Machine Learning 45 (1): 5\u201332. doi:10.1023\/A:1010933404324. http:\/\/link.springer.com\/10.1023\/A:1010933404324 .   \n \n\n\u2191 Beyramysoltan, Samira; Ventura, M\u00f3nica I.; Rosati, Jennifer Y.; Giffen-Lemieux, Justine E.; Musah, Rabi A. (7 April 2020). \"Identification of the Species Constituents of Maggot Populations Feeding on Decomposing Remains\u2014Facilitation of the Determination of Post Mortem Interval and Time Since Tissue Infestation through Application of Machine Learning and Direct Analysis in Real Time-Mass Spectrometry\" (in en). Analytical Chemistry 92 (7): 5439\u20135446. doi:10.1021\/acs.analchem.0c00199. ISSN 0003-2700. https:\/\/pubs.acs.org\/doi\/10.1021\/acs.analchem.0c00199 .   \n \n\n\u2191 Vergara, Daniela; Bidwell, L. Cinnamon; Gaudino, Reggie; Torres, Anthony; Du, Gary; Ruthenburg, Travis C.; deCesare, Kymron; Land, Donald P. et al. (19 April 2017). \"Compromised External Validity: Federally Produced Cannabis Does Not Reflect Legal Markets\" (in en). Scientific Reports 7 (1): 46528. doi:10.1038\/srep46528. ISSN 2045-2322. PMC PMC5395929. PMID 28422145. https:\/\/www.nature.com\/articles\/srep46528 .   \n \n\n\u2191 33.0 33.1 Zeki\u010d, Jure; Kri\u017eman, Mitja (11 December 2020). \"Development of Gas-Chromatographic Method for Simultaneous Determination of Cannabinoids and Terpenes in Hemp\" (in en). Molecules 25 (24): 5872. doi:10.3390\/molecules25245872. ISSN 1420-3049. PMC PMC7763075. PMID 33322595. https:\/\/www.mdpi.com\/1420-3049\/25\/24\/5872 .   \n \n\n\u2191 Dussy, Franz E.; Hamberg, Cornelia; Luginb\u00fchl, Marco; Schwerzmann, Thomas; Briellmann, Thomas A. (1 April 2005). \"Isolation of \u03949-THCA-A from hemp and analytical aspects concerning the determination of \u03949-THC in cannabis products\" (in en). Forensic Science International 149 (1): 3\u201310. doi:10.1016\/j.forsciint.2004.05.015. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0379073804003408 .   \n \n\n\u2191 Fischedick, Justin; Van Der Kooy, Frank; Verpoorte, Robert (2010). \"Cannabinoid Receptor 1 Binding Activity and Quantitative Analysis of Cannabis sativa L. Smoke and Vapor\" (in en). Chemical and Pharmaceutical Bulletin 58 (2): 201\u2013207. doi:10.1248\/cpb.58.201. ISSN 0009-2363. http:\/\/www.jstage.jst.go.jp\/article\/cpb\/58\/2\/58_2_201\/_article .   \n \n\n\u2191 36.0 36.1 Hazekamp, Arno; Simons, Ruud; Peltenburg\u2010Looman, Anja; Sengers, Melvin; van Zweden, Rianne; Verpoorte, Robert (1 January 2004). \"Preparative Isolation of Cannabinoids from Cannabis sativa by Centrifugal Partition Chromatography\" (in en). Journal of Liquid Chromatography & Related Technologies 27 (15): 2421\u20132439. doi:10.1081\/JLC-200028170. ISSN 1082-6076. https:\/\/www.tandfonline.com\/doi\/full\/10.1081\/JLC-200028170 .   \n \n\n\u2191 Hazekamp, A.; Fischedick, J. T. (1 July 2012). \"Cannabis - from cultivar to chemovar: Towards a better definition of Cannabis potency\" (in en). Drug Testing and Analysis 4 (7-8): 660\u2013667. doi:10.1002\/dta.407. https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/dta.407 .   \n \n\n\u2191 Hazekamp, Arno; Peltenburg, Anja; Verpoorte, Rob; Giroud, Christian (1 September 2005). \"Chromatographic and Spectroscopic Data of Cannabinoids from Cannabis sativa L.\" (in en). Journal of Liquid Chromatography & Related Technologies 28 (15): 2361\u20132382. doi:10.1080\/10826070500187558. ISSN 1082-6076. https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/10826070500187558 .   \n \n\n\u2191 Namdar, Dvory; Mazuz, Moran; Ion, Aurel; Koltai, Hinanit (1 March 2018). \"Variation in the compositions of cannabinoid and terpenoids in Cannabis sativa derived from inflorescence position along the stem and extraction methods\" (in en). Industrial Crops and Products 113: 376\u2013382. doi:10.1016\/j.indcrop.2018.01.060. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S092666901830061X .   \n \n\n\u2191 Namdar, Dvory; Charuvi, Dana; Ajjampura, Vinayka; Mazuz, Moran; Ion, Aurel; Kamara, Itzhak; Koltai, Hinanit (1 June 2019). \"LED lighting affects the composition and biological activity of Cannabis sativa secondary metabolites\" (in en). Industrial Crops and Products 132: 177\u2013185. doi:10.1016\/j.indcrop.2019.02.016. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0926669019301086 .   \n \n\n\u2191 Omar, Jone; Olivares, Maitane; Alzaga, Mikel; Etxebarria, Nestor (1 April 2013). \"Optimisation and characterisation of marihuana extracts obtained by supercritical fluid extraction and focused ultrasound extraction and retention time locking GC-MS: Gas Chromatography\" (in en). Journal of Separation Science 36 (8): 1397\u20131404. doi:10.1002\/jssc.201201103. https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/jssc.201201103 .   \n \n\n\u2191 Knight, Glenys; Hansen, Sean; Connor, Mark; Poulsen, Helen; McGovern, Catherine; Stacey, Janet (1 October 2010). \"The results of an experimental indoor hydroponic Cannabis growing study, using the \u2018Screen of Green\u2019 (ScrOG) method\u2014Yield, tetrahydrocannabinol (THC) and DNA analysis\" (in en). Forensic Science International 202 (1-3): 36\u201344. doi:10.1016\/j.forsciint.2010.04.022. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0379073810001969 .   \n \n\n\u2191 Leghissa, Allegra; Smuts, Jonathan; Qiu, Changling; Hildenbrand, Zacariah L.; Schug, Kevin A. (1 January 2018). \"Detection of cannabinoids and cannabinoid metabolites using gas chromatography with vacuum ultraviolet spectroscopy\" (in en). Separation Science Plus 1 (1): 37\u201342. doi:10.1002\/sscp.201700005. https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/sscp.201700005 .   \n \n\n\u2191 Gr\u00f6ger, Th.; Sch\u00e4ffer, M.; P\u00fctz, M.; Ahrens, B.; Drew, K.; Eschner, M.; Zimmermann, R. (1 July 2008). \"Application of two-dimensional gas chromatography combined with pixel-based chemometric processing for the chemical profiling of illicit drug samples\" (in en). Journal of Chromatography A 1200 (1): 8\u201316. doi:10.1016\/j.chroma.2008.05.028. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0021967308008297 .   \n \n\n\u2191 Omar, Jone; Olivares, Maitane; Amigo, Jos\u00e9 Manuel; Etxebarria, Nestor (1 April 2014). \"Resolution of co-eluting compounds of Cannabis Sativa in comprehensive two-dimensional gas chromatography\/mass spectrometry detection with Multivariate Curve Resolution-Alternating Least Squares\" (in en). Talanta 121: 273\u2013280. doi:10.1016\/j.talanta.2013.12.044. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0039914013010370 .   \n \n\n\u2191 Sanchez, Lee; Filter, Conor; Baltensperger, David; Kurouski, Dmitry (2020). \"Confirmatory non-invasive and non-destructive differentiation between hemp and cannabis using a hand-held Raman spectrometer\" (in en). RSC Advances 10 (6): 3212\u20133216. doi:10.1039\/C9RA08225E. ISSN 2046-2069. PMC PMC9048763. PMID 35497720. http:\/\/xlink.rsc.org\/?DOI=C9RA08225E .   \n \n\n\u2191 Chen, Zewei; Harrington, Peter de Boves (19 November 2019). \"Pipeline for High-Throughput Modeling of Marijuana and Hemp Extracts\" (in en). Analytical Chemistry 91 (22): 14489\u201314497. doi:10.1021\/acs.analchem.9b03290. ISSN 0003-2700. https:\/\/pubs.acs.org\/doi\/10.1021\/acs.analchem.9b03290 .   \n \n\n\u2191 dos Santos, Nayara A.; Souza, Lindamara M.; Domingos, Eloilson; Fran\u00e7a, Hildegardo S.; Lacerda, Valdemar; Beatriz, Adilson; Vaz, Boniek G.; Rodrigues, Rayza R.T. et al. (1 August 2016). \"Evaluating the selectivity of colorimetric test (Fast Blue BB salt) for the cannabinoids identification in marijuana street samples by UV\u2013Vis, TLC, ESI(+)FT-ICR MS and ESI(+)MS\/MS\" (in en). Forensic Chemistry 1: 13\u201321. doi:10.1016\/j.forc.2016.07.001. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170916300297 .   \n \n\n\u2191 Fran\u00e7a, Hildegardo S.; Acosta, Alexander; Jamal, Adeel; Romao, Wanderson; Mulloor, Jerome; Almirall, Jose R. (1 March 2020). \"Experimental and ab initio investigation of the products of reaction from \u03949-tetrahydrocannabinol (\u03949-THC) and the fast blue BB spot reagent in presumptive drug tests for cannabinoids\" (in en). Forensic Chemistry 17: 100212. doi:10.1016\/j.forc.2019.100212. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170919301092 .   \n \n\n\u2191 Jacobs, Alexander D.; Steiner, Robert R. (1 June 2014). \"Detection of the Duquenois\u2013Levine chromophore in a marijuana sample\" (in en). Forensic Science International 239: 1\u20135. doi:10.1016\/j.forsciint.2014.02.031. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0379073814000929 .   \n \n\n\u2191 Watanabe, Kazuhito; Honda, Go; Miyagi, Takeaki; Kanai, Masataka; Usami, Noriyuki; Yamaori, Satoshi; Iwamuro, Yoshiaki; Chinaka, Satoshi et al. (1 January 2017). \"The Duquenois reaction revisited: mass spectrometric estimation of chromophore structures derived from major phytocannabinoids\" (in en). Forensic Toxicology 35 (1): 185\u2013189. doi:10.1007\/s11419-016-0337-6. ISSN 1860-8965. http:\/\/link.springer.com\/10.1007\/s11419-016-0337-6 .   \n \n\n\u2191 52.0 52.1 Pingree, C. (1 November 2022). \"H.R.6645 - Hemp Advancement Act of 2022\". Congress.gov. Library of Congress. https:\/\/www.congress.gov\/bill\/117th-congress\/house-bill\/6645 .   \n \n\n\nNotes \nThis presentation is faithful to the original, with only a few minor changes to presentation. Some grammar and punctuation was cleaned up to improve readability. In some cases important information was missing from the references, and that information was added. The original lists references in alphabetical order; they are listed by order of appearance for this version, by design.\n\n\n\n\n\nSource: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\">https:\/\/www.cannaqa.wiki\/index.php?title=Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa<\/a>\nCategories: CannaQAwiki journal articles (added in 2023)CannaQAwiki journal articles (all)CannaQAwiki journal articles on cannabis researchCannaQAwiki journal articles on cannabis testingNavigation menuPage actionsJournalDiscussionView sourceHistoryPage actionsJournalDiscussionMoreToolsIn other languagesPersonal toolsLog inNavigationMain pageList of articlesRandom pageRecent changesHelpSearch\u00a0 ToolsWhat links hereRelated changesSpecial pagesPrintable versionPermanent linkPage information This page was last edited on 30 June 2023, at 00:28.Content is available under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License unless otherwise noted.Privacy policyAbout CannaQAWikiDisclaimers\n","e023a20c9343c9211ed31c45f339ab22_html":"<body class=\"mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-170 ns-subject page-Journal_Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa rootpage-Journal_Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa skin-monobook action-view skin--responsive\"><div id=\"rdp-ebb-globalWrapper\"><div id=\"rdp-ebb-column-content\"><div id=\"rdp-ebb-content\" class=\"mw-body\" role=\"main\"><a id=\"rdp-ebb-top\"><\/a>\n<h1 id=\"rdp-ebb-firstHeading\" class=\"firstHeading\" lang=\"en\">Journal:Combined ambient ionization mass spectrometric and chemometric approach for the differentiation of hemp and marijuana varieties of <i>Cannabis sativa<\/i><\/h1><div id=\"rdp-ebb-bodyContent\" class=\"mw-body-content\"><!-- start content --><div id=\"rdp-ebb-mw-content-text\" lang=\"en\" dir=\"ltr\" class=\"mw-content-ltr\"><div class=\"mw-parser-output\">\n\n\n<h2><span class=\"mw-headline\" id=\"Abstract\">Abstract<\/span><\/h2>\n<p><b>Background<\/b>: <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Hemp\" title=\"Hemp\" class=\"wiki-link\" data-key=\"c23e30b6cf1df54f1dc338492c9f9da2\">Hemp<\/a> and marijuana are the two major varieties of <i><a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabis_sativa\" title=\"Cannabis sativa\" class=\"wiki-link\" data-key=\"e003358742012354d1ff6002bc5781de\">Cannabis sativa<\/a><\/i>. While both contain <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Tetrahydrocannabinol\" title=\"Tetrahydrocannabinol\" class=\"wiki-link\" data-key=\"15f3b3e338baeb54c04c715818759ec9\">\u0394<sup>9<\/sup>-tetrahydrocannabinol<\/a> (THC), the primary <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Psychoactive_drug\" title=\"Psychoactive drug\" class=\"wiki-link\" data-key=\"abba12c7100bf1c8457208da20b4234b\">psychoactive<\/a> component of <i>C. sativa<\/i>, they differ in the amount of THC that they contain. Presently, U.S. federal laws stipulate that <i>C. sativa<\/i> containing greater than 0.3% THC is classified as marijuana, while plant material that contains less than or equal to 0.3% THC is hemp. Current methods to determine THC content are <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Chromatography\" title=\"Chromatography\" class=\"wiki-link\" data-key=\"1b40e146652470be00cebaf949c68b24\">chromatography<\/a>-based, which requires extensive <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Sample_(material)\" title=\"Sample (material)\" class=\"wiki-link\" data-key=\"2d7596b7c3616db8f56af698be478d3d\">sample<\/a> preparation to render the materials into <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabis_concentrate\" title=\"Cannabis concentrate\" class=\"wiki-link\" data-key=\"4820fde4ed6a3627d7a670471f273d33\">extracts<\/a> suitable for sample injection, for complete separation and differentiation of THC from all other analytes present. This can create problems for <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Forensic_science\" title=\"Forensic science\" class=\"wiki-link\" data-key=\"cbfab6e9db85c6efc9cbc7caa8ac6c65\">forensic<\/a> <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Laboratory\" title=\"Laboratory\" class=\"wiki-link\" data-key=\"ed14e6a67b4b14ad2c190c28455725f6\">laboratories<\/a> due to the increased workload associated with the need to analyze and quantify THC in all <i>C. sativa<\/i> materials.\n<\/p><p><b>Method<\/b>: The work presented herein combines <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Direct_analysis_in_real_time\" title=\"Direct analysis in real time\" class=\"wiki-link\" data-key=\"64cafa769866fd38d0027881b09bb791\">direct analysis in real time<\/a> <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Resolution_(mass_spectrometry)\" title=\"Resolution (mass spectrometry)\" class=\"wiki-link\" data-key=\"b8840a5aade28e7366129154985dcf3c\">high-resolution<\/a> <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Mass_spectrometry\" title=\"Mass spectrometry\" class=\"wiki-link\" data-key=\"18314b70982e52d5a81db4a757d6461f\">mass spectrometry<\/a> (DART-HRMS) and advanced <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Chemometrics\" title=\"Chemometrics\" class=\"wiki-link\" data-key=\"b2d14e445af0e6bd6ab5534a017591cc\">chemometrics<\/a> to differentiate hemp and marijuana plant materials. Samples were obtained from several sources (e.g., commercial vendors, DEA-registered suppliers, and the recreational <i><a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabis\" title=\"Cannabis\" class=\"wiki-link\" data-key=\"a70b76268930d795518ff1f98d7e500d\">Cannabis<\/a><\/i> market). DART-HRMS enabled the interrogation of plant materials with no sample pretreatment. Advanced multivariate data analysis approaches, including <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Random_forest\" title=\"Random forest\" class=\"wiki-link\" data-key=\"3a557b0006d996735586ad28712f4771\">random forest<\/a> and <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Principal_component_analysis\" title=\"Principal component analysis\" class=\"wiki-link\" data-key=\"f19d0fd1bc76a5685819562993c89812\">principal component analysis<\/a> (PCA), were used to optimally differentiate these two varieties with a high level of accuracy.\n<\/p><p><b>Results<\/b>: When PCA was applied to the hemp and marijuana data, distinct clustering that enabled their differentiation was observed. Furthermore, within the marijuana class, subclusters between recreational and DEA-supplied marijuana samples were observed. A separate investigation using the silhouette width index to determine the optimal number of clusters for the marijuana and hemp data revealed this number to be two. Internal validation of the model using random forest demonstrated an accuracy of 98%, while external validation samples were classified with 100% accuracy.\n<\/p><p><b>Discussion<\/b>: The results show that the developed approach would significantly aid in the analysis and differentiation of <i>C. sativa<\/i> plant materials prior to launching painstaking confirmatory testing using chromatography. However, to maintain and\/or enhance the accuracy of the prediction model and keep it from becoming outdated, it will be necessary to continue to expand it to include mass spectral data representative of emerging hemp and marijuana <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabis_strains\" title=\"Cannabis strains\" class=\"wiki-link\" data-key=\"31157c083080914bf8b5216f89d93bf8\">strains<\/a>\/<a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cultivar\" title=\"Cultivar\" class=\"wiki-link\" data-key=\"c83815cf6fd1ec12e6fcf4c650566a6f\">cultivars<\/a>.\n<\/p><p><b>Keywords<\/b>: <i>Cannabis sativa<\/i>, ambient ionization mass spectrometry, direct analysis in real time\u2014high-resolution mass spectrometry, multivariate data analysis, random forest, principal component analysis\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Background\">Background<\/span><\/h2>\n<p>Among the greatest challenges to emerge for U.S. <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Forensic_science\" title=\"Forensic science\" class=\"wiki-link\" data-key=\"cbfab6e9db85c6efc9cbc7caa8ac6c65\">forensic<\/a> <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Laboratory\" title=\"Laboratory\" class=\"wiki-link\" data-key=\"ed14e6a67b4b14ad2c190c28455725f6\">laboratories<\/a> in recent years are those attributed to the increased legalization and decriminalization of marijuana at the state level, in addition to the permitted production of <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Hemp\" title=\"Hemp\" class=\"wiki-link\" data-key=\"c23e30b6cf1df54f1dc338492c9f9da2\">hemp<\/a>. The 2019 National Institute of Justice (NIJ) <i>Report to Congress: Needs Assessment of Forensic Laboratories and Medical Examiner\/Coroner Offices<\/i> identified this area as requiring focused attention towards improving criminal justice practices in the USA.<sup id=\"rdp-ebb-cite_ref-1\" class=\"reference\"><a href=\"#cite_note-1\">[1]<\/a><\/sup> The challenge that hemp and marijuana present is as follows: both are major varieties of the same species <i><a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabis_sativa\" title=\"Cannabis sativa\" class=\"wiki-link\" data-key=\"e003358742012354d1ff6002bc5781de\">Cannabis sativa<\/a><\/i>, often referred to as <i><a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabis\" title=\"Cannabis\" class=\"wiki-link\" data-key=\"a70b76268930d795518ff1f98d7e500d\">Cannabis<\/a><\/i>. While they each contain <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Tetrahydrocannabinol\" title=\"Tetrahydrocannabinol\" class=\"wiki-link\" data-key=\"15f3b3e338baeb54c04c715818759ec9\">\u0394<sup>9<\/sup>-tetrahydrocannabinol<\/a> (THC), which is the primary psychoactive component of <i>C. sativa<\/i>, marijuana and hemp differ in the amount of this molecule that is present. In 2018, the U.S. federal guidelines stipulated that <i>C. sativa<\/i> which contains greater than 0.3% THC is a scheduled controlled substance (i.e., marijuana), while plant material that contains less than or equal to 0.3% is a legal agricultural commodity (i.e., hemp).<sup id=\"rdp-ebb-cite_ref-2\" class=\"reference\"><a href=\"#cite_note-2\">[2]<\/a><\/sup> This definition has imposed severe challenges on crime labs. Among them is the dramatic increase in workload that results from the need to analyze and quantify the THC content of all <i>C. sativa<\/i> <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Sample_(material)\" title=\"Sample (material)\" class=\"wiki-link\" data-key=\"2d7596b7c3616db8f56af698be478d3d\">samples<\/a> so that seized material can be appropriately designated. This is a time-consuming and resource-intensive enterprise that to greater and greater extents is consuming even larger forensic lab resources. Furthermore, defining the error cutoff for the 0.3% designation presents a challenge for the analysis of samples whose THC level is at the threshold.\n<\/p><p>Traditionally, hemp and marijuana plant materials are differentiated by determining the THC content through <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Chromatography\" title=\"Chromatography\" class=\"wiki-link\" data-key=\"1b40e146652470be00cebaf949c68b24\">chromatography<\/a>-based approaches such as <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Gas_chromatography\" title=\"Gas chromatography\" class=\"wiki-link\" data-key=\"5094f035cb8bead5003deb9181e398b8\">gas chromatography<\/a>-<a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Chromatography_detector\" title=\"Chromatography detector\" class=\"wiki-link\" data-key=\"7787a159a1c7c0a3b2cc28bbc3189587\">flame ionization detection<\/a> (GC-FID) and (GC\u2013MS)<sup id=\"rdp-ebb-cite_ref-:0_3-0\" class=\"reference\"><a href=\"#cite_note-:0-3\">[3]<\/a><\/sup>, in addition to <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=High-performance_liquid_chromatography\" title=\"High-performance liquid chromatography\" class=\"wiki-link\" data-key=\"4c5d83ffa9785383c3eb0be7ea78dd2b\">high-performance liquid chromatography<\/a> (HPLC) coupled to <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Ultraviolet%E2%80%93visible_spectroscopy\" title=\"Ultraviolet\u2013visible spectroscopy\" class=\"wiki-link\" data-key=\"544f7ef37bc31936b5584244f6f6e635\">ultraviolet<\/a> (UV) detection.<sup id=\"rdp-ebb-cite_ref-:1_4-0\" class=\"reference\"><a href=\"#cite_note-:1-4\">[4]<\/a><\/sup> However, to accurately determine the THC content with these approaches, THC must be separated from all other components in the material (i.e., <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabinoid\" title=\"Cannabinoid\" class=\"wiki-link\" data-key=\"c224c3041748677fcdce5b5209900b7b\">cannabinoids<\/a>, <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Terpene\" title=\"Terpene\" class=\"wiki-link\" data-key=\"6bcbb95d582c08073101e9c4cfc91931\">terpenes<\/a>, etc.) prior to quantification. One way to achieve this is to extend run times to allow for baseline separation between cannabinoids and other analytes present. Another option is to introduce a chemical derivatization step into the sample preparation protocol (which can be time-consuming), to differentiate between cannabinoids and their corresponding cannabinoid acids (e.g., THC and <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Tetrahydrocannabinolic_acid\" title=\"Tetrahydrocannabinolic acid\" class=\"wiki-link\" data-key=\"a047dcf0f7a85c57e1d2d308b56e5e4a\">tetrahydrocannabinolic acid<\/a> [THCA]). Although many investigations have been successful at differentiating between hemp and marijuana varieties or <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabis_strains\" title=\"Cannabis strains\" class=\"wiki-link\" data-key=\"31157c083080914bf8b5216f89d93bf8\">strains<\/a><sup id=\"rdp-ebb-cite_ref-:2_5-0\" class=\"reference\"><a href=\"#cite_note-:2-5\">[5]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:3_6-0\" class=\"reference\"><a href=\"#cite_note-:3-6\">[6]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:4_7-0\" class=\"reference\"><a href=\"#cite_note-:4-7\">[7]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:5_8-0\" class=\"reference\"><a href=\"#cite_note-:5-8\">[8]<\/a><\/sup>, the methods are reliant upon chromatography and are therefore susceptible to the aforementioned delineated challenges that can arise using this technique (i.e., lengthy run times, column contamination, etc.). Research towards developing, optimizing, and validating methods suitable for field testing of <i>Cannabis<\/i> materials has also been investigated.\n<\/p><p>Colorimetric tests represent a large percentage of these methods, which yield a presumptive result (by producing a color change)<sup id=\"rdp-ebb-cite_ref-9\" class=\"reference\"><a href=\"#cite_note-9\">[9]<\/a><\/sup> when <i>Cannabis<\/i>-related substances are present, without the need for additional instrumentation (i.e., it is visible to the naked eye). Some examples include the 4-aminophenol test<sup id=\"rdp-ebb-cite_ref-:6_10-0\" class=\"reference\"><a href=\"#cite_note-:6-10\">[10]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:7_11-0\" class=\"reference\"><a href=\"#cite_note-:7-11\">[11]<\/a><\/sup>, Fast Blue BB test<sup id=\"rdp-ebb-cite_ref-:7_11-1\" class=\"reference\"><a href=\"#cite_note-:7-11\">[11]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:8_12-0\" class=\"reference\"><a href=\"#cite_note-:8-12\">[12]<\/a><\/sup>, and Duquenois-Levine test.<sup id=\"rdp-ebb-cite_ref-:9_13-0\" class=\"reference\"><a href=\"#cite_note-:9-13\">[13]<\/a><\/sup> Similar to chromatography-based methods, these tests all rely upon the detection of THC specifically, which can complicate analyses because both marijuana and hemp contain this compound. Thus, while the distinction between marijuana and hemp has been defined based on THC levels, this is accompanied by several analytical challenges (i.e., baseline separation of molecules by chromatography-based methods, lengthy sample preparation protocols, and presumptive tests that can yield false positives<sup id=\"rdp-ebb-cite_ref-14\" class=\"reference\"><a href=\"#cite_note-14\">[14]<\/a><\/sup>, etc.).\n<\/p><p>An alternative less arbitrary approach is to base the distinction between them on the genome-defined differences in their metabolome signatures (i.e., small-molecule profiles). Studies utilizing the genetic profiles of <i>Cannabis<\/i>, such as genotyping-by-sequencing (GBS) and single-nucleotide polymorphisms (SNPs), have shown that, although they represent the same species, hemp and marijuana differ at the genome-wide level.<sup id=\"rdp-ebb-cite_ref-15\" class=\"reference\"><a href=\"#cite_note-15\">[15]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-16\" class=\"reference\"><a href=\"#cite_note-16\">[16]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:10_17-0\" class=\"reference\"><a href=\"#cite_note-:10-17\">[17]<\/a><\/sup> However, in addition to the fact that many crime laboratories are not positioned to integrate these types of analyses into current <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Workflow\" title=\"Workflow\" class=\"wiki-link\" data-key=\"c63a2fc4c276562dbdaf957f640d69cb\">workflows<\/a>, one of the bottlenecks to the routine use of the genome-defined small-molecule profiles for species attribution is the challenge of accessing this information quickly and reliably. One way to rapidly reveal this information, and subsequently distinguish between hemp and marijuana, is to combine an <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Ambient_ionization\" title=\"Ambient ionization\" class=\"wiki-link\" data-key=\"1755881a66c2f527c3cc718a56eba633\">ambient ionization<\/a> mass spectrometric technique\u2014e.g., <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Direct_analysis_in_real_time\" title=\"Direct analysis in real time\" class=\"wiki-link\" data-key=\"64cafa769866fd38d0027881b09bb791\">direct analysis in real time<\/a> <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Resolution_(mass_spectrometry)\" title=\"Resolution (mass spectrometry)\" class=\"wiki-link\" data-key=\"b8840a5aade28e7366129154985dcf3c\">high-resolution<\/a> <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Mass_spectrometry\" title=\"Mass spectrometry\" class=\"wiki-link\" data-key=\"18314b70982e52d5a81db4a757d6461f\">mass spectrometry<\/a> (DART-HRMS)<sup id=\"rdp-ebb-cite_ref-18\" class=\"reference\"><a href=\"#cite_note-18\">[18]<\/a><\/sup>\u2014with advanced statistical analysis. Ambient ionization methods (e.g., DART-HRMS, desorption electrospray ionization [DESI-MS]) have proven successful at screening for cannabinoids in <i>Cannabis<\/i> plant materials<sup id=\"rdp-ebb-cite_ref-:11_19-0\" class=\"reference\"><a href=\"#cite_note-:11-19\">[19]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-20\" class=\"reference\"><a href=\"#cite_note-20\">[20]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:12_21-0\" class=\"reference\"><a href=\"#cite_note-:12-21\">[21]<\/a><\/sup> and <i>Cannabis<\/i>-derived products (e.g., edibles, personal-care products, vape products, <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabis_concentrate\" title=\"Cannabis concentrate\" class=\"wiki-link\" data-key=\"4820fde4ed6a3627d7a670471f273d33\">concentrates<\/a>).<sup id=\"rdp-ebb-cite_ref-:11_19-1\" class=\"reference\"><a href=\"#cite_note-:11-19\">[19]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:12_21-1\" class=\"reference\"><a href=\"#cite_note-:12-21\">[21]<\/a><\/sup> The unique capabilities of DART-HRMS are well-suited for the analysis of complex plant materials; the results are characterized by having high chemical information content, and little to no sample preparation prior to interrogating the materials is required. When applied to DART-HRMS-derived spectra, statistical data processing has enabled the successful differentiation of psychoactive plant species<sup id=\"rdp-ebb-cite_ref-22\" class=\"reference\"><a href=\"#cite_note-22\">[22]<\/a><\/sup> and their headspace chemical signatures.<sup id=\"rdp-ebb-cite_ref-23\" class=\"reference\"><a href=\"#cite_note-23\">[23]<\/a><\/sup> A modified version of DART-MS analysis introduced <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Thermal_desorption_spectroscopy\" title=\"Thermal desorption spectroscopy\" class=\"wiki-link\" data-key=\"3d2b1b412e5a41ec05488f4ed9692c27\">thermal desorption<\/a> (TD) into the methodology (TD-DART-MS). One study utilized TD-DART-MS data to differentiate four hemp cultivars using PCA and partial least squares discriminant analysis (PLS-DA).<sup id=\"rdp-ebb-cite_ref-24\" class=\"reference\"><a href=\"#cite_note-24\">[24]<\/a><\/sup> Another found that the application of statistical analysis to DART-MS data derived from methanolic extracts of hemp and marijuana samples revealed the potential for utilizing this method for optimally differentiating hemp and marijuana varieties.<sup id=\"rdp-ebb-cite_ref-25\" class=\"reference\"><a href=\"#cite_note-25\">[25]<\/a><\/sup>\n<\/p><p>The study presented here, which is summarized in the scheme presented in Fig. 1, utilized DART-HRMS, for the first time, to investigate the complex genome-defined chemical fingerprints of hemp and marijuana (with no sample pretreatment) for the purpose of distinguishing between these two <i>C. sativa<\/i> varieties using multivariate statistical approaches. Advanced chemometrics was applied to the DART-HRMS data derived from commercial hemp, recreational marijuana, and marijuana samples from Drug Enforcement Administration (DEA)-registered suppliers to develop a robust model by which they (i.e., hemp and marijuana) could be readily differentiated. The success rate of the developed model\u2019s ability to predict external validation samples was 100%, indicating a high level of certainty. Importantly, the developed method circumvents the need to separate and differentiate cannabinoids by chromatography techniques (i.e., the traditional forensic approach for determining the THC concentration in a sample and which is used for differentiating between hemp and marijuana), in addition to bypassing all sample pretreatment steps.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=File:Fig1_Chambers_JofCannRes23_5.png\" class=\"image wiki-link\" data-key=\"96af5885507e44c5f0cd027575739a79\"><img alt=\"Fig1 Chambers JofCannRes23 5.png\" src=\"https:\/\/www.cannaqa.wiki\/images\/5\/53\/Fig1_Chambers_JofCannRes23_5.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Fig. 1.<\/b> Workflow for discrimination of hemp and marijuana samples.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h2><span class=\"mw-headline\" id=\"Materials_and_methods\">Materials and methods<\/span><\/h2>\n<h3><span class=\"mw-headline\" id=\"Cannabis_sativa_plant_materials\"><i>Cannabis sativa<\/i> plant materials<\/span><\/h3>\n<p>Twenty-nine <i>C. sativa<\/i> flower samples of the hemp variety were purchased from three online vendors: (1) CBD Hemp Direct (Las Vegas, Nevada, USA), (2) Berkshire CBD (Brattleboro, Vermont, USA), and (3) Plain Jane (Berkeley, California, USA). These samples were used to build the model (i.e., training set). An additional 12 samples of hemp plant material were purchased from Plain Jane (Medford, Oregon, USA) at a later date to test the model (i.e., they were used for external validation). Additional information (e.g., cultivar\/strain, vendor, batch number) for these hemp materials is provided (see Additional file 1).\n<\/p><p><i>C. sativa<\/i> plant material of the marijuana variety was obtained from two DEA-registered sources. The National Institute on Drug Abuse (NIDA) (Research Triangle Park (RTP), North Carolina, USA) Drug Supply Program, which is part of the National Institutes of Health (NIH), provided the following four samples (i.e., cultivars) with varying levels of THC and <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabidiol\" title=\"Cannabidiol\" class=\"wiki-link\" data-key=\"bbfcdd40d5eda5d20d1d5ab368c922a0\">cannabidiol<\/a> (CBD) (the major non-psychoactive constituent in <i>C. sativa<\/i>): 1 g low THC cultivar (low THC\/very high CBD), 1 g medium THC cultivar (medium THC\/medium CBD), 1 g high THC cultivar (high THC\/low CBD), and 1 g very high THC cultivar (very high THC\/low CBD). The National Institute of Standards and Technology (NIST) (Gaithersburg, Maryland, USA) provided eight 0.5 g samples of marijuana that were confiscated by local law enforcement at different times over the past few years. Twenty-one strains of recreational marijuana were purchased from Garden Remedies Marijuana Dispensary (Melrose, Massachusetts, USA). Ten of the recreational samples were randomly selected for use in the development of the training model, while the remaining 11 samples were used to test the model (i.e., for external validation). Information for all marijuana samples (e.g., sample name, brand, supplier\/vendor, batch number, etc.) is available (see Additional file 1).\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Mass_spectral_acquisition_and_analysis_of_DART-HRMS-derived_data\">Mass spectral acquisition and analysis of DART-HRMS-derived data<\/span><\/h3>\n<p>The collection of mass spectral data was achieved by employing DART-HRMS. Two DART-HRMS instruments were used: (1) mass spectral data collected for all hemp products and the marijuana samples from DEA-registered suppliers were analyzed using the DART-HRMS instrument at the University at Albany (UAlbany) (Albany, New York, USA) and were translated and calibrated prior to data processing; and (2) all recreational marijuana flower samples were analyzed at IonSense Inc. (Saugus, Massachusetts, USA), with the raw data files calibrated, processed, and evaluated at UAlbany. The DART SVP (simplified voltage and pressure) ion source at IonSense was coupled to a JEOL AccuTOF high-resolution <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Time-of-flight_mass_spectrometry\" title=\"Time-of-flight mass spectrometry\" class=\"wiki-link\" data-key=\"ef7fd91770fcb12af7d71d544136e620\">time-of-flight (TOF) mass spectrometer<\/a> (Peabody, Massachusetts, USA) with a resolving power of 6000 full width at half maximum (FWHM) and mass accuracy of 5 millimass units (mmu). Data were collected in positive-ion mode using a DART ion source grid voltage of 300 V with the following mass spectrometer settings: ring lens, 5 V; orifice 1, 20 V; orifice 2 voltage, 5 V; peak voltage, 600 V; and detector voltage, 2000 V. The DART SVP ion source at UAlbany was also coupled to a JEOL AccuTOF high-resolution TOF mass spectrometer. The only difference between the DART ion source settings used at the two facilities was that the grid voltage at UAlbany was 250 V instead of 300 V. All mass spectral data were collected at a DART gas temperature of 350 \u00b0C using ultra-high purity helium gas at a flow rate of 2 L\/min. Mass spectra were collected at a rate of 1 spectrum per second over a mass range of m\/z 60\u20131000. TSSPro 3.0 software from Shrader Software Solutions (Grosse Pointe, Michigan, USA) was used for the calibration, spectral averaging, background subtraction, and peak centroiding of mass spectral data. Polyethylene glycol (PEG 600) (Sigma Aldrich, St. Louis, Missouri, USA) was used as the mass calibrant for all samples. Processing of the mass spectra of hemp and marijuana samples was performed with the Mass Mountaineer software suite from RBC Software (Portsmouth, New Hampshire, USA).\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Multivariate_data_analysis\">Multivariate data analysis<\/span><\/h3>\n<p>The workflow which extended from DART-HRMS data collection to multivariate data analysis is displayed in Fig. 1. In Step 1, DART mass spectra of the <i>C. sativa<\/i> samples representing hemp and marijuana varieties were acquired. The spectra in the form of text files were imported into MATLAB 9.9.0, R2020b Software (The MathWorks, Inc., Natick, Massachusetts, USA) and R 3.5.1 (R Core Team 2018) for analysis. Each text file was comprised of a two-column matrix of m\/z values and their corresponding abundances (i.e., ion counts). In Step 2, peaks were aligned along common <i>m\/z<\/i> values by histogram estimation and nearest-neighbor correction methods using the \u201c<i>mspalign<\/i>\u201d function in MATLAB. The generated matrix contained the aligned spectra for the replicates of hemp and marijuana samples. The replicates for each sample were averaged, normalized, transformed (with log 10), and subjected to unsupervised (Step 3) and supervised analyses (Step 4). As shown in Step 3, PCA<sup id=\"rdp-ebb-cite_ref-26\" class=\"reference\"><a href=\"#cite_note-26\">[26]<\/a><\/sup> and k-means<sup id=\"rdp-ebb-cite_ref-27\" class=\"reference\"><a href=\"#cite_note-27\">[27]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-28\" class=\"reference\"><a href=\"#cite_note-28\">[28]<\/a><\/sup> were used to recognize the similarity and dissimilarity patterns of the samples and to reveal possible clusters, respectively. Silhouette width indexes were calculated to indicate the optimal number of clusters characterized by k-means and to validate the goodness of the clustering results. The data matrix was analyzed using supervised <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Random_forest\" title=\"Random forest\" class=\"wiki-link\" data-key=\"3a557b0006d996735586ad28712f4771\">random forest<\/a> (RF)<sup id=\"rdp-ebb-cite_ref-29\" class=\"reference\"><a href=\"#cite_note-29\">[29]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-30\" class=\"reference\"><a href=\"#cite_note-30\">[30]<\/a><\/sup> (Step 4) to create a model for differentiating hemp and marijuana plant materials. RF is an ensemble of individual tree predictors, in which each tree in the forest is grown based on the independent replicas of training samples and variables. The samples not included in the replicates for a given tree (1\/3 of the original dataset) are termed \u201cout-of-bag\u201d (OOB) for that tree. The overall accuracy and performance characteristics of the discrimination model were estimated based on the predictions of OOB observations and external validation samples.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Results\">Results<\/span><\/h2>\n<h3><span class=\"mw-headline\" id=\"DART-HRMS_analysis_of_Cannabis_sativa_plant_material\">DART-HRMS analysis of <i>Cannabis sativa<\/i> plant material<\/span><\/h3>\n<p>Initial investigations of <i>C. sativa<\/i> plant material focused on obtaining the DART-HRMS chemical profiles for both hemp and marijuana flower samples. Detailed information about the samples, including variety, cultivar\/strain, vendor, and the batch number (when available) is provided (see Additional file 1). All samples were analyzed by inserting the closed end of a glass melting point capillary tube into the material and presenting the coated surface into the DART gas stream for approximately five seconds. A total of 29 hemp strains (i.e., cultivars) were purchased from three vendors at the beginning of this study, which included 27 CBD flower products and two <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabigerol\" title=\"Cannabigerol\" class=\"wiki-link\" data-key=\"6fc007b4c33a4c59a485b2c621035714\">cannabigerol<\/a> (CBG) flower products. CBD flower contains high levels of CBD and <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabidiolic_acid\" title=\"Cannabidiolic acid\" class=\"wiki-link\" data-key=\"772927eeed478eff799c410179be18a1\">cannabidiolic acid<\/a> (CBDA), while CBG flower contains high levels of CBG and <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabigerolic_acid\" title=\"Cannabigerolic acid\" class=\"wiki-link\" data-key=\"d0a5399af99137ebf5aa893d34531e49\">cannabigerolic acid<\/a> (CBGA). An additional 12 hemp samples were purchased at a later date to test the developed model. Utilizing DART-HRMS is optimal for analyzing hemp and marijuana samples in their native forms (i.e., with no sample pretreatment, such as a <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Decarboxylation\" title=\"Decarboxylation\" class=\"wiki-link\" data-key=\"d193a62772bddc5385f7d9c5e8543446\">decarboxylation<\/a> step) to rapidly obtain the small-molecule profiles (i.e., in under one minute). The DART-HR mass spectra of all hemp flower samples (training-set hemp and test-set hemp) collected in positive-ion mode under soft ionization conditions (20 V) are available (see Additional file 2). \n<\/p><p>Figure 2 shows representative DART-HR mass spectra acquired in positive-ion mode from analysis of <i>C. sativa<\/i> plant materials, including CBD (panel A) and CBG (panel D) hemp flower samples. The DART-HR mass spectra of all CBD hemp flower samples are very similar to one another; protonated masses consistent with CBD and CBDA were detected at <i>m\/z<\/i> 315 and 359, respectively, in all samples. DART-HRMS analysis of the two CBG hemp flower samples also yielded these peaks, in addition to peaks at nominal m\/z 317 and 361, which are consistent with the protonated masses of CBG and CBGA, respectively. The DART-HR mass spectra of the CBG hemp flower samples retained similarities with the CBD hemp flower profiles. However, indicative of the high CBG levels reported in the CBG flower samples, the relative intensities of the peaks attributed to CBG and CBGA were much higher in the DART-HR mass spectra of the CBG flower products.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=File:Fig2_Chambers_JofCannRes23_5.png\" class=\"image wiki-link\" data-key=\"6373058121f23316711cb6987df816be\"><img alt=\"Fig2 Chambers JofCannRes23 5.png\" src=\"https:\/\/www.cannaqa.wiki\/images\/e\/e0\/Fig2_Chambers_JofCannRes23_5.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Fig. 2.<\/b> Representative DART-HR mass spectra of commercial hemp flower (panels <b>A<\/b> and <b>D<\/b>), marijuana samples supplied by NIST (panel <b>B<\/b>) and NIDA (panel <b>E<\/b>), and recreational marijuana flower products (panels <b>C<\/b> and <b>F<\/b>). Peaks consistent with the protonated masses of THC\/CBD, CBG, THCA\/CBDA, and CBGA at nominal <i>m\/z<\/i> 315, 317, 359, and 361, respectively, were detected in the various samples.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p><i>C. sativa<\/i> plant material of the marijuana variety was acquired from two U.S. DEA-registered sources: (1) NIDA supplied four marijuana samples (approximately 1 g each) through the NIDA\/NIH Drug Supply Program; and (2) NIST provided eight marijuana samples (0.5 g each). All 12 marijuana samples were received in powdered form and were analyzed by DART-HRMS in positive-ion mode using the capillary tube sampling technique. Figure 2 presents two spectra of representative NIST (panel B) and NIDA (panel E) marijuana materials. Commercially available recreational marijuana samples were also analyzed. The DART-HR mass spectra for all marijuana samples from these suppliers are available (see Additional file 2). In total, 21 recreational marijuana samples were purchased from the Garden Remedies Marijuana Dispensary Adult-Use Menu. These products spanned the various marijuana strain types available (i.e., <i><a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Cannabis_indica\" title=\"Cannabis indica\" class=\"wiki-link\" data-key=\"7ea23da7d68f4d25012bac401d531b8b\">indica<\/a><\/i>-dominant, <i>sativa<\/i>-dominant, hybrid), which represent <i>C. sativa<\/i> subspecies. Figure 2 presents two representative DART-HR mass spectra for <i>indica<\/i> (panel C) and <i>sativa<\/i> (panel F) dominant flower samples. The mass spectral profiles of all recreational marijuana flower products are available (see Additional file 2). Ten of the samples were randomly selected for inclusion in the training model. The remaining 11 recreational flower samples were used to test the prediction ability of the model (i.e., for external validation).\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Differentiation_of_hemp_and_marijuana_varieties_of_C._sativa\">Differentiation of hemp and marijuana varieties of <i>C. sativa<\/i><\/span><\/h3>\n<p>The aim of this work was to accomplish the following: (1) develop a rapid, easy-to-use, and efficient means by which to differentiate hemp and marijuana varieties of <i>C. sativa<\/i>, and by extension, a method to identify <i>C. sativa<\/i> unknowns; and (2) circumvent some of the challenges typically encountered during the analysis of <i>C. sativa<\/i> materials when using chromatography-based methods. The approach is founded on the hypothesis that inherent in the small-molecule profiles of hemp and marijuana is the necessary information for the differentiation of these <i>Cannabis<\/i> varieties. Prior to the application of multivariate analysis methods to the features of the DART-HRMS-derived chemical profiles of hemp and marijuana, the spectra of all samples were binned to create a common <i>m\/z<\/i> reference vector to ease their comparison. Accordingly, the \u201c<i>mspalign<\/i>\u201d function in MATLAB was performed with a hist resolution parameter of 0.01, while the peak relative abundance cutoff threshold was set to 0.1% of the maximum intensity to detect all potentially significant peaks. The marijuana samples provided by NIDA and NIST were packaged in plastic bags, the composition of which contributed to the DART-HRMS profiles of the samples. Thus, the<i> m\/z<\/i> values derived from the packaging (e.g., nominal <i>m\/z<\/i> 59, 75, 89, 107, 127) were removed from the data. Another <i>m\/z<\/i> value that was removed was nominal <i>m\/z<\/i> 371, which has been previously shown to be a plasticizer present on the capillary tubes used for sampling.<sup id=\"rdp-ebb-cite_ref-31\" class=\"reference\"><a href=\"#cite_note-31\">[31]<\/a><\/sup> The resulting matrix had dimensions of 430\u2009\u00d7\u2009390 and contained the aligned spectra for the five replicates of each of the 41 hemp samples, the five replicates of each of the 21 recreational marijuana samples, and the 10 replicates of each of the 12 marijuana samples supplied by NIDA and NIST. The results of the preliminary PCA analysis were examined by Q residuals and Hotelling\u2019s T<sup>2<\/sup> statistic to detect any outliers, and this resulted in three spectra being removed from the data. Outlier spectra included those whose acquisition was accompanied by poor mass calibration or those that were not representative of a typical chemical profile. The averaging of sample replicates resulted in a matrix with dimensions of 74\u2009\u00d7\u2009390. Following logarithm transformation, the matrix was subjected to further analysis. Figure 3 panel A presents the PCA results as a 2-dimensional (2D) score plot, where the color-coded classes appear in the coordinate space represented by the first two principal components (PCs), which cover 41% of the data variance. While the recreational marijuana samples (cyan triangles) are located in close proximity to the NIDA-supplied marijuana sample that was reported to contain medium levels of both THC and CBD, they were distant from the other NIDA and NIST samples. These results support previous studies that indicated differences between marijuana sold at dispensaries, and that provided for research purposes by DEA-registered suppliers.<sup id=\"rdp-ebb-cite_ref-:10_17-1\" class=\"reference\"><a href=\"#cite_note-:10-17\">[17]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-32\" class=\"reference\"><a href=\"#cite_note-32\">[32]<\/a><\/sup> Clustering by k-means using one minus correlation metrics resulted in the categorization of the hemp samples into one cluster (magenta circles) and the marijuana samples into the other cluster (cyan circles).\n<\/p><p><br \/>\n<a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=File:Fig3_Chambers_JofCannRes23_5.png\" class=\"image wiki-link\" data-key=\"f5f4eaa5005d7dde9e07610eca861aaa\"><img alt=\"Fig3 Chambers JofCannRes23 5.png\" src=\"https:\/\/www.cannaqa.wiki\/images\/e\/e0\/Fig3_Chambers_JofCannRes23_5.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Fig. 3.<\/b> 2D score plot resulting from PCA of hemp and marijuana sample spectra (panel <b>A<\/b>); 2D score plot of multidimensional scaling (MDS) analysis of the proximity matrix resulting from the application of supervised random forest (panel <b>B<\/b>). The magenta and cyan colors represent hemp and marijuana, respectively. The cyan triangles show the subset of recreational marijuana samples.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>Even though the DART-HR mass spectra of hemp and marijuana plant materials are readily visually apparent, a more objective approach to the assessment of the identity of <i>C. sativa<\/i> material was devised, using the random forest algorithm. This was applied to the 74\u2009\u00d7\u2009390 matrix. A total of 33 flower samples (12 hemp and 11 marijuana) of the 74 total <i>C. sativa<\/i> samples were randomly selected for external validation to examine the ability of the model to accurately predict the class assignments for new sample unknowns. The number of variables (which were randomly sampled as candidates at each split), and the number of trees found to be optimal were 20 and 500, respectively. Figure 3, panel B displays the proximity matrix generated from using supervised RF with a multidimensional scaling (MDS) method to show the pairwise similarities in a 2D Cartesian space, with the magenta and cyan points corresponding to the hemp and marijuana samples, respectively. It demonstrates the number of times that observations ended up in the same leaf node. According to Figure 3, panel B, although the NIDA marijuana sample reported as low THC\/very high CBD is located between the two groups, the samples belonging to each group are close together and separated from the samples of the other group.\n<\/p><p>The optimal number of clusters was estimated by computing the average silhouette (which measures the quality of the clustering) of observations for different numbers of clusters. Figure 4, panel A displays the average silhouette width over a range of the possible number of clusters. The optimal number of clusters is the one that maximizes the average silhouette width. Based on the information provided in Figure 4, panel A, the optimal number of clusters is two. The silhouette plot in Figure 4, panel B displays silhouette coefficients for each sample when the data are split into two clusters. The silhouette width of each sample is a measure of how similar each sample is to its respective cluster in comparison to the other cluster. As shown in Figure 4, the optimum number of clusters is two: cluster 1 (magenta) has 40 members with a mean width of 0.23, and cluster 2 (cyan) has 34 members with a mean width of 0.45. Cluster 1 and cluster 2 members correspond to the samples of hemp and marijuana, respectively. One hemp sample was falsely clustered with the marijuana samples. The average silhouette width for the cluster of marijuana samples is higher than the average silhouette width for the hemp samples. This demonstrates that the cluster of marijuana samples is denser and that the samples are more similar to one another.\n<\/p><p><br \/>\n<a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=File:Fig4_Chambers_JofCannRes23_5.png\" class=\"image wiki-link\" data-key=\"4495a29b2325cf001be9d294f9a8d07c\"><img alt=\"Fig4 Chambers JofCannRes23 5.png\" src=\"https:\/\/www.cannaqa.wiki\/images\/c\/ca\/Fig4_Chambers_JofCannRes23_5.png\" decoding=\"async\" style=\"width: 100%;max-width: 400px;height: auto;\" \/><\/a>\n<\/p>\n<div style=\"clear:both;\"><\/div>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table border=\"0\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\"><blockquote><p><b>Fig. 4.<\/b> The average silhouette width over a range of cluster numbers (2\u20136) reveals that the optimum number of clusters is 2 (panel <b>A<\/b>). A silhouette plot (i.e., the visualization of the silhouette width for each sample) reveals the results with two clusters (panel <b>B<\/b>). Cluster 1 contains 40 members and cluster 2 contains 34 members. Hemp samples are shown in magenta, while marijuana samples are shown in cyan.<\/p><\/blockquote>\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<p>To reveal the model\u2019s ability to distinguish between hemp and marijuana samples, Table 1 presents the confusion matrix for the prediction of OOB samples, while Table 2 contains the performance characteristics of the model (accuracy, sensitivity, specificity, and precision) for predicting the OOB samples. According to this table, the model performed well and the accuracy for predicting OOB samples is 98%.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"4\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 1.<\/b> Confusion matrix associated with the prediction of \u201cout-of-bag\u201d samples in the random forest model.\n<\/td><\/tr>\n<tr>\n<th colspan=\"2\" rowspan=\"2\" style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Confusion matrix\n<\/th>\n<th colspan=\"2\" style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Prediction\n<\/th><\/tr>\n<tr>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Hemp\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Marijuana\n<\/th><\/tr>\n<tr>\n<td rowspan=\"2\" style=\"background-color:white; padding-left:10px; padding-right:10px;\">True\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Hemp (29)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1.00\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.00\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Marijuana (22)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.04\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.96\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"4\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 2.<\/b> Performance results of the random forest model for prediction of \u201cout-of-bag\u201d and external validation samples.\n<\/td><\/tr>\n<tr>\n<th rowspan=\"3\" style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">\n<\/th>\n<th colspan=\"3\" style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Out-of-bag samples\n<\/th><\/tr>\n<tr>\n<th colspan=\"3\" style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Accuracy: 0.98 (98%)\n<\/th><\/tr>\n<tr>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Sensitivity\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Specificity\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Precision\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Hemp (29)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1.00\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.96\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.97\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Marijuana (22)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.96\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1.00\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1.00\n<\/td><\/tr>\n<tr>\n<th rowspan=\"3\" style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">\n<\/th>\n<th colspan=\"3\" style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">External <i>C. sativa<\/i> plant materials\n<\/th><\/tr>\n<tr>\n<th colspan=\"3\" style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Accuracy: 1.00 (100%)\n<\/th><\/tr>\n<tr>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Sensitivity\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Specificity\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Precision\n<\/th><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Hemp (12)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1.00\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1.00\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1.00\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Marijuana (11)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1.00\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1.00\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1.00\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h3><span class=\"mw-headline\" id=\"Classification_of_external_C._sativa_plant_materials\">Classification of external <i>C. sativa<\/i> plant materials<\/span><\/h3>\n<p>The remaining 11 recreational marijuana flower products that were not included in the training set, in addition to the 12 hemp products purchased after the model had been developed, were screened against the model to test its ability to classify samples that were unknown to the model. Table 3 shows the confusion matrix results for the prediction of the test samples (i.e., for external validation). In addition, Table 2 shows the performance characteristics of the model for predicting the external <i>C. sativa<\/i> samples, with all performance merits equal to 1 for both test sample sets (i.e., hemp and marijuana). The information presented in Tables 1, 2, and 3 reveal that the model is well-fitted for discriminating the two <i>C. sativa<\/i> varieties.\n<\/p>\n<table style=\"\">\n<tbody><tr>\n<td style=\"vertical-align:top;\">\n<table class=\"wikitable\" border=\"1\" cellpadding=\"5\" cellspacing=\"0\" style=\"\">\n\n<tbody><tr>\n<td colspan=\"4\" style=\"background-color:white; padding-left:10px; padding-right:10px;\"><b>Table 3.<\/b> Confusion matrix associated with the prediction of external validation samples using a random forest model.\n<\/td><\/tr>\n<tr>\n<th colspan=\"2\" rowspan=\"2\" style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Confusion matrix\n<\/th>\n<th colspan=\"2\" style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Prediction\n<\/th><\/tr>\n<tr>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Hemp\n<\/th>\n<th style=\"background-color:#e2e2e2; padding-left:10px; padding-right:10px;\">Marijuana\n<\/th><\/tr>\n<tr>\n<td rowspan=\"2\" style=\"background-color:white; padding-left:10px; padding-right:10px;\">True\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Hemp (12)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1.00\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.00\n<\/td><\/tr>\n<tr>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">Marijuana (11)\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">0.00\n<\/td>\n<td style=\"background-color:white; padding-left:10px; padding-right:10px;\">1.00\n<\/td><\/tr>\n<\/tbody><\/table>\n<\/td><\/tr><\/tbody><\/table>\n<h2><span class=\"mw-headline\" id=\"Discussion\">Discussion<\/span><\/h2>\n<p>The most common methods for differentiating hemp and marijuana plant materials are chromatography-based approaches (e.g., GC-FID, GC\u2013MS, HPLC\u2013UV)<sup id=\"rdp-ebb-cite_ref-:0_3-1\" class=\"reference\"><a href=\"#cite_note-:0-3\">[3]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:1_4-1\" class=\"reference\"><a href=\"#cite_note-:1-4\">[4]<\/a><\/sup>, with the categorization based upon THC content. Several reports have emphasized the use of GC-FID<sup id=\"rdp-ebb-cite_ref-:5_8-1\" class=\"reference\"><a href=\"#cite_note-:5-8\">[8]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:13_33-0\" class=\"reference\"><a href=\"#cite_note-:13-33\">[33]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-34\" class=\"reference\"><a href=\"#cite_note-34\">[34]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-35\" class=\"reference\"><a href=\"#cite_note-35\">[35]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:14_36-0\" class=\"reference\"><a href=\"#cite_note-:14-36\">[36]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-37\" class=\"reference\"><a href=\"#cite_note-37\">[37]<\/a><\/sup> and GC\u2013MS<sup id=\"rdp-ebb-cite_ref-:13_33-1\" class=\"reference\"><a href=\"#cite_note-:13-33\">[33]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-:14_36-1\" class=\"reference\"><a href=\"#cite_note-:14-36\">[36]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-38\" class=\"reference\"><a href=\"#cite_note-38\">[38]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-39\" class=\"reference\"><a href=\"#cite_note-39\">[39]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-40\" class=\"reference\"><a href=\"#cite_note-40\">[40]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-41\" class=\"reference\"><a href=\"#cite_note-41\">[41]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-42\" class=\"reference\"><a href=\"#cite_note-42\">[42]<\/a><\/sup> methods for detection of natural cannabinoids (among other <i>Cannabis<\/i>-derived molecules) in various <i>Cannabis<\/i> plant materials. Modifications to standard GC-FID and GC\u2013MS protocols include GC-vacuum UV (VUV) spectroscopy<sup id=\"rdp-ebb-cite_ref-43\" class=\"reference\"><a href=\"#cite_note-43\">[43]<\/a><\/sup>, two-dimensional GC-FID (GCxGC-FID)<sup id=\"rdp-ebb-cite_ref-44\" class=\"reference\"><a href=\"#cite_note-44\">[44]<\/a><\/sup>, and GCxGC-MS with multivariate curve resolution-alternating least squares (MCR-ALS).<sup id=\"rdp-ebb-cite_ref-45\" class=\"reference\"><a href=\"#cite_note-45\">[45]<\/a><\/sup> However, these methods rely upon the quantification of THC, which can be plagued with a number of analytical challenges, such as baseline separation of peaks and lengthy sample preparation protocols.\n<\/p><p>In an effort to circumvent the need to extend run times or incorporate extra sample preparation steps, several studies have investigated alternative sample collection techniques coupled with chromatography-based methods to differentiate <i>C. sativa<\/i> varieties. One study demonstrated the use of capillary microextraction of volatiles (CMV) coupled with GC\u2013MS to distinguish the headspace volatiles of marijuana and hemp products based on their apparently distinct volatile organic compound (VOC) profiles.<sup id=\"rdp-ebb-cite_ref-:2_5-1\" class=\"reference\"><a href=\"#cite_note-:2-5\">[5]<\/a><\/sup> However, this report revealed that potential adulterants and inconsistent packaging of samples may have contributed to the observed distinctions.<sup id=\"rdp-ebb-cite_ref-:2_5-2\" class=\"reference\"><a href=\"#cite_note-:2-5\">[5]<\/a><\/sup> Another study utilized GC\u2013MS coupled with dispersive pipette extraction (DPX) to investigate forensic casework marijuana and donated hemp samples.<sup id=\"rdp-ebb-cite_ref-:3_6-1\" class=\"reference\"><a href=\"#cite_note-:3-6\">[6]<\/a><\/sup> Although the approach was successful at differentiating the two varieties with greater than 98% accuracy, a significant reduction of THC stability after 48 hours indicated that the samples would need to be reanalyzed if there was a delay between sample preparation and instrumental analysis.<sup id=\"rdp-ebb-cite_ref-:3_6-2\" class=\"reference\"><a href=\"#cite_note-:3-6\">[6]<\/a><\/sup> Another GC-based study sought to differentiate hemp and marijuana through their cannabinoid and terpene profiles using GC-FID and <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Principal_component_analysis\" title=\"Principal component analysis\" class=\"wiki-link\" data-key=\"f19d0fd1bc76a5685819562993c89812\">principal component analysis<\/a> (PCA).<sup id=\"rdp-ebb-cite_ref-:4_7-1\" class=\"reference\"><a href=\"#cite_note-:4-7\">[7]<\/a><\/sup> This study, which included two recreational cultivars and three pharmacy <i>Cannabis<\/i> samples, successfully distinguished between the two <i>C. sativa<\/i> varieties.<sup id=\"rdp-ebb-cite_ref-:4_7-2\" class=\"reference\"><a href=\"#cite_note-:4-7\">[7]<\/a><\/sup> In this case, expanding the sample source diversity could strengthen the ability of the model to classify a wider range of <i>Cannabis<\/i> samples. Another study applied PCA algorithms to quantitative data acquired from <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=High-performance_liquid_chromatography\" title=\"High-performance liquid chromatography\" class=\"wiki-link\" data-key=\"4c5d83ffa9785383c3eb0be7ea78dd2b\">high-performance liquid chromatography<\/a>-<a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Mass_spectrometry\" title=\"Mass spectrometry\" class=\"wiki-link\" data-key=\"18314b70982e52d5a81db4a757d6461f\">mass spectrometry<\/a> (HPLC\u2013MS) analysis of <i>Cannabis<\/i> plant materials.<sup id=\"rdp-ebb-cite_ref-:5_8-2\" class=\"reference\"><a href=\"#cite_note-:5-8\">[8]<\/a><\/sup> This study identified several cannabinoids essential for differentiating between <i>Cannabis<\/i> strain types<sup id=\"rdp-ebb-cite_ref-:5_8-3\" class=\"reference\"><a href=\"#cite_note-:5-8\">[8]<\/a><\/sup> (i.e., strains within the marijuana variety) as opposed to specifically targeting the cannabinoids essential to differentiating <i>C. sativa<\/i> varieties (i.e., hemp and marijuana), which would be important for criminal justice purposes in the U.S. Although many of these investigations were successful at differentiating between hemp and marijuana varieties or strains, the methods are reliant upon chromatography and are therefore susceptible to the aforementioned delineated challenges that can arise using this technique (i.e., lengthy run times, column contamination, etc.).\n<\/p><p>Non-chromatographic approaches that circumvent the requirement to separate and\/or differentiate between cannabinoids have also been investigated for distinguishing hemp and marijuana. A hand-held Raman spectrometer coupled with orthogonal partial least squares-discriminant analysis (OPLS-DA) tools proved successful in differentiating between the two <i>C. sativa<\/i> varieties.<sup id=\"rdp-ebb-cite_ref-46\" class=\"reference\"><a href=\"#cite_note-46\">[46]<\/a><\/sup> However, \u201creal\u201d forensic casework samples are rarely received in pristine form, and as such, the Raman approach is susceptible to interferences from various components that may be associated with the complex matrix and interfere with the Raman signal. Another study utilized advanced statistical modeling of nuclear magnetic resonance (NMR) spectroscopy and mass spectral data of <i>C. sativa<\/i> extracts<sup id=\"rdp-ebb-cite_ref-47\" class=\"reference\"><a href=\"#cite_note-47\">[47]<\/a><\/sup>, which is unique in that it is typically difficult to utilize NMR for the analysis of complex matrices and mixtures. Although effective, this instrumentation is not commonly found in forensic or other <i>Cannabis<\/i> analysis laboratories due to expensive start-up and maintenance costs.\n<\/p><p>Colorimetric tests are also commonly used for differentiating between hemp and marijuana varieties of <i>Cannabis<\/i>, especially in forensic fieldwork, and these do not generally require instrumental analysis to arrive at a presumptive identification. A validated method utilizing the 4-aminophenol color test to differentiate hemp and marijuana revealed some degree of success.<sup id=\"rdp-ebb-cite_ref-:6_10-1\" class=\"reference\"><a href=\"#cite_note-:6-10\">[10]<\/a><\/sup> However, this test can yield inconclusive results with samples that have THC and CBD levels that are within a factor of three of one another.<sup id=\"rdp-ebb-cite_ref-:6_10-2\" class=\"reference\"><a href=\"#cite_note-:6-10\">[10]<\/a><\/sup> Another common color test for the identification of marijuana samples is the Fast Blue BB (FBBB) colorimetric test, which reacts with the cannabinoids present in <i>Cannabis<\/i> (primarily THC). A study utilizing this test found that hemp and marijuana plant materials could be classified correctly when linear discriminant analysis (LDA) was used to develop a model based on RGB (red, green, blue) numerical codes from both fluorescence and color images that resulted from the application of the FBBB color test.<sup id=\"rdp-ebb-cite_ref-:8_12-1\" class=\"reference\"><a href=\"#cite_note-:8-12\">[12]<\/a><\/sup> Positive-ion mode electrospray ionization Fourier transform-ion cyclotron resonance mass spectrometry (ESI(\u2009+)FT-ICR MS, ESI(\u2009+)MS\/MS, ultraviolet\u2013visible (UV\u2013Vis) spectroscopy, and <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Thin-layer_chromatography\" title=\"Thin-layer chromatography\" class=\"wiki-link\" data-key=\"c8f68c661f30ba974342dd75a4081655\">thin-layer chromatography<\/a> (TLC) techniques have been used to investigate the products (i.e., chromophores) resulting from the application of the FBBB test to marijuana samples.<sup id=\"rdp-ebb-cite_ref-48\" class=\"reference\"><a href=\"#cite_note-48\">[48]<\/a><\/sup> In addition, direct analysis in real time-mass spectrometry (DART-MS) and 1H NMR techniques were coupled to identify the chromophores produced when various cannabinoids react with the FBBB reagent.<sup id=\"rdp-ebb-cite_ref-49\" class=\"reference\"><a href=\"#cite_note-49\">[49]<\/a><\/sup> A third color test to identify marijuana through the presence of THC is the Duquenois-Levine test. Research has been conducted to characterize (by mass spectrometry) the chromophores formed when cannabinoids react with the Duquenois reagents.<sup id=\"rdp-ebb-cite_ref-:9_13-1\" class=\"reference\"><a href=\"#cite_note-:9-13\">[13]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-50\" class=\"reference\"><a href=\"#cite_note-50\">[50]<\/a><\/sup><sup id=\"rdp-ebb-cite_ref-51\" class=\"reference\"><a href=\"#cite_note-51\">[51]<\/a><\/sup> Similar to the chromatography-based methods described, these tests all rely upon detection of THC specifically, which can complicate analyses because both marijuana and hemp contain this compound. Thus, while the distinction between marijuana and hemp has been defined based on THC levels, this is accompanied by the several aforementioned analytical challenges. By using the entire metabolomic profiles of hemp and marijuana acquired through ambient ionization mass spectrometry, the method presented here does not rely solely on the presence of any one molecule (or set of molecules), ratios of molecules to one another, or the ability to differentiate between cannabinoid isomers (i.e., THC and CBD).\n<\/p><p>The overall results of this study reveal that DART-HRMS yields consistent and unique chemical profiles for analyzed <i>Cannabis<\/i> materials that enable hemp and marijuana samples to be accurately differentiated, while circumventing challenges typically encountered with traditional chromatography methods (difficulties with cannabinoid separation and extensive sample preparation) and presumptive color tests (inconclusive or false positive results). Furthermore, this study utilized a sample set that demonstrates a balance between the total number of samples included, the number of replicates obtained, and a diversity in sources from which the <i>C. sativa<\/i> materials were acquired. This research provides a strong foundation upon which to develop a comprehensive mass spectral database for identifying unknown <i>C. sativa<\/i> variants through the acquisition of their DART-HR mass spectra. While the approach does not aim to replace confirmatory testing for THC concentrations, the model accomplishes the following: (1) bypasses the typical sample preparation steps required for analyzing materials by chromatography-based methods that seek to differentiate the samples through separation of their constituent cannabinoids; (2) reduces the chances for false positives that can result from presumptive color tests; and (3) serves as a supplementary tool for forensic investigators that enables more targeted confirmatory testing. \n<\/p><p>This is timely and highly relevant, given the introduction in the U.S. House of Representatives of the \u201cH.R.6645 \u2013 Hemp Advancement Act of 2022\u201d bill.<sup id=\"rdp-ebb-cite_ref-:15_52-0\" class=\"reference\"><a href=\"#cite_note-:15-52\">[52]<\/a><\/sup> This act aims to amend the current federal ruling regarding hemp by: (1) changing the 0.3% [THC] designation to 1% and (2) replacing the word \u201cdelta-9\u201d with the word \u201ctotal\u201d to include the various isomers of THC that have emerged in recent years.<sup id=\"rdp-ebb-cite_ref-:15_52-1\" class=\"reference\"><a href=\"#cite_note-:15-52\">[52]<\/a><\/sup> The introduction of this bill underscores some of the disadvantages of utilizing THC cutoffs in particular as the sole means by which to identify hemp and marijuana. Among other issues, it upends well-established and long-standing practices in criminalistics in a fashion that is expensive to address, since it will require the development of an entirely new set of protocols and data processing steps. Furthermore, it may not stand the test of time, as the cutoff thresholds are subject to change in the future. A method such as the one presented here, and which does not solely rely upon a 0.3% THC cutoff, is not at risk of becoming outdated upon further advancements of this bill or others in the U.S. House and Senate.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Conclusions\">Conclusions<\/span><\/h2>\n<p>A combined ambient ionization mass spectrometric (i.e., DART-HRMS) and chemometric approach was successfully used to create a prediction model that facilitated rapid high-accuracy differentiation of <i>C. sativa<\/i> hemp and marijuana plant materials obtained from multiple sources (i.e., commercial, DEA-registered, recreational). This method, which circumvents sample pretreatment steps (i.e., <a href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Solvent\" title=\"Solvent\" class=\"wiki-link\" data-key=\"978fd0f359891b1cb3927105fa01e23f\">solvent<\/a> extractions), addresses some of the difficulties encountered when analyzing samples using more conventional forensic analysis methodologies. A primary example of this is eliminating the need to separate and differentiate cannabinoids by chromatography techniques in order to determine the sample\u2019s THC content, which is the primary basis for distinguishing between hemp and marijuana varieties of <i>Cannabis<\/i> for most methods. When new hemp and recreational marijuana flower products were screened against the model developed in this study, 100% accuracy in prediction was observed. The identities of <i>m\/z<\/i> values that were determined to be important for the optimal differentiation of hemp and marijuana are the subject of continuing investigations. In addition, it is possible that <i>C. sativa<\/i> materials (of either the hemp or marijuana variety) with atypical levels of minor cannabinoids (such as CBN or isomers of THC) may respond differently in the DART gas stream and that this, in turn, may influence the results predicted by the model. Therefore, samples such as these will be investigated (as was done with the analysis of the two CBG hemp flower samples), along with new samples\/strains from commercial and DEA-registered suppliers as they become available so that the model reflects ongoing changes in the chemical profiles of <i>Cannabis<\/i> products on the market.\n<\/p>\n<h2><span class=\"mw-headline\" id=\"Supplementary_information\">Supplementary information<\/span><\/h2>\n<ul><li><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/static-content.springer.com\/esm\/art%3A10.1186%2Fs42238-023-00173-0\/MediaObjects\/42238_2023_173_MOESM1_ESM.docx\" target=\"_blank\">Additional file 1<\/a> (.docx): Supplementary Mass Spectral Data and Sample Information. (1) Information about <i>C. sativa<\/i> plant materials analyzed in this study.<\/li><\/ul>\n<ul><li><a rel=\"external_link\" class=\"external text\" href=\"https:\/\/static-content.springer.com\/esm\/art%3A10.1186%2Fs42238-023-00173-0\/MediaObjects\/42238_2023_173_MOESM2_ESM.docx\" target=\"_blank\">Additional file 2<\/a> (.docx): Supplementary Mass Spectral Data for <i>C. sativa<\/i> Materials. (1) DART-HR mass spectra for hemp and marijuana materials.<\/li><\/ul>\n<h2><span id=\"rdp-ebb-Abbreviations,_acronyms,_and_initialisms\"><\/span><span class=\"mw-headline\" id=\"Abbreviations.2C_acronyms.2C_and_initialisms\">Abbreviations, acronyms, and initialisms<\/span><\/h2>\n<ul><li><b>2D<\/b>: two-dimensional<\/li>\n<li><b>CBD<\/b>: cannabidiol<\/li>\n<li><b>CBDA<\/b>: cannabidiolic acid<\/li>\n<li><b>CBG<\/b>: cannabigerol<\/li>\n<li><b>CBGA<\/b>: cannabigerolic acid<\/li>\n<li><b>CMV<\/b>: capillary microextraction of volatiles<\/li>\n<li><b>DART<\/b>: direct analysis in real-time<\/li>\n<li><b>DEA<\/b>: U.S. Drug Enforcement Administration<\/li>\n<li><b>DESI<\/b>: desorption electrospray ionization<\/li>\n<li><b>DPX<\/b>: dispersive pipette extraction<\/li>\n<li><b>ESI<\/b>: electrospray ionization<\/li>\n<li><b>FBBB<\/b>: Fast Blue BB<\/li>\n<li><b>FID<\/b>: flame ionization detection<\/li>\n<li><b>FT<\/b>: Fourier transform<\/li>\n<li><b>FWHM<\/b>: full width at half maximum<\/li>\n<li><b>GBS<\/b>: genotyping-by-sequencing<\/li>\n<li><b>GC<\/b>: gas chromatography<\/li>\n<li><b>GCxGC<\/b>: two-dimensional gas chromatography<\/li>\n<li><b>HR<\/b>: high-resolution<\/li>\n<li><b>HRMS<\/b>: high-resolution mass spectrometry<\/li>\n<li><b>HPLC<\/b>: high-performance liquid chromatography<\/li>\n<li><b>ICR<\/b>: ion cyclotron resonance<\/li>\n<li><b>LDA<\/b>: linear discriminant analysis<\/li>\n<li><b>MCD-ALS<\/b>: multivariate curve resolution-alternating least squares<\/li>\n<li><b>MDS<\/b>: multidimensional scaling<\/li>\n<li><b>mmu<\/b>: millimass unit<\/li>\n<li><b>MS<\/b>: mass spectrometry<\/li>\n<li><b>NIDA<\/b>: ational Institute on Drug Abuse<\/li>\n<li><b>NIH<\/b>: ational Institutes of Health<\/li>\n<li><b>NIJ<\/b>: ational Institute of Justice<\/li>\n<li><b>NIST<\/b>: ational Institute of Standards and Technology<\/li>\n<li><b>NMR<\/b>: uclear magnetic resonance<\/li>\n<li><b>OOB<\/b>: ut-of-bag<\/li>\n<li><b>OPLS-DA<\/b>: rthogonal partial least squares-discriminant analysis<\/li>\n<li><b>PC<\/b>: rincipal component<\/li>\n<li><b>PCA<\/b>: rincipal component analysis<\/li>\n<li><b>PEG<\/b>: olyethylene glycol<\/li>\n<li><b>PLS-DA<\/b>: artial least squares-discriminant analysis<\/li>\n<li><b>RF<\/b>: andom forest<\/li>\n<li><b>RGB<\/b>: ed, green, blue<\/li>\n<li><b>RTP<\/b>: Research Triangle Institute<\/li>\n<li><b>SNP<\/b>: single-nucleotide polymorphism<\/li>\n<li><b>SUNY<\/b>: State University of New York<\/li>\n<li><b>SVP<\/b>: simplified voltage and pressure<\/li>\n<li><b>TD<\/b>: thermal desorption<\/li>\n<li><b>THCA<\/b>: delta-9-tetrahydrocannabinolic acid<\/li>\n<li><b>TLC<\/b>: thin-layer chromatography<\/li>\n<li><b>TOF<\/b>: time-of-flight<\/li>\n<li><b>UAlbany<\/b>: The University at Albany<\/li>\n<li><b>UV<\/b>: ultraviolet<\/li>\n<li><b>Vis<\/b>: visible<\/li>\n<li><b>VOC<\/b>: volatile organic compounds<\/li>\n<li><b>VUV<\/b>: vacuum UV<\/li>\n<li><b>\u2206<sup>9<\/sup>-THC or THC<\/b>: delta-9-tetrahydrocannabinol<\/li><\/ul>\n<h2><span class=\"mw-headline\" id=\"Acknowledgements\">Acknowledgements<\/span><\/h2>\n<p>Thanks are extended to the National Institute on Drug Abuse\/National Institutes of Health (NIDA\/NIH) and the National Institute of Standards and Technology (NIST) for supplying <i>Cannabis sativa<\/i> marijuana samples analyzed in this study. Thanks are extended to IonSense, Inc. for the analysis of recreational <i>Cannabis<\/i> flower products and to Dr. Brent Wilson (NIST) for helpful assistance.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Author_contributions\">Author contributions<\/span><\/h3>\n<p>RAM conceived of the project, data analysis, project design, and project management and drafted the manuscript; MIC contributed to the experimental work and data analysis and drafted the manuscript; SB contributed to the data processing and data analysis and drafted the manuscript; BG contributed to the experimental work and data analysis. The authors read and approved the final manuscript.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Funding\">Funding<\/span><\/h3>\n<p>The financial support of the National Institute of Justice (NIJ), Office of Justice programs, U.S. Department of Justice (DOJ) under Grant Nos. 2015-DN-BX-K057, 2017-R2-CX-0020 and 2019-BU-DX-0026 to RAM; the U.S. National Science Foundation (NSF) under Grant No. 1429329 to RAM; the 2020 Northeastern Association of Forensic Scientists (NEAFS) Carol De Forest Research Grant to MIC; the Initiatives for Women Foundation (IFW) Karen R. Hitchcock New Frontiers award to MIC; and the Research Foundation of SUNY are gratefully acknowledged. The opinions, findings, and conclusions or recommendations expressed in this publication are those of the authors and do not necessarily reflect those of the DOJ and\/or the NSF.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Availability_of_data_and_materials\">Availability of data and materials<\/span><\/h3>\n<p>The datasets analyzed in the current study are available upon request at the discretion of the corresponding author.\n<\/p>\n<h3><span class=\"mw-headline\" id=\"Competing_interests\">Competing interests<\/span><\/h3>\n<p>The authors declare that they have no competing interests.\n<\/p><p><br \/>\n<\/p>\n<h2><span class=\"mw-headline\" id=\"References\">References<\/span><\/h2>\n<div class=\"reflist references-column-width\" style=\"-moz-column-width: 30em; -webkit-column-width: 30em; column-width: 30em; list-style-type: decimal;\">\n<div class=\"mw-references-wrap mw-references-columns\"><ol class=\"references\">\n<li id=\"cite_note-1\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-1\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">National Institute of Justice (April 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.ojp.gov\/pdffiles1\/nij\/253626.pdf\" target=\"_blank\">\"Report to Congress: Needs Assessment of Forensic Laboratories and Medical Examiner\/Coroner Offices\"<\/a> (PDF). U.S. Department of Justice. pp. 86\u201397<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.ojp.gov\/pdffiles1\/nij\/253626.pdf\" target=\"_blank\">https:\/\/www.ojp.gov\/pdffiles1\/nij\/253626.pdf<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Report+to+Congress%3A+Needs+Assessment+of+Forensic+Laboratories+and+Medical+Examiner%2FCoroner+Offices&rft.atitle=&rft.aulast=National+Institute+of+Justice&rft.au=National+Institute+of+Justice&rft.date=April+2019&rft.pages=pp.+86%E2%80%9397&rft.pub=U.S.+Department+of+Justice&rft_id=https%3A%2F%2Fwww.ojp.gov%2Fpdffiles1%2Fnij%2F253626.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-2\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-2\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Conaway, K.M. (20 December 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.congress.gov\/bill\/115th-congress\/house-bill\/112\" target=\"_blank\">\"H.R.2 - Agriculture Improvement Act of 2018\"<\/a>. <i>Congress.gov<\/i>. Library of Congress<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.congress.gov\/bill\/115th-congress\/house-bill\/112\" target=\"_blank\">https:\/\/www.congress.gov\/bill\/115th-congress\/house-bill\/112<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=H.R.2+-+Agriculture+Improvement+Act+of+2018&rft.atitle=Congress.gov&rft.aulast=Conaway%2C+K.M.&rft.au=Conaway%2C+K.M.&rft.date=20+December+2018&rft.pub=Library+of+Congress&rft_id=https%3A%2F%2Fwww.congress.gov%2Fbill%2F115th-congress%2Fhouse-bill%2F112&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:0-3\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:0_3-0\">3.0<\/a><\/sup> <sup><a href=\"#cite_ref-:0_3-1\">3.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Pourseyed Lazarjani, Masoumeh; Torres, Stephanie; Hooker, Thom; Fowlie, Chris; Young, Owen; Seyfoddin, Ali (1 December 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/jcannabisresearch.biomedcentral.com\/articles\/10.1186\/s42238-020-00040-2\" target=\"_blank\">\"Methods for quantification of cannabinoids: a narrative review\"<\/a> (in en). <i>Journal of Cannabis Research<\/i> <b>2<\/b> (1): 35. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1186%2Fs42238-020-00040-2\" target=\"_blank\">10.1186\/s42238-020-00040-2<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2522-5782\" target=\"_blank\">2522-5782<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.pubmedcentral.gov\/articlerender.fcgi?tool=pmcentrez&artid=PMC7819317\" target=\"_blank\">PMC7819317<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/33526084\" target=\"_blank\">33526084<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/jcannabisresearch.biomedcentral.com\/articles\/10.1186\/s42238-020-00040-2\" target=\"_blank\">https:\/\/jcannabisresearch.biomedcentral.com\/articles\/10.1186\/s42238-020-00040-2<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Methods+for+quantification+of+cannabinoids%3A+a+narrative+review&rft.jtitle=Journal+of+Cannabis+Research&rft.aulast=Pourseyed+Lazarjani&rft.aufirst=Masoumeh&rft.au=Pourseyed+Lazarjani%2C%26%2332%3BMasoumeh&rft.au=Torres%2C%26%2332%3BStephanie&rft.au=Hooker%2C%26%2332%3BThom&rft.au=Fowlie%2C%26%2332%3BChris&rft.au=Young%2C%26%2332%3BOwen&rft.au=Seyfoddin%2C%26%2332%3BAli&rft.date=1+December+2020&rft.volume=2&rft.issue=1&rft.pages=35&rft_id=info:doi\/10.1186%2Fs42238-020-00040-2&rft.issn=2522-5782&rft_id=info:pmc\/PMC7819317&rft_id=info:pmid\/33526084&rft_id=https%3A%2F%2Fjcannabisresearch.biomedcentral.com%2Farticles%2F10.1186%2Fs42238-020-00040-2&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:1-4\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:1_4-0\">4.0<\/a><\/sup> <sup><a href=\"#cite_ref-:1_4-1\">4.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation web\">Laboratory and Scientific Section, United Nations Office on Drugs and Crime (2009). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.unodc.org\/documents\/scientific\/ST-NAR-40-Ebook_1.pdf\" target=\"_blank\">\"Recommended methods for the identification and analysis of cannabis and cannabis products\"<\/a> (PDF). <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-92-1-148242-3<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.unodc.org\/documents\/scientific\/ST-NAR-40-Ebook_1.pdf\" target=\"_blank\">https:\/\/www.unodc.org\/documents\/scientific\/ST-NAR-40-Ebook_1.pdf<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Recommended+methods+for+the+identification+and+analysis+of+cannabis+and+cannabis+products&rft.atitle=&rft.aulast=Laboratory+and+Scientific+Section%2C+United+Nations+Office+on+Drugs+and+Crime&rft.au=Laboratory+and+Scientific+Section%2C+United+Nations+Office+on+Drugs+and+Crime&rft.date=2009&rft.isbn=978-92-1-148242-3&rft_id=https%3A%2F%2Fwww.unodc.org%2Fdocuments%2Fscientific%2FST-NAR-40-Ebook_1.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:2-5\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:2_5-0\">5.0<\/a><\/sup> <sup><a href=\"#cite_ref-:2_5-1\">5.1<\/a><\/sup> <sup><a href=\"#cite_ref-:2_5-2\">5.2<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Wiebelhaus, Nancy; Hamblin, D\u2019Nisha; Kreitals, Natasha M.; Almirall, Jose R. (1 November 2016). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170916300285\" target=\"_blank\">\"Differentiation of marijuana headspace volatiles from other plants and hemp products using capillary microextraction of volatiles (CMV) coupled to gas-chromatography\u2013mass spectrometry (GC\u2013MS)\"<\/a> (in en). <i>Forensic Chemistry<\/i> <b>2<\/b>: 1\u20138. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.forc.2016.08.004\" target=\"_blank\">10.1016\/j.forc.2016.08.004<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170916300285\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170916300285<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Differentiation+of+marijuana+headspace+volatiles+from+other+plants+and+hemp+products+using+capillary+microextraction+of+volatiles+%28CMV%29+coupled+to+gas-chromatography%E2%80%93mass+spectrometry+%28GC%E2%80%93MS%29&rft.jtitle=Forensic+Chemistry&rft.aulast=Wiebelhaus&rft.aufirst=Nancy&rft.au=Wiebelhaus%2C%26%2332%3BNancy&rft.au=Hamblin%2C%26%2332%3BD%E2%80%99Nisha&rft.au=Kreitals%2C%26%2332%3BNatasha+M.&rft.au=Almirall%2C%26%2332%3BJose+R.&rft.date=1+November+2016&rft.volume=2&rft.pages=1%E2%80%938&rft_id=info:doi\/10.1016%2Fj.forc.2016.08.004&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2468170916300285&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:3-6\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:3_6-0\">6.0<\/a><\/sup> <sup><a href=\"#cite_ref-:3_6-1\">6.1<\/a><\/sup> <sup><a href=\"#cite_ref-:3_6-2\">6.2<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Horne, Melissa; Mastrianni, Kaylee R.; Amick, Gray; Hardy, Rachel; Renneker, Elissa; Miller, Kevin W.P. (1 September 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/1556-4029.14525\" target=\"_blank\">\"Fast Discrimination of Marijuana using Automated High\u2010throughput Cannabis Sample Preparation and Analysis by Gas Chromatography\u2013Mass Spectrometry\"<\/a> (in en). <i>Journal of Forensic Sciences<\/i> <b>65<\/b> (5): 1709\u20131715. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1111%2F1556-4029.14525\" target=\"_blank\">10.1111\/1556-4029.14525<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0022-1198\" target=\"_blank\">0022-1198<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/1556-4029.14525\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/1556-4029.14525<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Fast+Discrimination+of+Marijuana+using+Automated+High%E2%80%90throughput+Cannabis+Sample+Preparation+and+Analysis+by+Gas+Chromatography%E2%80%93Mass+Spectrometry&rft.jtitle=Journal+of+Forensic+Sciences&rft.aulast=Horne&rft.aufirst=Melissa&rft.au=Horne%2C%26%2332%3BMelissa&rft.au=Mastrianni%2C%26%2332%3BKaylee+R.&rft.au=Amick%2C%26%2332%3BGray&rft.au=Hardy%2C%26%2332%3BRachel&rft.au=Renneker%2C%26%2332%3BElissa&rft.au=Miller%2C%26%2332%3BKevin+W.P.&rft.date=1+September+2020&rft.volume=65&rft.issue=5&rft.pages=1709%E2%80%931715&rft_id=info:doi\/10.1111%2F1556-4029.14525&rft.issn=0022-1198&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1111%2F1556-4029.14525&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:4-7\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:4_7-0\">7.0<\/a><\/sup> <sup><a href=\"#cite_ref-:4_7-1\">7.1<\/a><\/sup> <sup><a href=\"#cite_ref-:4_7-2\">7.2<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Pacula, Rosalie Liccardo; Jacobson, Mireille; Maksabedian, Ervant J. (1 June 2016). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/add.13282\" target=\"_blank\">\"In the weeds: a baseline view of cannabis use among legalizing states and their neighbours: In the weeds: a baseline view of cannabis use among legalizing states and their neighbours\"<\/a> (in en). <i>Addiction<\/i> <b>111<\/b> (6): 973\u2013980. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1111%2Fadd.13282\" target=\"_blank\">10.1111\/add.13282<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.pubmedcentral.gov\/articlerender.fcgi?tool=pmcentrez&artid=PMC5216038\" target=\"_blank\">PMC5216038<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/26687431\" target=\"_blank\">26687431<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/add.13282\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/add.13282<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=In+the+weeds%3A+a+baseline+view+of+cannabis+use+among+legalizing+states+and+their+neighbours%3A+In+the+weeds%3A+a+baseline+view+of+cannabis+use+among+legalizing+states+and+their+neighbours&rft.jtitle=Addiction&rft.aulast=Pacula&rft.aufirst=Rosalie+Liccardo&rft.au=Pacula%2C%26%2332%3BRosalie+Liccardo&rft.au=Jacobson%2C%26%2332%3BMireille&rft.au=Maksabedian%2C%26%2332%3BErvant+J.&rft.date=1+June+2016&rft.volume=111&rft.issue=6&rft.pages=973%E2%80%93980&rft_id=info:doi\/10.1111%2Fadd.13282&rft_id=info:pmc\/PMC5216038&rft_id=info:pmid\/26687431&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1111%2Fadd.13282&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:5-8\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:5_8-0\">8.0<\/a><\/sup> <sup><a href=\"#cite_ref-:5_8-1\">8.1<\/a><\/sup> <sup><a href=\"#cite_ref-:5_8-2\">8.2<\/a><\/sup> <sup><a href=\"#cite_ref-:5_8-3\">8.3<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Fischedick, Justin Thomas; Hazekamp, Arno; Erkelens, Tjalling; Choi, Young Hae; Verpoorte, Rob (1 December 2010). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S003194221000381X\" target=\"_blank\">\"Metabolic fingerprinting of Cannabis sativa L., cannabinoids and terpenoids for chemotaxonomic and drug standardization purposes\"<\/a> (in en). <i>Phytochemistry<\/i> <b>71<\/b> (17-18): 2058\u20132073. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.phytochem.2010.10.001\" target=\"_blank\">10.1016\/j.phytochem.2010.10.001<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S003194221000381X\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S003194221000381X<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Metabolic+fingerprinting+of+Cannabis+sativa+L.%2C+cannabinoids+and+terpenoids+for+chemotaxonomic+and+drug+standardization+purposes&rft.jtitle=Phytochemistry&rft.aulast=Fischedick&rft.aufirst=Justin+Thomas&rft.au=Fischedick%2C%26%2332%3BJustin+Thomas&rft.au=Hazekamp%2C%26%2332%3BArno&rft.au=Erkelens%2C%26%2332%3BTjalling&rft.au=Choi%2C%26%2332%3BYoung+Hae&rft.au=Verpoorte%2C%26%2332%3BRob&rft.date=1+December+2010&rft.volume=71&rft.issue=17-18&rft.pages=2058%E2%80%932073&rft_id=info:doi\/10.1016%2Fj.phytochem.2010.10.001&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS003194221000381X&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-9\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-9\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Philp, Morgan; Shimmon, Ronald; Tahtouh, Mark; Fu, Shanlin (5 February 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.jove.com\/t\/57045\/color-spot-test-as-a-presumptive-tool-for-the-rapid-detection-of-synthetic-cathinones\" target=\"_blank\">\"Color Spot Test As a Presumptive Tool for the Rapid Detection of Synthetic Cathinones\"<\/a> (in en). <i>Journal of Visualized Experiments<\/i> (132): 57045. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3791%2F57045\" target=\"_blank\">10.3791\/57045<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1940-087X\" target=\"_blank\">1940-087X<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.pubmedcentral.gov\/articlerender.fcgi?tool=pmcentrez&artid=PMC5912360\" target=\"_blank\">PMC5912360<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/29443096\" target=\"_blank\">29443096<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.jove.com\/t\/57045\/color-spot-test-as-a-presumptive-tool-for-the-rapid-detection-of-synthetic-cathinones\" target=\"_blank\">https:\/\/www.jove.com\/t\/57045\/color-spot-test-as-a-presumptive-tool-for-the-rapid-detection-of-synthetic-cathinones<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Color+Spot+Test+As+a+Presumptive+Tool+for+the+Rapid+Detection+of+Synthetic+Cathinones&rft.jtitle=Journal+of+Visualized+Experiments&rft.aulast=Philp&rft.aufirst=Morgan&rft.au=Philp%2C%26%2332%3BMorgan&rft.au=Shimmon%2C%26%2332%3BRonald&rft.au=Tahtouh%2C%26%2332%3BMark&rft.au=Fu%2C%26%2332%3BShanlin&rft.date=5+February+2018&rft.issue=132&rft.pages=57045&rft_id=info:doi\/10.3791%2F57045&rft.issn=1940-087X&rft_id=info:pmc\/PMC5912360&rft_id=info:pmid\/29443096&rft_id=https%3A%2F%2Fwww.jove.com%2Ft%2F57045%2Fcolor-spot-test-as-a-presumptive-tool-for-the-rapid-detection-of-synthetic-cathinones&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:6-10\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:6_10-0\">10.0<\/a><\/sup> <sup><a href=\"#cite_ref-:6_10-1\">10.1<\/a><\/sup> <sup><a href=\"#cite_ref-:6_10-2\">10.2<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Lewis, Kenna; Wagner, Rebecca; Rodriguez\u2010Cruz, Sandra E.; Weaver, Michael J.; Dumke, Jonathan C. (1 January 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/1556-4029.14562\" target=\"_blank\">\"Validation of the 4\u2010aminophenol color test for the differentiation of marijuana\u2010type and hemp\u2010type cannabis\"<\/a> (in en). <i>Journal of Forensic Sciences<\/i> <b>66<\/b> (1): 285\u2013294. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1111%2F1556-4029.14562\" target=\"_blank\">10.1111\/1556-4029.14562<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0022-1198\" target=\"_blank\">0022-1198<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/1556-4029.14562\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/1556-4029.14562<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Validation+of+the+4%E2%80%90aminophenol+color+test+for+the+differentiation+of+marijuana%E2%80%90type+and+hemp%E2%80%90type+cannabis&rft.jtitle=Journal+of+Forensic+Sciences&rft.aulast=Lewis&rft.aufirst=Kenna&rft.au=Lewis%2C%26%2332%3BKenna&rft.au=Wagner%2C%26%2332%3BRebecca&rft.au=Rodriguez%E2%80%90Cruz%2C%26%2332%3BSandra+E.&rft.au=Weaver%2C%26%2332%3BMichael+J.&rft.au=Dumke%2C%26%2332%3BJonathan+C.&rft.date=1+January+2021&rft.volume=66&rft.issue=1&rft.pages=285%E2%80%93294&rft_id=info:doi\/10.1111%2F1556-4029.14562&rft.issn=0022-1198&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1111%2F1556-4029.14562&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:7-11\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:7_11-0\">11.0<\/a><\/sup> <sup><a href=\"#cite_ref-:7_11-1\">11.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Acosta, Alexander; Li, Li; Weaver, Mike; Capote, Ryan; Perr, Jeannette; Almirall, Jos\u00e9 (1 December 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170922000510\" target=\"_blank\">\"Validation of a combined Fast blue BB and 4-Aminophenol colorimetric test for indication of Hemp-type and Marijuana-type cannabis\"<\/a> (in en). <i>Forensic Chemistry<\/i> <b>31<\/b>: 100448. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.forc.2022.100448\" target=\"_blank\">10.1016\/j.forc.2022.100448<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170922000510\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170922000510<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Validation+of+a+combined+Fast+blue+BB+and+4-Aminophenol+colorimetric+test+for+indication+of+Hemp-type+and+Marijuana-type+cannabis&rft.jtitle=Forensic+Chemistry&rft.aulast=Acosta&rft.aufirst=Alexander&rft.au=Acosta%2C%26%2332%3BAlexander&rft.au=Li%2C%26%2332%3BLi&rft.au=Weaver%2C%26%2332%3BMike&rft.au=Capote%2C%26%2332%3BRyan&rft.au=Perr%2C%26%2332%3BJeannette&rft.au=Almirall%2C%26%2332%3BJos%C3%A9&rft.date=1+December+2022&rft.volume=31&rft.pages=100448&rft_id=info:doi\/10.1016%2Fj.forc.2022.100448&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2468170922000510&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:8-12\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:8_12-0\">12.0<\/a><\/sup> <sup><a href=\"#cite_ref-:8_12-1\">12.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Acosta, Alexander; Almirall, Jos\u00e9 (1 December 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170921000722\" target=\"_blank\">\"Differentiation between hemp-type and marijuana-type cannabis using the Fast Blue BB colorimetric test\"<\/a> (in en). <i>Forensic Chemistry<\/i> <b>26<\/b>: 100376. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.forc.2021.100376\" target=\"_blank\">10.1016\/j.forc.2021.100376<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170921000722\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170921000722<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Differentiation+between+hemp-type+and+marijuana-type+cannabis+using+the+Fast+Blue+BB+colorimetric+test&rft.jtitle=Forensic+Chemistry&rft.aulast=Acosta&rft.aufirst=Alexander&rft.au=Acosta%2C%26%2332%3BAlexander&rft.au=Almirall%2C%26%2332%3BJos%C3%A9&rft.date=1+December+2021&rft.volume=26&rft.pages=100376&rft_id=info:doi\/10.1016%2Fj.forc.2021.100376&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2468170921000722&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:9-13\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:9_13-0\">13.0<\/a><\/sup> <sup><a href=\"#cite_ref-:9_13-1\">13.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation web\">Forrester, D.E. (15 April 1997). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.proquest.com\/openview\/b2bc339ba2dc1b9f55ab83bb9a112ec9\/1?pq-origsite=gscholar&cbl=18750&diss=y\" target=\"_blank\">\"The Duquenois Color Test for Marijuana: Spectroscopic and Chemical Studies\"<\/a>. Georgetown University<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.proquest.com\/openview\/b2bc339ba2dc1b9f55ab83bb9a112ec9\/1?pq-origsite=gscholar&cbl=18750&diss=y\" target=\"_blank\">https:\/\/www.proquest.com\/openview\/b2bc339ba2dc1b9f55ab83bb9a112ec9\/1?pq-origsite=gscholar&cbl=18750&diss=y<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=The+Duquenois+Color+Test+for+Marijuana%3A+Spectroscopic+and+Chemical+Studies&rft.atitle=&rft.aulast=Forrester%2C+D.E.&rft.au=Forrester%2C+D.E.&rft.date=15+April+1997&rft.pub=Georgetown+University&rft_id=https%3A%2F%2Fwww.proquest.com%2Fopenview%2Fb2bc339ba2dc1b9f55ab83bb9a112ec9%2F1%3Fpq-origsite%3Dgscholar%26cbl%3D18750%26diss%3Dy&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-14\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-14\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Gabrielson, R.; Sanders, T. (7 July 2016). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.propublica.org\/article\/common-roadside-drug-test-routinely-produces-false-positives\" target=\"_blank\">\"Busted: Tens of thousands of people every year are sent to jail based on the results of a $2 roadside drug test. Widespread evidence shows that these tests routinely produce false positives. Why are police departments and prosecutors still using them?\"<\/a>. <i>ProPublica<\/i><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.propublica.org\/article\/common-roadside-drug-test-routinely-produces-false-positives\" target=\"_blank\">https:\/\/www.propublica.org\/article\/common-roadside-drug-test-routinely-produces-false-positives<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Busted%3A+Tens+of+thousands+of+people+every+year+are+sent+to+jail+based+on+the+results+of+a+%242+roadside+drug+test.+Widespread+evidence+shows+that+these+tests+routinely+produce+false+positives.+Why+are+police+departments+and+prosecutors+still+using+them%3F&rft.atitle=ProPublica&rft.aulast=Gabrielson%2C+R.%3B+Sanders%2C+T.&rft.au=Gabrielson%2C+R.%3B+Sanders%2C+T.&rft.date=7+July+2016&rft_id=https%3A%2F%2Fwww.propublica.org%2Farticle%2Fcommon-roadside-drug-test-routinely-produces-false-positives&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-15\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-15\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Sawler, Jason; Stout, Jake M.; Gardner, Kyle M.; Hudson, Darryl; Vidmar, John; Butler, Laura; Page, Jonathan E.; Myles, Sean (26 August 2015). Tinker, Nicholas A.. ed. <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/dx.plos.org\/10.1371\/journal.pone.0133292\" target=\"_blank\">\"The Genetic Structure of Marijuana and Hemp\"<\/a> (in en). <i>PLOS ONE<\/i> <b>10<\/b> (8): e0133292. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1371%2Fjournal.pone.0133292\" target=\"_blank\">10.1371\/journal.pone.0133292<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1932-6203\" target=\"_blank\">1932-6203<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.pubmedcentral.gov\/articlerender.fcgi?tool=pmcentrez&artid=PMC4550350\" target=\"_blank\">PMC4550350<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/26308334\" target=\"_blank\">26308334<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/dx.plos.org\/10.1371\/journal.pone.0133292\" target=\"_blank\">https:\/\/dx.plos.org\/10.1371\/journal.pone.0133292<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+Genetic+Structure+of+Marijuana+and+Hemp&rft.jtitle=PLOS+ONE&rft.aulast=Sawler&rft.aufirst=Jason&rft.au=Sawler%2C%26%2332%3BJason&rft.au=Stout%2C%26%2332%3BJake+M.&rft.au=Gardner%2C%26%2332%3BKyle+M.&rft.au=Hudson%2C%26%2332%3BDarryl&rft.au=Vidmar%2C%26%2332%3BJohn&rft.au=Butler%2C%26%2332%3BLaura&rft.au=Page%2C%26%2332%3BJonathan+E.&rft.au=Myles%2C%26%2332%3BSean&rft.date=26+August+2015&rft.volume=10&rft.issue=8&rft.pages=e0133292&rft_id=info:doi\/10.1371%2Fjournal.pone.0133292&rft.issn=1932-6203&rft_id=info:pmc\/PMC4550350&rft_id=info:pmid\/26308334&rft_id=https%3A%2F%2Fdx.plos.org%2F10.1371%2Fjournal.pone.0133292&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-16\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-16\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Roman, Madeline G.; Houston, Rachel (1 November 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1344622320300936\" target=\"_blank\">\"Investigation of chloroplast regions rps16 and clpP for determination of Cannabis sativa crop type and biogeographical origin\"<\/a> (in en). <i>Legal Medicine<\/i> <b>47<\/b>: 101759. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.legalmed.2020.101759\" target=\"_blank\">10.1016\/j.legalmed.2020.101759<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1344622320300936\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1344622320300936<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Investigation+of+chloroplast+regions+rps16+and+clpP+for+determination+of+Cannabis+sativa+crop+type+and+biogeographical+origin&rft.jtitle=Legal+Medicine&rft.aulast=Roman&rft.aufirst=Madeline+G.&rft.au=Roman%2C%26%2332%3BMadeline+G.&rft.au=Houston%2C%26%2332%3BRachel&rft.date=1+November+2020&rft.volume=47&rft.pages=101759&rft_id=info:doi\/10.1016%2Fj.legalmed.2020.101759&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS1344622320300936&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:10-17\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:10_17-0\">17.0<\/a><\/sup> <sup><a href=\"#cite_ref-:10_17-1\">17.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Schwabe, Anna L.; Hansen, Connor J.; Hyslop, Richard M.; McGlaughlin, Mitchell E. (29 September 2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpls.2021.675770\/full\" target=\"_blank\">\"Comparative Genetic Structure of Cannabis sativa Including Federally Produced, Wild Collected, and Cultivated Samples\"<\/a>. <i>Frontiers in Plant Science<\/i> <b>12<\/b>: 675770. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3389%2Ffpls.2021.675770\" target=\"_blank\">10.3389\/fpls.2021.675770<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1664-462X\" target=\"_blank\">1664-462X<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.pubmedcentral.gov\/articlerender.fcgi?tool=pmcentrez&artid=PMC8544287\" target=\"_blank\">PMC8544287<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/34707624\" target=\"_blank\">34707624<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpls.2021.675770\/full\" target=\"_blank\">https:\/\/www.frontiersin.org\/articles\/10.3389\/fpls.2021.675770\/full<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Comparative+Genetic+Structure+of+Cannabis+sativa+Including+Federally+Produced%2C+Wild+Collected%2C+and+Cultivated+Samples&rft.jtitle=Frontiers+in+Plant+Science&rft.aulast=Schwabe&rft.aufirst=Anna+L.&rft.au=Schwabe%2C%26%2332%3BAnna+L.&rft.au=Hansen%2C%26%2332%3BConnor+J.&rft.au=Hyslop%2C%26%2332%3BRichard+M.&rft.au=McGlaughlin%2C%26%2332%3BMitchell+E.&rft.date=29+September+2021&rft.volume=12&rft.pages=675770&rft_id=info:doi\/10.3389%2Ffpls.2021.675770&rft.issn=1664-462X&rft_id=info:pmc\/PMC8544287&rft_id=info:pmid\/34707624&rft_id=https%3A%2F%2Fwww.frontiersin.org%2Farticles%2F10.3389%2Ffpls.2021.675770%2Ffull&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-18\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-18\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Cody, Robert B.; Laram\u00e9e, James A.; Durst, H. Dupont (1 April 2005). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/ac050162j\" target=\"_blank\">\"Versatile New Ion Source for the Analysis of Materials in Open Air under Ambient Conditions\"<\/a> (in en). <i>Analytical Chemistry<\/i> <b>77<\/b> (8): 2297\u20132302. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1021%2Fac050162j\" target=\"_blank\">10.1021\/ac050162j<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0003-2700\" target=\"_blank\">0003-2700<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/ac050162j\" target=\"_blank\">https:\/\/pubs.acs.org\/doi\/10.1021\/ac050162j<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Versatile+New+Ion+Source+for+the+Analysis+of+Materials+in+Open+Air+under+Ambient+Conditions&rft.jtitle=Analytical+Chemistry&rft.aulast=Cody&rft.aufirst=Robert+B.&rft.au=Cody%2C%26%2332%3BRobert+B.&rft.au=Laram%C3%A9e%2C%26%2332%3BJames+A.&rft.au=Durst%2C%26%2332%3BH.+Dupont&rft.date=1+April+2005&rft.volume=77&rft.issue=8&rft.pages=2297%E2%80%932302&rft_id=info:doi\/10.1021%2Fac050162j&rft.issn=0003-2700&rft_id=https%3A%2F%2Fpubs.acs.org%2Fdoi%2F10.1021%2Fac050162j&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:11-19\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:11_19-0\">19.0<\/a><\/sup> <sup><a href=\"#cite_ref-:11_19-1\">19.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Chambers, Megan I.; Musah, Rabi A. (1 March 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170921000783\" target=\"_blank\">\"DART-HRMS as a triage approach for the rapid analysis of cannabinoid-infused edible matrices, personal-care products and Cannabis sativa hemp plant material\"<\/a> (in en). <i>Forensic Chemistry<\/i> <b>27<\/b>: 100382. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.forc.2021.100382\" target=\"_blank\">10.1016\/j.forc.2021.100382<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170921000783\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170921000783<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=DART-HRMS+as+a+triage+approach+for+the+rapid+analysis+of+cannabinoid-infused+edible+matrices%2C+personal-care+products+and+Cannabis+sativa+hemp+plant+material&rft.jtitle=Forensic+Chemistry&rft.aulast=Chambers&rft.aufirst=Megan+I.&rft.au=Chambers%2C%26%2332%3BMegan+I.&rft.au=Musah%2C%26%2332%3BRabi+A.&rft.date=1+March+2022&rft.volume=27&rft.pages=100382&rft_id=info:doi\/10.1016%2Fj.forc.2021.100382&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2468170921000783&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-20\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-20\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Rodriguez-Cruz, Sandra E. (15 January 2006). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/rcm.2267\" target=\"_blank\">\"Rapid analysis of controlled substances using desorption electrospray ionization mass spectrometry\"<\/a> (in en). <i>Rapid Communications in Mass Spectrometry<\/i> <b>20<\/b> (1): 53\u201360. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1002%2Frcm.2267\" target=\"_blank\">10.1002\/rcm.2267<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0951-4198\" target=\"_blank\">0951-4198<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/rcm.2267\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/rcm.2267<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Rapid+analysis+of+controlled+substances+using+desorption+electrospray+ionization+mass+spectrometry&rft.jtitle=Rapid+Communications+in+Mass+Spectrometry&rft.aulast=Rodriguez-Cruz&rft.aufirst=Sandra+E.&rft.au=Rodriguez-Cruz%2C%26%2332%3BSandra+E.&rft.date=15+January+2006&rft.volume=20&rft.issue=1&rft.pages=53%E2%80%9360&rft_id=info:doi\/10.1002%2Frcm.2267&rft.issn=0951-4198&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1002%2Frcm.2267&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:12-21\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:12_21-0\">21.0<\/a><\/sup> <sup><a href=\"#cite_ref-:12_21-1\">21.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Chambers, Megan I.; Musah, Rabi A. (1 May 2023). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S246817092300005X\" target=\"_blank\">\"DART-HRMS triage approach part 2 \u2013 Application to the detection of cannabinoids and terpenes in recreational Cannabis products\"<\/a> (in en). <i>Forensic Chemistry<\/i> <b>33<\/b>: 100469. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.forc.2023.100469\" target=\"_blank\">10.1016\/j.forc.2023.100469<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S246817092300005X\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S246817092300005X<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=DART-HRMS+triage+approach+part+2+%E2%80%93+Application+to+the+detection+of+cannabinoids+and+terpenes+in+recreational+Cannabis+products&rft.jtitle=Forensic+Chemistry&rft.aulast=Chambers&rft.aufirst=Megan+I.&rft.au=Chambers%2C%26%2332%3BMegan+I.&rft.au=Musah%2C%26%2332%3BRabi+A.&rft.date=1+May+2023&rft.volume=33&rft.pages=100469&rft_id=info:doi\/10.1016%2Fj.forc.2023.100469&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS246817092300005X&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-22\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-22\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Beyramysoltan, Samira; Abdul-Rahman, Nana-Hawwa; Musah, Rabi A. (1 November 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0039914019306290\" target=\"_blank\">\"Call it a \u201cnightshade\u201d\u2014A hierarchical classification approach to identification of hallucinogenic Solanaceae spp. using DART-HRMS-derived chemical signatures\"<\/a> (in en). <i>Talanta<\/i> <b>204<\/b>: 739\u2013746. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.talanta.2019.06.010\" target=\"_blank\">10.1016\/j.talanta.2019.06.010<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0039914019306290\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0039914019306290<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Call+it+a+%E2%80%9Cnightshade%E2%80%9D%E2%80%94A+hierarchical+classification+approach+to+identification+of+hallucinogenic+Solanaceae+spp.+using+DART-HRMS-derived+chemical+signatures&rft.jtitle=Talanta&rft.aulast=Beyramysoltan&rft.aufirst=Samira&rft.au=Beyramysoltan%2C%26%2332%3BSamira&rft.au=Abdul-Rahman%2C%26%2332%3BNana-Hawwa&rft.au=Musah%2C%26%2332%3BRabi+A.&rft.date=1+November+2019&rft.volume=204&rft.pages=739%E2%80%93746&rft_id=info:doi\/10.1016%2Fj.talanta.2019.06.010&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0039914019306290&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-23\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-23\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Appley, Meghan Grace; Beyramysoltan, Samira; Musah, Rabi Ann (24 September 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acsomega.9b02145\" target=\"_blank\">\"Random Forest Processing of Direct Analysis in Real-Time Mass Spectrometric Data Enables Species Identification of Psychoactive Plants from Their Headspace Chemical Signatures\"<\/a> (in en). <i>ACS Omega<\/i> <b>4<\/b> (13): 15636\u201315644. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1021%2Facsomega.9b02145\" target=\"_blank\">10.1021\/acsomega.9b02145<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2470-1343\" target=\"_blank\">2470-1343<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.pubmedcentral.gov\/articlerender.fcgi?tool=pmcentrez&artid=PMC6761758\" target=\"_blank\">PMC6761758<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/31572865\" target=\"_blank\">31572865<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acsomega.9b02145\" target=\"_blank\">https:\/\/pubs.acs.org\/doi\/10.1021\/acsomega.9b02145<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Random+Forest+Processing+of+Direct+Analysis+in+Real-Time+Mass+Spectrometric+Data+Enables+Species+Identification+of+Psychoactive+Plants+from+Their+Headspace+Chemical+Signatures&rft.jtitle=ACS+Omega&rft.aulast=Appley&rft.aufirst=Meghan+Grace&rft.au=Appley%2C%26%2332%3BMeghan+Grace&rft.au=Beyramysoltan%2C%26%2332%3BSamira&rft.au=Musah%2C%26%2332%3BRabi+Ann&rft.date=24+September+2019&rft.volume=4&rft.issue=13&rft.pages=15636%E2%80%9315644&rft_id=info:doi\/10.1021%2Facsomega.9b02145&rft.issn=2470-1343&rft_id=info:pmc\/PMC6761758&rft_id=info:pmid\/31572865&rft_id=https%3A%2F%2Fpubs.acs.org%2Fdoi%2F10.1021%2Facsomega.9b02145&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-24\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-24\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Dong, Wen; Liang, Jian; Barnett, Isabella; Kline, Paul C.; Altman, Elliot; Zhang, Mengliang (1 December 2019). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/s00216-019-02200-7\" target=\"_blank\">\"The classification of Cannabis hemp cultivars by thermal desorption direct analysis in real time mass spectrometry (TD-DART-MS) with chemometrics\"<\/a> (in en). <i>Analytical and Bioanalytical Chemistry<\/i> <b>411<\/b> (30): 8133\u20138142. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2Fs00216-019-02200-7\" target=\"_blank\">10.1007\/s00216-019-02200-7<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1618-2642\" target=\"_blank\">1618-2642<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/s00216-019-02200-7\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/s00216-019-02200-7<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+classification+of+Cannabis+hemp+cultivars+by+thermal+desorption+direct+analysis+in+real+time+mass+spectrometry+%28TD-DART-MS%29+with+chemometrics&rft.jtitle=Analytical+and+Bioanalytical+Chemistry&rft.aulast=Dong&rft.aufirst=Wen&rft.au=Dong%2C%26%2332%3BWen&rft.au=Liang%2C%26%2332%3BJian&rft.au=Barnett%2C%26%2332%3BIsabella&rft.au=Kline%2C%26%2332%3BPaul+C.&rft.au=Altman%2C%26%2332%3BElliot&rft.au=Zhang%2C%26%2332%3BMengliang&rft.date=1+December+2019&rft.volume=411&rft.issue=30&rft.pages=8133%E2%80%938142&rft_id=info:doi\/10.1007%2Fs00216-019-02200-7&rft.issn=1618-2642&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2Fs00216-019-02200-7&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-25\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-25\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation web\">Pieslak, J.R. (2021). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/hdl.handle.net\/2144\/43518\" target=\"_blank\">\"Analytical techniques for the differentiation of hemp and marijuana\"<\/a>. <i>OpenBU<\/i>. Boston University Libraries<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/hdl.handle.net\/2144\/43518\" target=\"_blank\">https:\/\/hdl.handle.net\/2144\/43518<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=Analytical+techniques+for+the+differentiation+of+hemp+and+marijuana&rft.atitle=OpenBU&rft.aulast=Pieslak%2C+J.R.&rft.au=Pieslak%2C+J.R.&rft.date=2021&rft.pub=Boston+University+Libraries&rft_id=https%3A%2F%2Fhdl.handle.net%2F2144%2F43518&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-26\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-26\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Jolliffe, Ian T.; Cadima, Jorge (13 April 2016). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/royalsocietypublishing.org\/doi\/10.1098\/rsta.2015.0202\" target=\"_blank\">\"Principal component analysis: a review and recent developments\"<\/a> (in en). <i>Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences<\/i> <b>374<\/b> (2065): 20150202. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1098%2Frsta.2015.0202\" target=\"_blank\">10.1098\/rsta.2015.0202<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1364-503X\" target=\"_blank\">1364-503X<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.pubmedcentral.gov\/articlerender.fcgi?tool=pmcentrez&artid=PMC4792409\" target=\"_blank\">PMC4792409<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/26953178\" target=\"_blank\">26953178<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/royalsocietypublishing.org\/doi\/10.1098\/rsta.2015.0202\" target=\"_blank\">https:\/\/royalsocietypublishing.org\/doi\/10.1098\/rsta.2015.0202<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Principal+component+analysis%3A+a+review+and+recent+developments&rft.jtitle=Philosophical+Transactions+of+the+Royal+Society+A%3A+Mathematical%2C+Physical+and+Engineering+Sciences&rft.aulast=Jolliffe&rft.aufirst=Ian+T.&rft.au=Jolliffe%2C%26%2332%3BIan+T.&rft.au=Cadima%2C%26%2332%3BJorge&rft.date=13+April+2016&rft.volume=374&rft.issue=2065&rft.pages=20150202&rft_id=info:doi\/10.1098%2Frsta.2015.0202&rft.issn=1364-503X&rft_id=info:pmc\/PMC4792409&rft_id=info:pmid\/26953178&rft_id=https%3A%2F%2Froyalsocietypublishing.org%2Fdoi%2F10.1098%2Frsta.2015.0202&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-27\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-27\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation book\">Sammut, Claude; Webb, Geoffrey I., eds. (2010) (in en). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/978-0-387-30164-8\" target=\"_blank\"><i>Encyclopedia of Machine Learning<\/i><\/a>. Boston, MA: Springer US. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2F978-0-387-30164-8\" target=\"_blank\">10.1007\/978-0-387-30164-8<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Book_Number\" data-key=\"f64947ba21e884434bd70e8d9e60bae6\">ISBN<\/a> 978-0-387-30768-8<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/978-0-387-30164-8\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/978-0-387-30164-8<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.btitle=Encyclopedia+of+Machine+Learning&rft.date=2010&rft.place=Boston%2C+MA&rft.pub=Springer+US&rft_id=info:doi\/10.1007%2F978-0-387-30164-8&rft.isbn=978-0-387-30768-8&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2F978-0-387-30164-8&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-28\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-28\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Lloyd, S. (1 March 1982). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/ieeexplore.ieee.org\/document\/1056489\/\" target=\"_blank\">\"Least squares quantization in PCM\"<\/a> (in en). <i>IEEE Transactions on Information Theory<\/i> <b>28<\/b> (2): 129\u2013137. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1109%2FTIT.1982.1056489\" target=\"_blank\">10.1109\/TIT.1982.1056489<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0018-9448\" target=\"_blank\">0018-9448<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/ieeexplore.ieee.org\/document\/1056489\/\" target=\"_blank\">http:\/\/ieeexplore.ieee.org\/document\/1056489\/<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Least+squares+quantization+in+PCM&rft.jtitle=IEEE+Transactions+on+Information+Theory&rft.aulast=Lloyd&rft.aufirst=S.&rft.au=Lloyd%2C%26%2332%3BS.&rft.date=1+March+1982&rft.volume=28&rft.issue=2&rft.pages=129%E2%80%93137&rft_id=info:doi\/10.1109%2FTIT.1982.1056489&rft.issn=0018-9448&rft_id=http%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F1056489%2F&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-29\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-29\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Liaw, A.; Wiener, M. (2002). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/cogns.northwestern.edu\/cbmg\/LiawAndWiener2002.pdf\" target=\"_blank\">\"Classification and Regression by randomForest\"<\/a> (PDF). <i>R News<\/i> <b>2<\/b> (3): 18\u201322. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1609-3631\" target=\"_blank\">1609-3631<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/cogns.northwestern.edu\/cbmg\/LiawAndWiener2002.pdf\" target=\"_blank\">https:\/\/cogns.northwestern.edu\/cbmg\/LiawAndWiener2002.pdf<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Classification+and+Regression+by+randomForest&rft.jtitle=R+News&rft.aulast=Liaw%2C+A.%3B+Wiener%2C+M.&rft.au=Liaw%2C+A.%3B+Wiener%2C+M.&rft.date=2002&rft.volume=2&rft.issue=3&rft.pages=18%E2%80%9322&rft.issn=1609-3631&rft_id=https%3A%2F%2Fcogns.northwestern.edu%2Fcbmg%2FLiawAndWiener2002.pdf&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-30\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-30\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Breiman, Leo (2001). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1023\/A:1010933404324\" target=\"_blank\">\"Random Forests\"<\/a>. <i>Machine Learning<\/i> <b>45<\/b> (1): 5\u201332. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1023%2FA%3A1010933404324\" target=\"_blank\">10.1023\/A:1010933404324<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1023\/A:1010933404324\" target=\"_blank\">http:\/\/link.springer.com\/10.1023\/A:1010933404324<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Random+Forests&rft.jtitle=Machine+Learning&rft.aulast=Breiman&rft.aufirst=Leo&rft.au=Breiman%2C%26%2332%3BLeo&rft.date=2001&rft.volume=45&rft.issue=1&rft.pages=5%E2%80%9332&rft_id=info:doi\/10.1023%2FA%3A1010933404324&rft_id=http%3A%2F%2Flink.springer.com%2F10.1023%2FA%3A1010933404324&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-31\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-31\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Beyramysoltan, Samira; Ventura, M\u00f3nica I.; Rosati, Jennifer Y.; Giffen-Lemieux, Justine E.; Musah, Rabi A. (7 April 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acs.analchem.0c00199\" target=\"_blank\">\"Identification of the Species Constituents of Maggot Populations Feeding on Decomposing Remains\u2014Facilitation of the Determination of Post Mortem Interval and Time Since Tissue Infestation through Application of Machine Learning and Direct Analysis in Real Time-Mass Spectrometry\"<\/a> (in en). <i>Analytical Chemistry<\/i> <b>92<\/b> (7): 5439\u20135446. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1021%2Facs.analchem.0c00199\" target=\"_blank\">10.1021\/acs.analchem.0c00199<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0003-2700\" target=\"_blank\">0003-2700<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acs.analchem.0c00199\" target=\"_blank\">https:\/\/pubs.acs.org\/doi\/10.1021\/acs.analchem.0c00199<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Identification+of+the+Species+Constituents+of+Maggot+Populations+Feeding+on+Decomposing+Remains%E2%80%94Facilitation+of+the+Determination+of+Post+Mortem+Interval+and+Time+Since+Tissue+Infestation+through+Application+of+Machine+Learning+and+Direct+Analysis+in+Real+Time-Mass+Spectrometry&rft.jtitle=Analytical+Chemistry&rft.aulast=Beyramysoltan&rft.aufirst=Samira&rft.au=Beyramysoltan%2C%26%2332%3BSamira&rft.au=Ventura%2C%26%2332%3BM%C3%B3nica+I.&rft.au=Rosati%2C%26%2332%3BJennifer+Y.&rft.au=Giffen-Lemieux%2C%26%2332%3BJustine+E.&rft.au=Musah%2C%26%2332%3BRabi+A.&rft.date=7+April+2020&rft.volume=92&rft.issue=7&rft.pages=5439%E2%80%935446&rft_id=info:doi\/10.1021%2Facs.analchem.0c00199&rft.issn=0003-2700&rft_id=https%3A%2F%2Fpubs.acs.org%2Fdoi%2F10.1021%2Facs.analchem.0c00199&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-32\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-32\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Vergara, Daniela; Bidwell, L. Cinnamon; Gaudino, Reggie; Torres, Anthony; Du, Gary; Ruthenburg, Travis C.; deCesare, Kymron; Land, Donald P. <i>et al.<\/i> (19 April 2017). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.nature.com\/articles\/srep46528\" target=\"_blank\">\"Compromised External Validity: Federally Produced Cannabis Does Not Reflect Legal Markets\"<\/a> (in en). <i>Scientific Reports<\/i> <b>7<\/b> (1): 46528. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1038%2Fsrep46528\" target=\"_blank\">10.1038\/srep46528<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2045-2322\" target=\"_blank\">2045-2322<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.pubmedcentral.gov\/articlerender.fcgi?tool=pmcentrez&artid=PMC5395929\" target=\"_blank\">PMC5395929<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/28422145\" target=\"_blank\">28422145<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.nature.com\/articles\/srep46528\" target=\"_blank\">https:\/\/www.nature.com\/articles\/srep46528<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Compromised+External+Validity%3A+Federally+Produced+Cannabis+Does+Not+Reflect+Legal+Markets&rft.jtitle=Scientific+Reports&rft.aulast=Vergara&rft.aufirst=Daniela&rft.au=Vergara%2C%26%2332%3BDaniela&rft.au=Bidwell%2C%26%2332%3BL.+Cinnamon&rft.au=Gaudino%2C%26%2332%3BReggie&rft.au=Torres%2C%26%2332%3BAnthony&rft.au=Du%2C%26%2332%3BGary&rft.au=Ruthenburg%2C%26%2332%3BTravis+C.&rft.au=deCesare%2C%26%2332%3BKymron&rft.au=Land%2C%26%2332%3BDonald+P.&rft.au=Hutchison%2C%26%2332%3BKent+E.&rft.date=19+April+2017&rft.volume=7&rft.issue=1&rft.pages=46528&rft_id=info:doi\/10.1038%2Fsrep46528&rft.issn=2045-2322&rft_id=info:pmc\/PMC5395929&rft_id=info:pmid\/28422145&rft_id=https%3A%2F%2Fwww.nature.com%2Farticles%2Fsrep46528&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:13-33\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:13_33-0\">33.0<\/a><\/sup> <sup><a href=\"#cite_ref-:13_33-1\">33.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Zeki\u010d, Jure; Kri\u017eman, Mitja (11 December 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.mdpi.com\/1420-3049\/25\/24\/5872\" target=\"_blank\">\"Development of Gas-Chromatographic Method for Simultaneous Determination of Cannabinoids and Terpenes in Hemp\"<\/a> (in en). <i>Molecules<\/i> <b>25<\/b> (24): 5872. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.3390%2Fmolecules25245872\" target=\"_blank\">10.3390\/molecules25245872<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1420-3049\" target=\"_blank\">1420-3049<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.pubmedcentral.gov\/articlerender.fcgi?tool=pmcentrez&artid=PMC7763075\" target=\"_blank\">PMC7763075<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/33322595\" target=\"_blank\">33322595<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.mdpi.com\/1420-3049\/25\/24\/5872\" target=\"_blank\">https:\/\/www.mdpi.com\/1420-3049\/25\/24\/5872<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Development+of+Gas-Chromatographic+Method+for+Simultaneous+Determination+of+Cannabinoids+and+Terpenes+in+Hemp&rft.jtitle=Molecules&rft.aulast=Zeki%C4%8D&rft.aufirst=Jure&rft.au=Zeki%C4%8D%2C%26%2332%3BJure&rft.au=Kri%C5%BEman%2C%26%2332%3BMitja&rft.date=11+December+2020&rft.volume=25&rft.issue=24&rft.pages=5872&rft_id=info:doi\/10.3390%2Fmolecules25245872&rft.issn=1420-3049&rft_id=info:pmc\/PMC7763075&rft_id=info:pmid\/33322595&rft_id=https%3A%2F%2Fwww.mdpi.com%2F1420-3049%2F25%2F24%2F5872&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-34\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-34\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Dussy, Franz E.; Hamberg, Cornelia; Luginb\u00fchl, Marco; Schwerzmann, Thomas; Briellmann, Thomas A. (1 April 2005). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0379073804003408\" target=\"_blank\">\"Isolation of \u03949-THCA-A from hemp and analytical aspects concerning the determination of \u03949-THC in cannabis products\"<\/a> (in en). <i>Forensic Science International<\/i> <b>149<\/b> (1): 3\u201310. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.forsciint.2004.05.015\" target=\"_blank\">10.1016\/j.forsciint.2004.05.015<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0379073804003408\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0379073804003408<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Isolation+of+%CE%949-THCA-A+from+hemp+and+analytical+aspects+concerning+the+determination+of+%CE%949-THC+in+cannabis+products&rft.jtitle=Forensic+Science+International&rft.aulast=Dussy&rft.aufirst=Franz+E.&rft.au=Dussy%2C%26%2332%3BFranz+E.&rft.au=Hamberg%2C%26%2332%3BCornelia&rft.au=Luginb%C3%BChl%2C%26%2332%3BMarco&rft.au=Schwerzmann%2C%26%2332%3BThomas&rft.au=Briellmann%2C%26%2332%3BThomas+A.&rft.date=1+April+2005&rft.volume=149&rft.issue=1&rft.pages=3%E2%80%9310&rft_id=info:doi\/10.1016%2Fj.forsciint.2004.05.015&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0379073804003408&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-35\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-35\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Fischedick, Justin; Van Der Kooy, Frank; Verpoorte, Robert (2010). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.jstage.jst.go.jp\/article\/cpb\/58\/2\/58_2_201\/_article\" target=\"_blank\">\"Cannabinoid Receptor 1 Binding Activity and Quantitative Analysis of Cannabis sativa L. Smoke and Vapor\"<\/a> (in en). <i>Chemical and Pharmaceutical Bulletin<\/i> <b>58<\/b> (2): 201\u2013207. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1248%2Fcpb.58.201\" target=\"_blank\">10.1248\/cpb.58.201<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0009-2363\" target=\"_blank\">0009-2363<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/www.jstage.jst.go.jp\/article\/cpb\/58\/2\/58_2_201\/_article\" target=\"_blank\">http:\/\/www.jstage.jst.go.jp\/article\/cpb\/58\/2\/58_2_201\/_article<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Cannabinoid+Receptor+1+Binding+Activity+and+Quantitative+Analysis+of+Cannabis+sativa+L.+Smoke+and+Vapor&rft.jtitle=Chemical+and+Pharmaceutical+Bulletin&rft.aulast=Fischedick&rft.aufirst=Justin&rft.au=Fischedick%2C%26%2332%3BJustin&rft.au=Van+Der+Kooy%2C%26%2332%3BFrank&rft.au=Verpoorte%2C%26%2332%3BRobert&rft.date=2010&rft.volume=58&rft.issue=2&rft.pages=201%E2%80%93207&rft_id=info:doi\/10.1248%2Fcpb.58.201&rft.issn=0009-2363&rft_id=http%3A%2F%2Fwww.jstage.jst.go.jp%2Farticle%2Fcpb%2F58%2F2%2F58_2_201%2F_article&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:14-36\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:14_36-0\">36.0<\/a><\/sup> <sup><a href=\"#cite_ref-:14_36-1\">36.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Hazekamp, Arno; Simons, Ruud; Peltenburg\u2010Looman, Anja; Sengers, Melvin; van Zweden, Rianne; Verpoorte, Robert (1 January 2004). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.tandfonline.com\/doi\/full\/10.1081\/JLC-200028170\" target=\"_blank\">\"Preparative Isolation of Cannabinoids from Cannabis sativa by Centrifugal Partition Chromatography\"<\/a> (in en). <i>Journal of Liquid Chromatography & Related Technologies<\/i> <b>27<\/b> (15): 2421\u20132439. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1081%2FJLC-200028170\" target=\"_blank\">10.1081\/JLC-200028170<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1082-6076\" target=\"_blank\">1082-6076<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.tandfonline.com\/doi\/full\/10.1081\/JLC-200028170\" target=\"_blank\">https:\/\/www.tandfonline.com\/doi\/full\/10.1081\/JLC-200028170<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Preparative+Isolation+of+Cannabinoids+from+Cannabis+sativa+by+Centrifugal+Partition+Chromatography&rft.jtitle=Journal+of+Liquid+Chromatography+%26+Related+Technologies&rft.aulast=Hazekamp&rft.aufirst=Arno&rft.au=Hazekamp%2C%26%2332%3BArno&rft.au=Simons%2C%26%2332%3BRuud&rft.au=Peltenburg%E2%80%90Looman%2C%26%2332%3BAnja&rft.au=Sengers%2C%26%2332%3BMelvin&rft.au=van+Zweden%2C%26%2332%3BRianne&rft.au=Verpoorte%2C%26%2332%3BRobert&rft.date=1+January+2004&rft.volume=27&rft.issue=15&rft.pages=2421%E2%80%932439&rft_id=info:doi\/10.1081%2FJLC-200028170&rft.issn=1082-6076&rft_id=https%3A%2F%2Fwww.tandfonline.com%2Fdoi%2Ffull%2F10.1081%2FJLC-200028170&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-37\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-37\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Hazekamp, A.; Fischedick, J. T. (1 July 2012). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/dta.407\" target=\"_blank\">\"Cannabis - from cultivar to chemovar: Towards a better definition of Cannabis potency\"<\/a> (in en). <i>Drug Testing and Analysis<\/i> <b>4<\/b> (7-8): 660\u2013667. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1002%2Fdta.407\" target=\"_blank\">10.1002\/dta.407<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/dta.407\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/dta.407<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Cannabis+-+from+cultivar+to+chemovar%3A+Towards+a+better+definition+of+Cannabis+potency&rft.jtitle=Drug+Testing+and+Analysis&rft.aulast=Hazekamp&rft.aufirst=A.&rft.au=Hazekamp%2C%26%2332%3BA.&rft.au=Fischedick%2C%26%2332%3BJ.+T.&rft.date=1+July+2012&rft.volume=4&rft.issue=7-8&rft.pages=660%E2%80%93667&rft_id=info:doi\/10.1002%2Fdta.407&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1002%2Fdta.407&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-38\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-38\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Hazekamp, Arno; Peltenburg, Anja; Verpoorte, Rob; Giroud, Christian (1 September 2005). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/10826070500187558\" target=\"_blank\">\"Chromatographic and Spectroscopic Data of Cannabinoids from Cannabis sativa L.\"<\/a> (in en). <i>Journal of Liquid Chromatography & Related Technologies<\/i> <b>28<\/b> (15): 2361\u20132382. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1080%2F10826070500187558\" target=\"_blank\">10.1080\/10826070500187558<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1082-6076\" target=\"_blank\">1082-6076<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/10826070500187558\" target=\"_blank\">https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/10826070500187558<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Chromatographic+and+Spectroscopic+Data+of+Cannabinoids+from+Cannabis+sativa+L.&rft.jtitle=Journal+of+Liquid+Chromatography+%26+Related+Technologies&rft.aulast=Hazekamp&rft.aufirst=Arno&rft.au=Hazekamp%2C%26%2332%3BArno&rft.au=Peltenburg%2C%26%2332%3BAnja&rft.au=Verpoorte%2C%26%2332%3BRob&rft.au=Giroud%2C%26%2332%3BChristian&rft.date=1+September+2005&rft.volume=28&rft.issue=15&rft.pages=2361%E2%80%932382&rft_id=info:doi\/10.1080%2F10826070500187558&rft.issn=1082-6076&rft_id=https%3A%2F%2Fwww.tandfonline.com%2Fdoi%2Ffull%2F10.1080%2F10826070500187558&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-39\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-39\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Namdar, Dvory; Mazuz, Moran; Ion, Aurel; Koltai, Hinanit (1 March 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S092666901830061X\" target=\"_blank\">\"Variation in the compositions of cannabinoid and terpenoids in Cannabis sativa derived from inflorescence position along the stem and extraction methods\"<\/a> (in en). <i>Industrial Crops and Products<\/i> <b>113<\/b>: 376\u2013382. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.indcrop.2018.01.060\" target=\"_blank\">10.1016\/j.indcrop.2018.01.060<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S092666901830061X\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S092666901830061X<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Variation+in+the+compositions+of+cannabinoid+and+terpenoids+in+Cannabis+sativa+derived+from+inflorescence+position+along+the+stem+and+extraction+methods&rft.jtitle=Industrial+Crops+and+Products&rft.aulast=Namdar&rft.aufirst=Dvory&rft.au=Namdar%2C%26%2332%3BDvory&rft.au=Mazuz%2C%26%2332%3BMoran&rft.au=Ion%2C%26%2332%3BAurel&rft.au=Koltai%2C%26%2332%3BHinanit&rft.date=1+March+2018&rft.volume=113&rft.pages=376%E2%80%93382&rft_id=info:doi\/10.1016%2Fj.indcrop.2018.01.060&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS092666901830061X&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-40\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-40\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Namdar, Dvory; Charuvi, Dana; Ajjampura, Vinayka; Mazuz, Moran; Ion, Aurel; Kamara, Itzhak; Koltai, Hinanit (1 June 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0926669019301086\" target=\"_blank\">\"LED lighting affects the composition and biological activity of Cannabis sativa secondary metabolites\"<\/a> (in en). <i>Industrial Crops and Products<\/i> <b>132<\/b>: 177\u2013185. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.indcrop.2019.02.016\" target=\"_blank\">10.1016\/j.indcrop.2019.02.016<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0926669019301086\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0926669019301086<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=LED+lighting+affects+the+composition+and+biological+activity+of+Cannabis+sativa+secondary+metabolites&rft.jtitle=Industrial+Crops+and+Products&rft.aulast=Namdar&rft.aufirst=Dvory&rft.au=Namdar%2C%26%2332%3BDvory&rft.au=Charuvi%2C%26%2332%3BDana&rft.au=Ajjampura%2C%26%2332%3BVinayka&rft.au=Mazuz%2C%26%2332%3BMoran&rft.au=Ion%2C%26%2332%3BAurel&rft.au=Kamara%2C%26%2332%3BItzhak&rft.au=Koltai%2C%26%2332%3BHinanit&rft.date=1+June+2019&rft.volume=132&rft.pages=177%E2%80%93185&rft_id=info:doi\/10.1016%2Fj.indcrop.2019.02.016&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0926669019301086&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-41\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-41\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Omar, Jone; Olivares, Maitane; Alzaga, Mikel; Etxebarria, Nestor (1 April 2013). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/jssc.201201103\" target=\"_blank\">\"Optimisation and characterisation of marihuana extracts obtained by supercritical fluid extraction and focused ultrasound extraction and retention time locking GC-MS: Gas Chromatography\"<\/a> (in en). <i>Journal of Separation Science<\/i> <b>36<\/b> (8): 1397\u20131404. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1002%2Fjssc.201201103\" target=\"_blank\">10.1002\/jssc.201201103<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/jssc.201201103\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/jssc.201201103<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Optimisation+and+characterisation+of+marihuana+extracts+obtained+by+supercritical+fluid+extraction+and+focused+ultrasound+extraction+and+retention+time+locking+GC-MS%3A+Gas+Chromatography&rft.jtitle=Journal+of+Separation+Science&rft.aulast=Omar&rft.aufirst=Jone&rft.au=Omar%2C%26%2332%3BJone&rft.au=Olivares%2C%26%2332%3BMaitane&rft.au=Alzaga%2C%26%2332%3BMikel&rft.au=Etxebarria%2C%26%2332%3BNestor&rft.date=1+April+2013&rft.volume=36&rft.issue=8&rft.pages=1397%E2%80%931404&rft_id=info:doi\/10.1002%2Fjssc.201201103&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1002%2Fjssc.201201103&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-42\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-42\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Knight, Glenys; Hansen, Sean; Connor, Mark; Poulsen, Helen; McGovern, Catherine; Stacey, Janet (1 October 2010). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0379073810001969\" target=\"_blank\">\"The results of an experimental indoor hydroponic Cannabis growing study, using the \u2018Screen of Green\u2019 (ScrOG) method\u2014Yield, tetrahydrocannabinol (THC) and DNA analysis\"<\/a> (in en). <i>Forensic Science International<\/i> <b>202<\/b> (1-3): 36\u201344. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.forsciint.2010.04.022\" target=\"_blank\">10.1016\/j.forsciint.2010.04.022<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0379073810001969\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0379073810001969<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+results+of+an+experimental+indoor+hydroponic+Cannabis+growing+study%2C+using+the+%E2%80%98Screen+of+Green%E2%80%99+%28ScrOG%29+method%E2%80%94Yield%2C+tetrahydrocannabinol+%28THC%29+and+DNA+analysis&rft.jtitle=Forensic+Science+International&rft.aulast=Knight&rft.aufirst=Glenys&rft.au=Knight%2C%26%2332%3BGlenys&rft.au=Hansen%2C%26%2332%3BSean&rft.au=Connor%2C%26%2332%3BMark&rft.au=Poulsen%2C%26%2332%3BHelen&rft.au=McGovern%2C%26%2332%3BCatherine&rft.au=Stacey%2C%26%2332%3BJanet&rft.date=1+October+2010&rft.volume=202&rft.issue=1-3&rft.pages=36%E2%80%9344&rft_id=info:doi\/10.1016%2Fj.forsciint.2010.04.022&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0379073810001969&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-43\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-43\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Leghissa, Allegra; Smuts, Jonathan; Qiu, Changling; Hildenbrand, Zacariah L.; Schug, Kevin A. (1 January 2018). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/sscp.201700005\" target=\"_blank\">\"Detection of cannabinoids and cannabinoid metabolites using gas chromatography with vacuum ultraviolet spectroscopy\"<\/a> (in en). <i>Separation Science Plus<\/i> <b>1<\/b> (1): 37\u201342. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1002%2Fsscp.201700005\" target=\"_blank\">10.1002\/sscp.201700005<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/sscp.201700005\" target=\"_blank\">https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/sscp.201700005<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Detection+of+cannabinoids+and+cannabinoid+metabolites+using+gas+chromatography+with+vacuum+ultraviolet+spectroscopy&rft.jtitle=Separation+Science+Plus&rft.aulast=Leghissa&rft.aufirst=Allegra&rft.au=Leghissa%2C%26%2332%3BAllegra&rft.au=Smuts%2C%26%2332%3BJonathan&rft.au=Qiu%2C%26%2332%3BChangling&rft.au=Hildenbrand%2C%26%2332%3BZacariah+L.&rft.au=Schug%2C%26%2332%3BKevin+A.&rft.date=1+January+2018&rft.volume=1&rft.issue=1&rft.pages=37%E2%80%9342&rft_id=info:doi\/10.1002%2Fsscp.201700005&rft_id=https%3A%2F%2Fonlinelibrary.wiley.com%2Fdoi%2F10.1002%2Fsscp.201700005&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-44\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-44\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Gr\u00f6ger, Th.; Sch\u00e4ffer, M.; P\u00fctz, M.; Ahrens, B.; Drew, K.; Eschner, M.; Zimmermann, R. (1 July 2008). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0021967308008297\" target=\"_blank\">\"Application of two-dimensional gas chromatography combined with pixel-based chemometric processing for the chemical profiling of illicit drug samples\"<\/a> (in en). <i>Journal of Chromatography A<\/i> <b>1200<\/b> (1): 8\u201316. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.chroma.2008.05.028\" target=\"_blank\">10.1016\/j.chroma.2008.05.028<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0021967308008297\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0021967308008297<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Application+of+two-dimensional+gas+chromatography+combined+with+pixel-based+chemometric+processing+for+the+chemical+profiling+of+illicit+drug+samples&rft.jtitle=Journal+of+Chromatography+A&rft.aulast=Gr%C3%B6ger&rft.aufirst=Th.&rft.au=Gr%C3%B6ger%2C%26%2332%3BTh.&rft.au=Sch%C3%A4ffer%2C%26%2332%3BM.&rft.au=P%C3%BCtz%2C%26%2332%3BM.&rft.au=Ahrens%2C%26%2332%3BB.&rft.au=Drew%2C%26%2332%3BK.&rft.au=Eschner%2C%26%2332%3BM.&rft.au=Zimmermann%2C%26%2332%3BR.&rft.date=1+July+2008&rft.volume=1200&rft.issue=1&rft.pages=8%E2%80%9316&rft_id=info:doi\/10.1016%2Fj.chroma.2008.05.028&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0021967308008297&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-45\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-45\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Omar, Jone; Olivares, Maitane; Amigo, Jos\u00e9 Manuel; Etxebarria, Nestor (1 April 2014). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0039914013010370\" target=\"_blank\">\"Resolution of co-eluting compounds of Cannabis Sativa in comprehensive two-dimensional gas chromatography\/mass spectrometry detection with Multivariate Curve Resolution-Alternating Least Squares\"<\/a> (in en). <i>Talanta<\/i> <b>121<\/b>: 273\u2013280. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.talanta.2013.12.044\" target=\"_blank\">10.1016\/j.talanta.2013.12.044<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0039914013010370\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0039914013010370<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Resolution+of+co-eluting+compounds+of+Cannabis+Sativa+in+comprehensive+two-dimensional+gas+chromatography%2Fmass+spectrometry+detection+with+Multivariate+Curve+Resolution-Alternating+Least+Squares&rft.jtitle=Talanta&rft.aulast=Omar&rft.aufirst=Jone&rft.au=Omar%2C%26%2332%3BJone&rft.au=Olivares%2C%26%2332%3BMaitane&rft.au=Amigo%2C%26%2332%3BJos%C3%A9+Manuel&rft.au=Etxebarria%2C%26%2332%3BNestor&rft.date=1+April+2014&rft.volume=121&rft.pages=273%E2%80%93280&rft_id=info:doi\/10.1016%2Fj.talanta.2013.12.044&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0039914013010370&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-46\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-46\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Sanchez, Lee; Filter, Conor; Baltensperger, David; Kurouski, Dmitry (2020). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/xlink.rsc.org\/?DOI=C9RA08225E\" target=\"_blank\">\"Confirmatory non-invasive and non-destructive differentiation between hemp and cannabis using a hand-held Raman spectrometer\"<\/a> (in en). <i>RSC Advances<\/i> <b>10<\/b> (6): 3212\u20133216. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1039%2FC9RA08225E\" target=\"_blank\">10.1039\/C9RA08225E<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/2046-2069\" target=\"_blank\">2046-2069<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Central\" data-key=\"c85bdffd69dd30e02024b9cc3d7679e2\">PMC<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.pubmedcentral.gov\/articlerender.fcgi?tool=pmcentrez&artid=PMC9048763\" target=\"_blank\">PMC9048763<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/PubMed_Identifier\" data-key=\"1d34e999f13d8801964a6b3e9d7b4e30\">PMID<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/35497720\" target=\"_blank\">35497720<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/xlink.rsc.org\/?DOI=C9RA08225E\" target=\"_blank\">http:\/\/xlink.rsc.org\/?DOI=C9RA08225E<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Confirmatory+non-invasive+and+non-destructive+differentiation+between+hemp+and+cannabis+using+a+hand-held+Raman+spectrometer&rft.jtitle=RSC+Advances&rft.aulast=Sanchez&rft.aufirst=Lee&rft.au=Sanchez%2C%26%2332%3BLee&rft.au=Filter%2C%26%2332%3BConor&rft.au=Baltensperger%2C%26%2332%3BDavid&rft.au=Kurouski%2C%26%2332%3BDmitry&rft.date=2020&rft.volume=10&rft.issue=6&rft.pages=3212%E2%80%933216&rft_id=info:doi\/10.1039%2FC9RA08225E&rft.issn=2046-2069&rft_id=info:pmc\/PMC9048763&rft_id=info:pmid\/35497720&rft_id=http%3A%2F%2Fxlink.rsc.org%2F%3FDOI%3DC9RA08225E&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-47\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-47\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Chen, Zewei; Harrington, Peter de Boves (19 November 2019). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acs.analchem.9b03290\" target=\"_blank\">\"Pipeline for High-Throughput Modeling of Marijuana and Hemp Extracts\"<\/a> (in en). <i>Analytical Chemistry<\/i> <b>91<\/b> (22): 14489\u201314497. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1021%2Facs.analchem.9b03290\" target=\"_blank\">10.1021\/acs.analchem.9b03290<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/0003-2700\" target=\"_blank\">0003-2700<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/pubs.acs.org\/doi\/10.1021\/acs.analchem.9b03290\" target=\"_blank\">https:\/\/pubs.acs.org\/doi\/10.1021\/acs.analchem.9b03290<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Pipeline+for+High-Throughput+Modeling+of+Marijuana+and+Hemp+Extracts&rft.jtitle=Analytical+Chemistry&rft.aulast=Chen&rft.aufirst=Zewei&rft.au=Chen%2C%26%2332%3BZewei&rft.au=Harrington%2C%26%2332%3BPeter+de+Boves&rft.date=19+November+2019&rft.volume=91&rft.issue=22&rft.pages=14489%E2%80%9314497&rft_id=info:doi\/10.1021%2Facs.analchem.9b03290&rft.issn=0003-2700&rft_id=https%3A%2F%2Fpubs.acs.org%2Fdoi%2F10.1021%2Facs.analchem.9b03290&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-48\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-48\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">dos Santos, Nayara A.; Souza, Lindamara M.; Domingos, Eloilson; Fran\u00e7a, Hildegardo S.; Lacerda, Valdemar; Beatriz, Adilson; Vaz, Boniek G.; Rodrigues, Rayza R.T. <i>et al.<\/i> (1 August 2016). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170916300297\" target=\"_blank\">\"Evaluating the selectivity of colorimetric test (Fast Blue BB salt) for the cannabinoids identification in marijuana street samples by UV\u2013Vis, TLC, ESI(+)FT-ICR MS and ESI(+)MS\/MS\"<\/a> (in en). <i>Forensic Chemistry<\/i> <b>1<\/b>: 13\u201321. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.forc.2016.07.001\" target=\"_blank\">10.1016\/j.forc.2016.07.001<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170916300297\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170916300297<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Evaluating+the+selectivity+of+colorimetric+test+%28Fast+Blue+BB+salt%29+for+the+cannabinoids+identification+in+marijuana+street+samples+by+UV%E2%80%93Vis%2C+TLC%2C+ESI%28%2B%29FT-ICR+MS+and+ESI%28%2B%29MS%2FMS&rft.jtitle=Forensic+Chemistry&rft.aulast=dos+Santos&rft.aufirst=Nayara+A.&rft.au=dos+Santos%2C%26%2332%3BNayara+A.&rft.au=Souza%2C%26%2332%3BLindamara+M.&rft.au=Domingos%2C%26%2332%3BEloilson&rft.au=Fran%C3%A7a%2C%26%2332%3BHildegardo+S.&rft.au=Lacerda%2C%26%2332%3BValdemar&rft.au=Beatriz%2C%26%2332%3BAdilson&rft.au=Vaz%2C%26%2332%3BBoniek+G.&rft.au=Rodrigues%2C%26%2332%3BRayza+R.T.&rft.au=Carvalho%2C%26%2332%3BVer%C3%B4nica+V.&rft.date=1+August+2016&rft.volume=1&rft.pages=13%E2%80%9321&rft_id=info:doi\/10.1016%2Fj.forc.2016.07.001&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2468170916300297&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-49\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-49\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Fran\u00e7a, Hildegardo S.; Acosta, Alexander; Jamal, Adeel; Romao, Wanderson; Mulloor, Jerome; Almirall, Jose R. (1 March 2020). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170919301092\" target=\"_blank\">\"Experimental and ab initio investigation of the products of reaction from \u03949-tetrahydrocannabinol (\u03949-THC) and the fast blue BB spot reagent in presumptive drug tests for cannabinoids\"<\/a> (in en). <i>Forensic Chemistry<\/i> <b>17<\/b>: 100212. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.forc.2019.100212\" target=\"_blank\">10.1016\/j.forc.2019.100212<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170919301092\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2468170919301092<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Experimental+and+ab+initio+investigation+of+the+products+of+reaction+from+%CE%949-tetrahydrocannabinol+%28%CE%949-THC%29+and+the+fast+blue+BB+spot+reagent+in+presumptive+drug+tests+for+cannabinoids&rft.jtitle=Forensic+Chemistry&rft.aulast=Fran%C3%A7a&rft.aufirst=Hildegardo+S.&rft.au=Fran%C3%A7a%2C%26%2332%3BHildegardo+S.&rft.au=Acosta%2C%26%2332%3BAlexander&rft.au=Jamal%2C%26%2332%3BAdeel&rft.au=Romao%2C%26%2332%3BWanderson&rft.au=Mulloor%2C%26%2332%3BJerome&rft.au=Almirall%2C%26%2332%3BJose+R.&rft.date=1+March+2020&rft.volume=17&rft.pages=100212&rft_id=info:doi\/10.1016%2Fj.forc.2019.100212&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2468170919301092&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-50\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-50\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Jacobs, Alexander D.; Steiner, Robert R. (1 June 2014). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0379073814000929\" target=\"_blank\">\"Detection of the Duquenois\u2013Levine chromophore in a marijuana sample\"<\/a> (in en). <i>Forensic Science International<\/i> <b>239<\/b>: 1\u20135. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1016%2Fj.forsciint.2014.02.031\" target=\"_blank\">10.1016\/j.forsciint.2014.02.031<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0379073814000929\" target=\"_blank\">https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0379073814000929<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Detection+of+the+Duquenois%E2%80%93Levine+chromophore+in+a+marijuana+sample&rft.jtitle=Forensic+Science+International&rft.aulast=Jacobs&rft.aufirst=Alexander+D.&rft.au=Jacobs%2C%26%2332%3BAlexander+D.&rft.au=Steiner%2C%26%2332%3BRobert+R.&rft.date=1+June+2014&rft.volume=239&rft.pages=1%E2%80%935&rft_id=info:doi\/10.1016%2Fj.forsciint.2014.02.031&rft_id=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0379073814000929&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-51\"><span class=\"mw-cite-backlink\"><a href=\"#cite_ref-51\">\u2191<\/a><\/span> <span class=\"reference-text\"><span class=\"citation Journal\">Watanabe, Kazuhito; Honda, Go; Miyagi, Takeaki; Kanai, Masataka; Usami, Noriyuki; Yamaori, Satoshi; Iwamuro, Yoshiaki; Chinaka, Satoshi <i>et al.<\/i> (1 January 2017). <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/link.springer.com\/10.1007\/s11419-016-0337-6\" target=\"_blank\">\"The Duquenois reaction revisited: mass spectrometric estimation of chromophore structures derived from major phytocannabinoids\"<\/a> (in en). <i>Forensic Toxicology<\/i> <b>35<\/b> (1): 185\u2013189. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/Digital_object_identifier\" data-key=\"ae6d69c760ab710abc2dd89f3937d2f4\">doi<\/a>:<a rel=\"external_link\" class=\"external text\" href=\"http:\/\/dx.doi.org\/10.1007%2Fs11419-016-0337-6\" target=\"_blank\">10.1007\/s11419-016-0337-6<\/a>. <a rel=\"nofollow\" class=\"external text wiki-link\" href=\"http:\/\/en.wikipedia.org\/wiki\/International_Standard_Serial_Number\" data-key=\"a5dec3e4d005e654c29ad167ab53f53a\">ISSN<\/a> <a rel=\"external_link\" class=\"external text\" href=\"http:\/\/www.worldcat.org\/issn\/1860-8965\" target=\"_blank\">1860-8965<\/a><span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"http:\/\/link.springer.com\/10.1007\/s11419-016-0337-6\" target=\"_blank\">http:\/\/link.springer.com\/10.1007\/s11419-016-0337-6<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+Duquenois+reaction+revisited%3A+mass+spectrometric+estimation+of+chromophore+structures+derived+from+major+phytocannabinoids&rft.jtitle=Forensic+Toxicology&rft.aulast=Watanabe&rft.aufirst=Kazuhito&rft.au=Watanabe%2C%26%2332%3BKazuhito&rft.au=Honda%2C%26%2332%3BGo&rft.au=Miyagi%2C%26%2332%3BTakeaki&rft.au=Kanai%2C%26%2332%3BMasataka&rft.au=Usami%2C%26%2332%3BNoriyuki&rft.au=Yamaori%2C%26%2332%3BSatoshi&rft.au=Iwamuro%2C%26%2332%3BYoshiaki&rft.au=Chinaka%2C%26%2332%3BSatoshi&rft.au=Aramaki%2C%26%2332%3BHironori&rft.date=1+January+2017&rft.volume=35&rft.issue=1&rft.pages=185%E2%80%93189&rft_id=info:doi\/10.1007%2Fs11419-016-0337-6&rft.issn=1860-8965&rft_id=http%3A%2F%2Flink.springer.com%2F10.1007%2Fs11419-016-0337-6&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<li id=\"cite_note-:15-52\"><span class=\"mw-cite-backlink\">\u2191 <sup><a href=\"#cite_ref-:15_52-0\">52.0<\/a><\/sup> <sup><a href=\"#cite_ref-:15_52-1\">52.1<\/a><\/sup><\/span> <span class=\"reference-text\"><span class=\"citation web\">Pingree, C. (1 November 2022). <a rel=\"external_link\" class=\"external text\" href=\"https:\/\/www.congress.gov\/bill\/117th-congress\/house-bill\/6645\" target=\"_blank\">\"H.R.6645 - Hemp Advancement Act of 2022\"<\/a>. <i>Congress.gov<\/i>. Library of Congress<span class=\"printonly\">. <a rel=\"external_link\" class=\"external free\" href=\"https:\/\/www.congress.gov\/bill\/117th-congress\/house-bill\/6645\" target=\"_blank\">https:\/\/www.congress.gov\/bill\/117th-congress\/house-bill\/6645<\/a><\/span>.<\/span><span class=\"Z3988\" title=\"ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.btitle=H.R.6645+-+Hemp+Advancement+Act+of+2022&rft.atitle=Congress.gov&rft.aulast=Pingree%2C+C.&rft.au=Pingree%2C+C.&rft.date=1+November+2022&rft.pub=Library+of+Congress&rft_id=https%3A%2F%2Fwww.congress.gov%2Fbill%2F117th-congress%2Fhouse-bill%2F6645&rfr_id=info:sid\/en.wikipedia.org:Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\"><span style=\"display: none;\"> <\/span><\/span>\n<\/span>\n<\/li>\n<\/ol><\/div><\/div>\n<h2><span class=\"mw-headline\" id=\"Notes\">Notes<\/span><\/h2>\n<p>This presentation is faithful to the original, with only a few minor changes to presentation. Some grammar and punctuation was cleaned up to improve readability. In some cases important information was missing from the references, and that information was added. The original lists references in alphabetical order; they are listed by order of appearance for this version, by design.\n<\/p>\n<!-- \nNewPP limit report\nCached time: 20231215073617\nCache expiry: 86400\nDynamic content: false\nComplications: []\nCPU time usage: 1.008 seconds\nReal time usage: 1.036 seconds\nPreprocessor visited node count: 52298\/1000000\nPost\u2010expand include size: 468443\/2097152 bytes\nTemplate argument size: 148601\/2097152 bytes\nHighest expansion depth: 25\/40\nExpensive parser function count: 0\/100\nUnstrip recursion depth: 0\/20\nUnstrip post\u2010expand size: 131757\/5000000 bytes\n-->\n<!--\nTransclusion expansion time report (%,ms,calls,template)\n100.00% 926.479 1 -total\n 90.59% 839.258 1 Template:Reflist\n 75.63% 700.663 52 Template:Citation\/core\n 69.33% 642.358 44 Template:Cite_journal\n 13.78% 127.712 51 Template:Date\n 10.05% 93.087 7 Template:Cite_web\n 8.65% 80.138 88 Template:Citation\/identifier\n 4.70% 43.541 1 Template:Infobox_journal_article\n 4.27% 39.554 1 Template:Infobox\n 3.31% 30.643 176 Template:Hide_in_print\n-->\n\n<!-- Saved in parser cache with key cannaqa_wiki:pcache:idhash:5996-0!canonical and timestamp 20231215073616 and revision id 18398. Serialized with JSON.\n -->\n<\/div><\/div><div class=\"printfooter\">Source: <a rel=\"external_link\" class=\"external\" href=\"https:\/\/www.cannaqa.wiki\/index.php?title=Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa\">https:\/\/www.cannaqa.wiki\/index.php?title=Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa<\/a><\/div>\n<!-- end content --><div class=\"visualClear\"><\/div><\/div><\/div><div class=\"visualClear\"><\/div><\/div><!-- end of the left (by default at least) column --><div class=\"visualClear\"><\/div><\/div>\n<\/body>","e023a20c9343c9211ed31c45f339ab22_images":["https:\/\/www.cannaqa.wiki\/images\/5\/53\/Fig1_Chambers_JofCannRes23_5.png","https:\/\/www.cannaqa.wiki\/images\/e\/e0\/Fig2_Chambers_JofCannRes23_5.png","https:\/\/www.cannaqa.wiki\/images\/e\/e0\/Fig3_Chambers_JofCannRes23_5.png","https:\/\/www.cannaqa.wiki\/images\/c\/ca\/Fig4_Chambers_JofCannRes23_5.png"],"e023a20c9343c9211ed31c45f339ab22_timestamp":1702682170,"6b03770f3ae1d0327e8eb69d5900fa48":{"type":"chapter","title":"1. Cannabis analysis and research","key":"6b03770f3ae1d0327e8eb69d5900fa48"}},"link":"https:\/\/www.limswiki.org\/index.php\/Book:LIMSjournal_-_Fall_2023","price_currency":"","price_amount":"","book_size":"","download_url":"https:\/\/www.limsforum.com?ebb_action=book_download&book_id=109321","language":"","cta_button_content":"","toc":[{"type":"chapter","name":"1. Cannabis analysis and research","id":"6b03770f3ae1d0327e8eb69d5900fa48","children":[{"type":"article","name":"Combined ambient ionization mass spectrometric and chemometric approach for the differentiation of hemp and marijuana varieties of Cannabis sativa (Chambers et al. 2023)","id":"e023a20c9343c9211ed31c45f339ab22","pageUrl":"https:\/\/www.cannaqa.wiki\/index.php?title=Journal:Combined_ambient_ionization_mass_spectrometric_and_chemometric_approach_for_the_differentiation_of_hemp_and_marijuana_varieties_of_Cannabis_sativa"},{"type":"article","name":"High levels of pesticides found in illicit cannabis inflorescence compared to licensed samples in Canadian study using expanded 327 pesticides multiresidue method (Gagnon et al. 2023)","id":"147ac42afe5c27b45d4ed44a531a6754","pageUrl":"https:\/\/www.cannaqa.wiki\/index.php?title=Journal:High_levels_of_pesticides_found_in_illicit_cannabis_inflorescence_compared_to_licensed_samples_in_Canadian_study_using_expanded_327_pesticides_multiresidue_method"},{"type":"article","name":"Cadmium bioconcentration and translocation potential in day-neutral and photoperiod-sensitive hemp grown hydroponically for the medicinal market (Marebesi et al. 2023)","id":"34fb9a0e0648cc8855dc5ea15c3c3a92","pageUrl":"https:\/\/www.cannaqa.wiki\/index.php?title=Journal:Cadmium_bioconcentration_and_translocation_potential_in_day-neutral_and_photoperiod-sensitive_hemp_grown_hydroponically_for_the_medicinal_market"}]},{"type":"chapter","name":"2. Health informatics","id":"e8741c9012e81ca525e09827f947d4d0","children":[{"type":"article","name":"Transforming healthcare analytics with FHIR: A framework for standardizing and analyzing clinical data (Ayaz et al. 2023)","id":"96ca1abdbcc7bf60389fe942b678382d","pageUrl":"https:\/\/www.limswiki.org\/index.php\/Journal:Transforming_healthcare_analytics_with_FHIR:_A_framework_for_standardizing_and_analyzing_clinical_data"},{"type":"article","name":"Guideline for software life cycle in health informatics (Hauschild et al. 2022)","id":"3a2816a67d7d45f854c1e2fb9ec00f31","pageUrl":"https:\/\/www.limswiki.org\/index.php\/Journal:Guideline_for_software_life_cycle_in_health_informatics"},{"type":"article","name":"FAIR Health Informatics: A health informatics framework for verifiable and explainable data analysis (Siddiqi et al. 2023)","id":"affac9ff82db9d386600be5eb3d77056","pageUrl":"https:\/\/www.limswiki.org\/index.php\/Journal:FAIR_Health_Informatics:_A_health_informatics_framework_for_verifiable_and_explainable_data_analysis"}]},{"type":"chapter","name":"3. Laboratory informatics and management","id":"6abf69c411fdbc28921e83caae4aa6b5","children":[{"type":"article","name":"Autonomous experimental systems in materials science (Ishizuki et al. 2023)","id":"537370f6a0e7e5345701b0b5a2fb2d41","pageUrl":"https:\/\/www.limswiki.org\/index.php\/Journal:Autonomous_experimental_systems_in_materials_science"},{"type":"article","name":"Establishing reliable research data management by integrating measurement devices utilizing intelligent digital twins (Lehmann et al. 2023)","id":"9ace6d7c38d417b5bea5133d24ffe1a9","pageUrl":"https:\/\/www.limswiki.org\/index.php\/Journal:Establishing_reliable_research_data_management_by_integrating_measurement_devices_utilizing_intelligent_digital_twins"},{"type":"article","name":"Laboratory automation, informatics, and artificial intelligence: Current and future perspectives in clinical microbiology (Mencacci et al. 2023)","id":"49083cfbd81897f5701da971794583e5","pageUrl":"https:\/\/www.limswiki.org\/index.php\/Journal:Laboratory_automation,_informatics,_and_artificial_intelligence:_Current_and_future_perspectives_in_clinical_microbiology"}]},{"type":"chapter","name":"4. Laboratory informatics applications","id":"f315a6366c26c252951edbc8987a4f81","children":[{"type":"article","name":"A web application to support the coordination of reflexive, interpretative toxicology testing (Pablo et al. 2023)","id":"b283f4f5c78061a91491bc16c8576d36","pageUrl":"https:\/\/www.limswiki.org\/index.php\/Journal:A_web_application_to_support_the_coordination_of_reflexive,_interpretative_toxicology_testing"},{"type":"article","name":"Development of an integrated and comprehensive clinical trial process management system (Shen et al. 2023)","id":"e65552a701c8075f24aa45db5f398e2e","pageUrl":"https:\/\/www.limswiki.org\/index.php\/Journal:Development_of_an_integrated_and_comprehensive_clinical_trial_process_management_system"},{"type":"article","name":"NIMS-OS: An automation software to implement a closed loop between artificial intelligence and robotic experiments in materials science (Tamura et al. 2023)","id":"9e1d6433c962801d5a75756c1046599c","pageUrl":"https:\/\/www.limswiki.org\/index.php\/Journal:NIMS-OS:_An_automation_software_to_implement_a_closed_loop_between_artificial_intelligence_and_robotic_experiments_in_materials_science"},{"type":"article","name":"Registered data-centered lab management system based on data ownership security architecture (Zheng et al. 2023)","id":"ab540fa1cc32f92043343fd9cf67ec0d","pageUrl":"https:\/\/www.limswiki.org\/index.php\/Journal:Registered_data-centered_lab_management_system_based_on_data_ownership_security_architecture"}]}],"settings":{"show_cover":0,"show_title":0,"show_subtitle":0,"show_full_title":0,"show_editor":0,"show_editor_pic":0,"show_publisher":0,"show_language":0,"show_size":0,"show_toc":0,"show_content_beneath_cover":0,"toc_links":"","cta_button":"","content_location":"","log_in_msg":"","cover_size":""},"title_image":"https:\/\/s3.limsforum.com\/www.limsforum.com\/wp-content\/uploads\/Fig1_Ishizuki_SciTechAdvMatMeth2023_3-1.jpeg"}}
LIMSjournal - Fall 2023
Volume 9, Issue 3
Editor: Shawn Douglas
Publisher: LabLynx Press
Copyright LabLynx Inc. All rights reserved.