The asterisk (*) denotes his corresponding authorship.
The idealism of a hemodynamic software
The complexity of hemodynamic models has prevented clinicians from getting the insights out of the models when relating the clinical issues with the hemodynamic model. Visualization is the most persuasive way to illustrate a hemodynamic equation, and simulation is needed to visualize how the equation changes upon the manipulation of the coefficient of equations. Thus, the success of the hemodynamic software depends on how easy it is to work with visualizing the hemodynamic model and how effective it is for clinicians to draw insights from the models.
Additionally, it would be better if the following conditions are fulfilled: -1) an engineer takes care of the CPU time and memory management when combining and implementing numerous hemodynamic models published so far; -2) the simulation software provides an alternative interface other than GUI, which could enable experts to work more flexibly with the hemodynamic model; -3) components such as device interface, medical statistics, and artificial intelligence are coherently integrated in order to facilitate hemodynamic research.
Infrastructural aspects of each component
Each component will be the basis upon which other components can be built. This circulative data flow in the architecture diagram will eventually contribute to the development of other components synergistically. In other words, when considering the final overall goal of this software project as facilitating the data flow according to the software architecture, one part of the development will benefit the other part of the research.
The hemodynamic workbench software will be implemented to provide the following infrastructural functionalities: (1) To receive signals from the hemodynamic instrument; (2) To extract necessary information by wavelet analyses; (3) To understand the data according to the hemodynamic model and simulation; (4) To provide medical statistics; (5) To perform an action by reinforcement of the learning process.
Why the thoracic cavity for hemodynamic software and robotic surgery?
The thoracic cavity is intriguing in regards to its demanding physiological and computational potential. It is physiologically intriguing how the lungs and the heart are directly governed by the laws of physics: the hemodynamics during blood circulation and respiration with relation to auscultation, electrocardiography, ECMO and anesthetic machines. Computationally, a kernel-level device driver and Bayesian-based machine learning algorithm can be employed for (1) monitoring of the states of the thoracic organs, (2) computer-assisted hemodynamic modeling and simulation, and (3) machine learning for information processing. In addition, the thoracic cavity is ideal for a specialty that sits on the cusp between surgery and engineering to perform intellectually and technically challenging surgical robotic R&D projects on the organs encased by bones, which are best accessed and manipulated by a thin robotic hand instrument with ergonomic advantages. This will widen the indication of robotic cardiovascular surgery with new surgical procedures that integrate various additional hemodynamic devices and computational support.
"Surgeons must progress beyond the traditional techniques of cutting and sewing that have been their province since surgeons were barbers to a future in which approaches involving minimal access to the abdominal cavity are only the beginning." - Pappas et al. (2004) N Engl J Med.
Device driver interface component will enable the software to access raw data directly from a device. Biomedical companies seem to welcome the idea of enabling third parties to write software for their devices, which is exemplified by 3M providing an SDK (Software Development Kit) to allow people to write software for its Bluetooth stethoscope. However, my ultimate goal will be to make one step further by implementing the kernel-level device driver that would connect devices more fundamentally (as compared to existing SDK) and, therefore, to establish an integrative and flexible hemodynamic workbench.
    Some EKG classification articles (Lee, 2013) (Lihuang, 2010) relied exclusively on the MIT-BIH arrhythmia database or the standard test material to evaluate their arrhythmia detection algorithms. However, to the best of our knowledge, the difficulty of acquiring additional new raw EKG dataset due to the absence of open-source device interface for EKG instrument may be at least partially attributed to those researchers's having to work exclusively on MIT-BIH arrhythmia database. Therefore, if this software can receive the EKG raw stream over a WiFi or USB connection from instruments, future engineers can acquire additional test materials by collecting further raw EKG data alongside with corresponding EKG diagnoses, directly.
    Nonetheless, companies would be cautious about opening their device protocols for my implementing the kernel-level device interface, since doing so might change the company's marketing strategies and policies. Therefore, continuous improvement of Project nGene.org® in the long-term to gain agreement concerning its clinical pragmatism and to embrace clinicians' needs by providing an easy-to-write environment for their own scripts will have to be prioritized over this kernel component.
"(2) Waveform Analyses" component pre-processes the raw wavelet data directly from the devices via the "(1) Device interface" component. In order to handle the raw wavelet dataset, such as EKG, lung and heart sounds, etc., two core algorithms have been chosen to be common denominating features: Independent Component Analysis (ICA) separates the mixed wavelets, whereas Support Vector Machine (SVM) classifies things after being trained.
    Its benefit can be illustrated by how this feature may change the existing flow. These machine-learning components can be used tentatively, until a more precise implementation of the classification for wavelets is implemented later in the point of time. For example, machine-learning algorithms for classifying EKG would be no match for a manually-written conditional statements implemented according to the Sokolow-Lyon Criteria for left ventricular hypertrophy (LVH) (Sokolow, 1949), as it would be nonsensical for training SVM to distinguish whether the summation of the S wave in V1 and the R wave in V5 or V6 is greater than, specifically, 35mm or not for LVH. However, until the manually-implemented code is developed according to certain criteria, it may be better to employ machine-learning features to accommodate wavelets in order to accelerate research and development in the meanwhile.
    For an example of embedding this software into the educational CPR kit mentioned above, the AED (Automated External Defibrillation) algorithm requires distinguishing normal EKG from various arrhythmia cases. However, since the MIT-BIH "arrhythmia" database does not have normal EKG dataset, the "(1) Device Interface" component can be used to collect a normal EKG raw dataset. Once normal EKG data with diagnoses are accumulated, then the SVM algorithm can be trained to classify whether it should be defibrillated, synchronized cardioversion, non-shockable, and normal, until the development of a more accurate manually-programmed classifying algorithm.
Project nGene.org® intends to facilitate research on the hemodynamic model, not only to better understand the physiology, and but also to gain further insights into improving the model. There are numerous equations published already and in the future and it may be too late if we just wait for the echocardiography manufacturing engineer to implement the module for the equation we need. Unless it is open-sourced, it cannot possibly follow the speed of insights during research. Yale Neuron is open-sourced with GUI for simulating neuron network; however, in my opinion, no matter how flexibly a software architect may implement its GUI, it cannot be on a par with the flexibility and creativity of new equations and insights of clinicians in the future.
    Therefore, Project nGene.org® tries to circumvent this problem by integrating R script so that clinicians can add their equations to test those features during echocardiographic measurements on the flies. At the same time, I believe that the success of earning popularity depends on how easy and generic it is for clinicians to add and modify the source code. Since clinicians do not have time to spend on learning, it is very important to make it very intuitive to make them willing to invest their time. I think that clinicians will invest their time only if they can get it intuitively.
(4) Medical Statistics & (5) Machine Learningindex
"(4) Medical Statistics" is something that I do, not as a destination, but as a necessary step. To put it straightforwardly, the ultimate goal is "(5) Machine Learning". "(5) Machine Learning" component is pushed back on the priority list in the Masterplan Chart, because the software is designed to provide the following different types of dataset for the machine-learning algorithms: (i) Directly from hardware via the kernel program part, "(1) Device Interface"; (ii) Indirectly processing the wavelets raw data from instruments, "(2) Waveform Analyses"; (iii) Parsing and processing articles, especially meta-analysis and survival curve data, "(4) Medical Statistics", via a semantic web.
    The semantic web is a very suitable piece for medicine due to several reasons: (1) It is very flexible to integrate other semantic webs together, such that it can be used as a knowledge database with numerical information. (2) This numerical information with a network form can be fed into Bayesian-based machine learning. (3) Meta-Analysis is one of the forms of very specialized information that are available in the domain of medicine, and getting the hazard ratio from the survival curve for meta-analysis was, in my opinion, the most difficult methodology and the most challenging technical barrier when building a semantic web database.
Software Architecture (The 2024 Edition)
As both a medical doctor and a software engineer, with experience in echocardiography and serving as an IRB chair, I bring a unique, chimeric perspective to the development of Project nGene.org®. This dual expertise is crucial in navigating the challenges outlined in three seminal works: The Mythical Man-Month, The Innovator's Prescription, and Crossing the Chasm.
The Mythical Man-Month: In the interdisciplinary world of software and medicine, I have learned that communication is key to bridging the gap between different fields—what I call the "Apple and Orange" problem. This lesson was driven home by my experiences and reinforced by Fred Brooks' The Mythical Man-Month. Brooks warns that simply adding more manpower to a project often increases complexity rather than reducing it. As a chimera, trained in both fields, I strive to minimize this intercommunication complexity, ensuring that the app remains manageable and effective without the need to constantly increase resources.
The Innovator's Prescription: The Project nGene.org app is not designed to guarantee perfect accuracy in recognizing visual or auditory data through its camera or microphone. Instead, drawing from The Innovator's Prescription, the app's primary objective is to disrupt traditional clinical workflows by simplifying and democratizing complex medical processes. My goal is to enhance the clinical experience, making it more efficient and cost-effective, while keeping the app accessible to a broader audience. Additionally, by making parts of the codebase open-source, we are fostering a collaborative environment that invites continuous innovation and improvement.
Crossing the Chasm: Finally, in alignment with Geoffrey Moore's Crossing the Chasm, this app is strategically focused on identifying and capturing its niche market within the healthcare industry. By targeting a specific segment that values innovation, efficiency, and cost-effectiveness, the app aims to establish a strong foothold and gradually expand its user base. I am committed to ensuring that the app not only provides core technology but also offers a comprehensive ecosystem of support and services. This approach ensures seamless integration into existing clinical workflows, addressing the pragmatic needs of a broader user group and facilitating the app's transition from early adopters to the early majority.
The software project is meticulously crafted, with each component acting as a foundational pillar for subsequent innovations, establishing a circular data flow within its architectural framework. This methodology is anticipated to synergistically propel the evolution of the platform's elements. The project's paramount objective is to refine data circulation to mirror its architectural blueprint, ensuring that progress in one domain reciprocally amplifies research endeavors across the board. The hemodynamic workbench software is poised to offer essential functionalities: (1) capturing signals from hemodynamic instruments, (2) distilling vital information via wavelet analyses, (3) decoding data through hemodynamic models and simulations, (4) compiling medical statistics, and (5) executing actions based on a reinforcement learning algorithm.
Implementing the software marks the recrystallization of his professional journey, serving as a compass to navigate his career. This endeavor will not only guide him towards new horizons but also enrich his understanding for further development, ultimately fulfilling his life's purpose and enhancing his sense of satisfaction.
Why the thoracic cavity for hemodynamic software and robotic surgery?
The thoracic cavity, encasing critical organs such as the heart and lungs, presents a unique intersection of physiology and technology, demonstrating the profound influence of physical laws on biological functions. From a computational perspective, the integration of kernel-level device drivers with machine learning algorithms offers transformative potential in thoracic medicine. These technologies enable continuous monitoring of thoracic organ states through advanced waveform analyses, including ECG and ventilation monitoring waveforms (pressure, flow, volume), and auscultated mixed heart and lung sounds. Such detailed data acquisition is crucial for effective decision-making and patient management in real-time scenarios. The computational modeling capabilities, particularly in hemodynamic simulations, are further enhanced by incorporating echocardiography data. This integration is especially pivotal in addressing complex conditions like pulmonary hypertension, where accurate hemodynamic models can significantly improve the outcomes of interventions such as congenital heart defect surgeries in neonates. By simulating various physiological conditions, surgeons and clinicians can predict the effects of surgical interventions, thereby planning surgeries with higher precision and better prognostic outcomes. Moreover, the field of robotic surgery in the thoracic cavity is advancing rapidly, driven by machine learning algorithms that learn from thousands of surgeries performed by human doctors. This data not only informs the development of autonomous surgical robots but also supports the creation of new surgical techniques that integrate hemodynamic devices and computational support. The advent of slender robotic hand instruments designed specifically for the ergonomic constraints of thoracic surgery further underscores the technical sophistication in this field.
"Surgeons must progress beyond the traditional techniques of cutting and sewing that have been their province since surgeons were barbers to a future in which approaches involving minimal access to the abdominal cavity are only the beginning." - Pappas et al. (2004) N Engl J Med.
The integration of computational modeling and simulation has revolutionized the field of hemodynamics, transforming the way cardiovascular conditions are studied and treated. The dynamic and interactive nature of hemodynamic simulations, as discussed in "Computational Thinking" by Peter J. Denning and Matti Tedre, goes beyond the capabilities of traditional graph drawing, which often falls short when dealing with the complex, variable nature of biological systems. Unlike static graphs that display a fixed dataset, simulations provide a real-time, interactive platform that allows researchers to modify parameters and observe how these changes affect the cardiovascular system. This interactivity is crucial for a detailed understanding of how blood flow and pressure react to various physiological changes, making simulations an indispensable tool in predicting the effects of alterations within the cardiovascular system and aiding in the development of effective treatments for heart diseases.
Advanced modeling and simulation techniques are particularly impactful in addressing the challenges of congenital heart defects (CHD) and pulmonary arterial hypertension (PAH). For instance, the development of logistic-based equations for estimating Pulmonary Artery Pressure (PAP), as noted in Project nGene.org®, underscores the practical application of theoretical models in a clinical setting. These simulations enable the visualization and analysis of cardiovascular responses to treatments in a risk-free environment, which is especially crucial in designing interventions for vulnerable populations such as neonates with CHD. The traditional approach to surgical interventions, fraught with significant risks, highlights the need for non-invasive methods facilitated by simulations. By simulating specific cardiovascular conditions associated with CHD and PAH, Project nGene.org® not only provides insights into the intricate factors influencing patient outcomes but also enhances the potential for successful treatments while minimizing risks.
The ongoing initiative to harness hemodynamic modeling and simulation in the development of neonatal CHD surgery simulations exemplifies the shift towards simulation-based planning and execution of surgical interventions. This approach not only refines the understanding and management of PAH within the context of CHD but also pioneers new methodologies for surgical planning. By creating highly accurate, virtual models where surgical strategies can be tested and refined, simulations ensure the highest level of safety and efficacy in neonatal CHD treatments.
(4) Medical Statistics & (5) Machine Learningindex
Integrating "(4) Medical Statistics" into my work is not merely a destination but a vital step towards a broader objective: mastering "(5) Machine Learning". This component is strategically deferred in the Masterplan Chart, as the software is intricately designed to curate diverse datasets for machine learning algorithms through various means: (i) directly from hardware via the kernel in the "(1) Device Interface"; (ii) by processing raw wavelet data from instruments in "(2) Waveform Analyses"; and (iii) by parsing and analyzing medical literature, particularly meta-analyses and survival curve data, through "(4) Medical Statistics", utilizing a semantic web (or Web 3.0) approach. Initially, the semantic web seemed perfectly aligned with medical applications for several reasons: (1) Its inherent flexibility facilitates the integration of multiple semantic webs, creating a comprehensive knowledge database enriched with numerical data. (2) This numerically dense network is ideal for Bayesian-based machine learning applications. (3) Specifically, meta-analysis represents a form of highly specialized information within the medical domain, where deriving hazard ratios from survival curves posed a significant technical challenge and a methodological bottleneck in developing a semantic web database.
However, the rapid evolution of machine learning algorithms necessitated a shift in methodological approach. Acknowledging the advancements in deep neural networks and linear algebra techniques, especially Singular Value Decomposition (SVD), these methods now appear more apt for these objectives. This change in methodology is driven by the emerging efficiencies and capabilities of these algorithms in machine learning, signifying a pivotal adaptation to the evolving landscape of data analysis. This recalibration of approach, moving from a Bayesian-based semantic web to emphasizing deep learning and SVD, reflects a commitment to leveraging the most effective and advanced methodologies available in the field of machine learning. It underlines readiness to adapt and evolve in response to the dynamic nature of technological advancement and the continuous quest for more refined and powerful analytical tools.
The reconsideration of Bayesian algorithms also draws from a historical challenge in the field of artificial intelligence. Despite the Bayesian approach's flexibility and appeal, its application is marred by complexity in calculations beyond simple, restrictive assumptions. This complexity often necessitates approximation methods or sampling, which, while practical, diverge from dealing with the real posterior distribution directly. Further complicating the landscape was the neural network's initial inability to solve the exclusive OR (XOR) problem, a straightforward task achievable with basic digital logic gates but unattainable by a single-layer perceptron. Although it was known that multi-layer perceptrons could theoretically execute such tasks, the lack of effective training methods led to significant disillusionment and a temporary retreat from neural network research. This historical bottleneck highlights the limitations of early machine learning approaches and underlines the strategic pivot towards more advanced and capable methodologies, such as deep learning, that have since overcome these early challenges. (On February 5th, 2024, this segment of the software architecture underwent a revision to include sophisticated deep learning and SVD techniques.)
Robotic Surgery
RCT Meta-analysis: Robotic vs. Laparoscopic Surgery (Frank, 2018)
Importance This review provides a comprehensive comparison of treatment outcomes between robot- assisted laparoscopic surgery (RLS) and conventional laparoscopic surgery (CLS) based on randomly-controlled trials (RCTs).
Objectives We employed RCTs to provide a systematic review that will enable the relevant community to weigh the effectiveness and efficacy of surgical robotics in controversial fields on surgical procedures both overall and on each individual surgical procedure.
Evidence review A search was conducted for RCTs in PubMed, EMBASE, and Cochrane databases from 1981 to 2016. Among a total of 1,517 articles, 27 clinical reports with a mean sample size of 65 patients per report (32.7 patients who underwent RLS and 32.5 who underwent CLS), met the inclusion criteria.
Findings RLS shows significant advantages in total operative time, net operative time, total complica- tion rate, and operative cost (p < 0.05 in all cases), whereas the estimated blood loss was less in RLS (p < 0.05). As subgroup analyses, conversion rate on colectomy and length of hospital stay on hysterectomy statistically favors RLS (p < 0.05).
Conclusions Despite higher operative cost, RLS does not result in statistically better treatment outcomes, with the exception of lower estimated blood loss. Operative time and total complication rate are significantly more favorable with CLS.
Robotic surgery cost, under the hood
Regarding the cost-effectiveness of robot-assisted laparoscopic surgery (RLS), it is generally perceived as more expensive. This perception raises questions about the viability of further employing RLS, especially amid concerns over its advantages in complications, conversion rates, and the extended operative time. However, from a patient's perspective, although numerous articles have closely compared the total operative costs between RLS and conventional laparoscopic surgery (CLS), finding a common objective ground is complicated—not to mention considering the exchange rate at the time of surgery (Morino, 2006). Moreover, the information may not be practically relevant to patients, as the total operation cost does not directly correlate with the actual payment by patients due to varying insurance policies across different companies, hospitals, and countries. Aboumarzouk et al. highlighted in their meta-analysis that the so-called 'total cost' fails to account for the 'social cost analysis', which considers the benefits of quicker recovery and shorter convalescence (Aboumarzouk, 2012).
Similarly, from the hospitals' perspective, the profitability of RLS should take into account not only the quantitative aspects such as the cost of equipment, operation time, training surgeons for both CLS and RLS considering their respective learning curves, and the impact of RLS's longer operative time on hospital revenue, hospital stay, blood loss, and insurance policies, but also qualitative factors. These include the surgeon's safety from infections like HIV, repeated radioactive exposure from bedside X-rays, and the comfort of surgeons during surgery by allowing them to sit. Lin et al. also noted that insufficient data and significant heterogeneity due to differences in skill, the extent of lymph node dissection, and the duration of the learning curve preclude a comprehensive meta-analysis of cost-effectiveness (Lin, 2011). Moreover, the unique capability of RLS for remote surgery in scenarios like war and rural areas should not be overlooked. Furthermore, it is empirically understood that the cost of new technology tends to decrease over decades. From the perspective of the public or investors in surgical robotics, it is advisable to consider these underlying factors when evaluating the cost-effectiveness of robotic surgery.
My general subjective opinion on surgical robotics
It may be surprising that the criticisms leveled at laparoscopic pioneers between the 1950s and 1990s bear a striking resemblance to those currently directed at surgical robotics. Most of the criticisms of conventional laparoscopic surgery (CLS), including 'higher complication rates than laparotomies ... attributable mainly to inexperience, and [e]ach procedure normally done via laparotomy [being] re-invented [with] trial and error,' (Page, 2008) are similarly applicable to robot-assisted laparoscopic surgery (RLS). Despite the harsh criticisms in the late 20th century, CLS has now become widely acknowledged as an indispensable surgical method (Pappas, 2004). Thus, mirroring the history of CLS, there remains the potential for RLS to achieve better clinical outcomes in the future, as knowledge and experience continue to accumulate through trial and error across society. This is especially relevant considering that the industry has now entered the era of Industry 4.0, or robotics.
ECMO (ExtraCorporeal Membrane Oxygenation)
ECMO meta-analysis on hazard ratios: Cardiopulmonary Diseases (Frank, 2020)
Extracorporeal membrane oxygenation meta-analysis of time-to-event data in cardiopulmonary disease in adults
In recognition of the benefits of extracorporeal membrane oxygenation (ECMO)[1], clinical outcomes have been the subject of multiple meta-analyses. Previous meta-analyses of ECMO treatment reported forest plots based on relative risks. Unlike a hazard ratio (HR), a relative risk does not consider the time to event and censoring and runs the risk of not using all the available information[2]. In other words, with respect to the patient mortality, the relative risk between ECMO and no-ECMO patient groups cannot avoid overlooking the critical factor of how ECMO has influenced the timing of each event or patient death over the course of disease progression.
Previous meta-analyses have focused on a single indication presumably because, given the wide range of potential applications for ECMO, studying a particular patient population separately is a crucial step in terms of reducing confounding factors. The present study endeavors to investigate ECMO indications of cardiopulmonary disease as a whole and to list the findings of ECMO mortality in individual indications as subgroup analyses. This was done to ensure that a positive result of a particular indication is not automatically applied to a different patient population that may not have the same benefit, and thereby to prevent a potentially unnecessary intervention. Based on the ECMO indications[3, 4], the present study applies time-to-event data to evaluations of both the overall and individual cardiopulmonary indications of ECMO in adult patients in relation to relevant meta-analyses.
To the best of our knowledge, the present meta-analysis is the first attempt to use time-to-event HR data to illustrate a forest plot of all-cause mortality from the use of ECMO in adult patients, in terms of both overall cardiopulmonary indications and individual indications as a subgroup analysis. As shown by the results of the overall analysis, across various indications of ECMO in cardiopulmonary diseases in adults, outcomes favored neither the ECMO group nor the no-ECMO group. However, as to the subgroup analyses, the reduction in mortality in the ECMO group was found in respiratory failure, whereas increased mortality in the ECMO group was noted in post-LTx, bridge to HTx, and post-HTx.
These results should be understood not only in the context of weighing the benefits and adverse effects of ECMO, but also in consideration of patient selection issues. We could not help but notice the propensity to allocate the ECMO treatment to the poor patient conditions. In other words, the no-ECMO groups were selected and specified as groups of patients who required no invasive support[23, 24, 49]. Presumably, this was so because, in a daily practice, ECMO are used in desperate cases such as a cardiogenic shock where, without ECMO implantation, the mortality is critically high. This discriminate propensity of ECMO allocation appears to reflect the wide recognition of the benefits of ECMO treatments[1]but, at the same time, indicates a patient selection bias issue of a meta-analysis on the retrospective studies. Therefore, in addition to the intrinsic benefits and adverse effects of ECMO treatment, biased allocation of ECMO based on patient conditions as a whole appeared to contribute to the aforementioned results.
In this regard, the significant reduction in mortality of the ECMO group in the patients with respiratory failure compared with the no-ECMO group is worthy of mention. That is, against the patient selection biases that presumably favor the superior outcome in the no-ECMO group, the significantly improved patient outcomes in respiratory failure in the concurring ECMO group is evident. Our result favoring the ECMO group in respiratory failure is consistent with previous meta-analyses for H1N1 pneumonia[65]and ARDS[66]. It can be tentatively proposed thatthe inclusion of the two RCTs, which is less apt to be influenced by the patient selection bias, may contribute to the significant reduction in mortalityof the forest plot due to the increased statistical power of the pooled studies. In addition,Annichet al.stated that themajority of patients with respiratory failure including ARDS has been well supported with veno-venous (V-V) ECMO[1]. In this regard, the increased likelihood of normal cardiac function in respiratory failure conditions could enable the more frequent use of V-V ECMO (or all the use of V-V ECMO[22]), which could avoid the complications of veno-arterial (V-A) ECMO, such as systemic embolization, arterial trauma, and increased left ventricular afterload[67, 68]. However, in consideration ofnumerous possible confounding factors of heterogeneities that may have influenced the mortality results, this hypothesisneeds to be enlightened by more meticulous reasoning that unleashes which factorscontributed to this deviation of respiratory failure subgroup analysis from the overall global analysis.
Although we are aware of the fact that other ECMO meta-analyses conducted database searches on PubMed, EMBASE, Cochrane, and so forth, we searched against the PubMed database only[69], due to the following reasons. During the pilot study, we found that this study required quite an inclusive search of keywords for various cardiopulmonary ECMO indications, compared with meta-analyses on a single indication, as manifested by the total number of articles we worked with. In addition, unlike meta-analyses on relative risks and mean differences, a full-text was laboriously required to confidently make a decision to exclude its corresponding article, because the survival analysis is usually not the main topic of the referenced study but typically comprising just one line of hazard ratio information in the result table or one Kaplan-Meier survival curve figure. Nonetheless, we acknowledge that the risk of missing appropriate articles by not searching against multiple databases could have lowered the reliability of our study[70].
Whenever HRs and their variances were not reported explicitly, we estimated them from the information reported in the studies. Therefore, the significance of the results of the forest plot should have been diminished by our estimates of HR and variances. In further research, reporting numerical hazard ratios explicitly to facilitate later meta-analysis should be encouraged to investigate the mortality associated with ECMO use.
ECMO meta-analysis on hazard ratios: Respiratory failure (Frank, 2020)
Extracorporeal membrane oxygenation meta-analysis of time-to-event data in respiratory failure in adults
In recognition of the benefits of extracorporeal membrane oxygenation (ECMO) [1], clinical outcomes have been the subject of multiple meta-analyses. Respiratory failure incorporates 'oxygenation failure' of acquiring oxygen and 'ventilatory failure' of eliminating carbon dioxide [2], which are, respectively, exemplified to ECMO indications of "acute respiratory disease syndrome" (ARDS) and "hypercapnic respiratory failure" [3, 4]. The controversial efficacy of ECMO on patient mortality in respiratory failure has been statistically assessed by previous meta-analyses based on relative risks [5-9].
Unlike a hazard ratio (HR), the relative risk does not consider the time to event or censoring and runs the risk of not using all the available information [10]. In other words, with respect to patient mortality, the relative risk between ECMO and non-ECMO patient groups cannot avoid overlooking the critical factor of how ECMO has influenced the timing of each event or patient death over the course of disease progression. In consideration of heterogeneities such as veno-arterial (VA) and veno-venous (VV) types, this present study applies time-to-event data to evaluations of the utility of ECMO in patients with respiratory failure.
To the best of our knowledge, the present meta-analysis is the first attempt to use time-to-event data to illustrate a forest plot of mortality from the use of ECMO in adult patients, comprising both VA type and a majority of VV type, in respiratory failure of 'oxygenation failure' and 'ventilatory failure', compared against no ECMO group. When confining to only VV-ECMO, significant reduction in mortality was also noted.
These results should be understood not only in the context of weighing the benefits and adverse effects of ECMO, but also in consideration of patient selection issues. Although the propensity to allocate the ECMO treatment to poor patient condition was not explicitly located in the referenced studies [27-31], the non-ECMO groups were reportedly selected and specified as groups of patients who required no invasive support [33-35]. This discriminate propensity of ECMO allocation appears to reflect the wide recognition of the benefits of ECMO treatments [1] but, at the same time, indicates a patient selection bias issue of a meta-analysis on the retrospective studies. Therefore, in addition to the intrinsic benefits and adverse effects of ECMO treatment, biased allocation of ECMO based on patient conditions as a whole appeared to contribute to the aforementioned results.
In this regard, the significant reduction in mortality of the ECMO group in the patients with respiratory failure compared with the non-ECMO group is worthy of mention. Although VV-ECMO could avoid the complications of VA-ECMO, such as systemic embolization, arterial trauma, and increased left ventricular afterload [36, 37], even VV-ECMO alone is still associated with risk of haemorrhage [27, 28, 30] and circuit-associated complications [5]. That is, against the known complications of the ECMOs and the patient selection biases that presumably favor the superior outcome in the non-ECMO group, the significantly improved patient outcomes in respiratory failure in the ECMO group is evident. Our result favoring the ECMO group in respiratory failure is consistent with previous meta-analyses for H1N1 pneumonia [7] and ARDS [5]. It can be tentatively proposed that the inclusion of the two RCTs, which is less apt to be influenced by the patient selection bias, may partially contribute to the significant reduction in mortality of the forest plot due to the increased statistical power of the pooled studies. In addition, the majority of ECMO in the referenced studies was veno-venous type, possibly due to the increased likelihood of normal cardiac function in respiratory failure conditions, which enable the more frequent use of VV-ECMO (or only the use of VV-ECMO [30]) and could avoid the complications of VA-ECMO. However, in consideration of numerous possible confounding factors of heterogeneities that may have influenced the mortality results, this hypothesis needs to be enlightened by more meticulous reasoning which unleashes what factors contributed to the positive results of respiratory failure indication.
In reality, the number of ECMO studies tend to be small compared to those on relative risks, and relevant mortality studies on ECMO were not always explicitly designed to meet one subcategory of respiratory failure classification, such as 'ARDS' and 'acute respiratory failure', strictly and mutually exclusively. Thus, the scope of this current study on respiratory failure comprises mortality of respiratory failure by either 'oxygenation failure' or 'ventilation failure.' In the meanwhile, technically speaking, respiratory failure type III occurs during perioperative periods that can be related to cardiopulmonary ECMO indications, to name a few, of "bridge to lung transplantation" [3, 4]; while respiratory failure type IV results from shock, which can be related to "myocardial infraction-association cardiogenic shock" [3, 4]. Nonetheless, for more focused investigation, this study condenses to the mortality of hypoxemic (type I: oxygenation failure) and hypercapnic (type II: ventilation failure) respiratory failure.
Although we are aware of the fact that other ECMO meta-analyses conducted database searches on PubMed, EMBASE, Cochrane, and so forth, we searched against the PubMed database only [38], due to the following reasons. During the pilot study, we found that this study required quite an inclusive search of keywords, as manifested by the total number of articles we worked with. In addition, unlike meta-analyses on relative risks and mean differences, a full-text was laboriously required to confidently make a decision to exclude its corresponding article, because the survival analysis is usually not the main topic of the referenced study but typically comprising just one line of hazard ratio information in the result table or one Kaplan-Meier survival curve figure. Nonetheless, we acknowledge that the risk of missing appropriate articles by not searching against multiple databases could have lowered the reliability of our study [39].
Whenever HRs and their variances were not reported explicitly, we estimated them from the information reported in the studies. Therefore, the significance of the results of the forest plot should have been diminished by our estimates of HR and variances. In further research, reporting numerical hazard ratios explicitly to facilitate later meta-analysis should be encouraged to investigate the mortality associated with ECMO use.
Based on the time-to-event data of respiratory failure, ECMO comprising both VV and VA types and the VV type alone has shown to provide advantages over alternative therapy. Although VV-ECMO alone on respiratory failure was mainly addressed in this study, future investigation of the efficacy of VA-ECMO alone in respiratory failure may be more informative, due to being a more common modality of ECMO yet with greater complications [5]. The accumulation of ECMO time-to-event data studies in respiratory failure will enable more focused mortality assessments, for example, on ARDS, exclusively.
It is acknowledged that the ECMO technology from 1975 has changed immensely such that mortality may be correlated with the year, which is exemplified in the improved mortality over years in-between 1995-2000 and 2001-2004 [32]. For the referenced studies, the meta-regression analysis of the midpoint of the study period versus the hazard ratio (Figure 5) illustrates an insignificance (p-value = 0.8011) and neither positive nor negative correlation (r = 0.0635) in the scope of this study.
My Thoughts about Relevant Books, Films, and Media
Artificial Intelligence
In Ethem Alpaydin's "Machine Learning," while machine learning enables systems to adapt and learn from data in dynamic environments, artificial intelligence encompasses the broader capacity for systems to perform tasks requiring human-like intelligence, including but not limited to learning.
- A Perspective from 'AI Assistants' by Roberto Pieraccini
Deep Learning's Impact on Speech Recognition and Understanding: The evolution of machine learning, as detailed in "AI Assistants" by Roberto Pieraccini, particularly through deep learning, has fundamentally altered the approach to creating machine intelligence, transitioning from constructing complex programs to enabling machines to learn from examples. This shift has markedly enhanced the capabilities of machines in understanding and generating speech, with automatic speech recognition (ASR) and natural language understanding (NLU) seeing significant advancements. The advent of deep neural networks (DNNs) has led to the development of systems that can process speech and language in an end-to-end manner, obviating the need to optimize individual components separately. This holistic approach has not only streamlined the process but also resulted in synthetic speech nearly indistinguishable from human speech and performance levels previously unimaginable. Deep learning's role in these advancements underscores its singular effectiveness, outpacing traditional methods and embodying the new mainstream in crafting machine intelligence.
Divinity in Recommendation History: Recommendation engines are pivotal in shaping human decisions, much like the steam engine revolutionized the industrial age. The lineage of recommendation systems is intertwined with humanity's quest for wisdom and self-reflection. Historically, these systems were seen as divine conduits, with oracles and astrologers in ancient civilizations like Greece, China, and Rome offering insights into life's dilemmas. The I Ching, for instance, stands as a primeval algorithmic system, compensating its lack of personal data with a rich, evocative user experience. Despite the shift from divine to data-driven systems over millennia, the human yearning for guidance remains unaltered, underscoring our continuous search for meaningful advice and self-discovery.
Singular Value Decomposition (SVD): In the realm of recommendation engines, Singular Value Decomposition (SVD) plays a crucial role. This mathematical technique reduces data to its most informative elements, uncovering 'latent features' that influence user preferences. By breaking down complex matrices into simpler submatrices, SVD reveals hidden connections and similarities between users' tastes, transcending the limitations of traditional nearest neighbor methods. It's a profound tool that can identify shared preferences even among users with no direct rating overlaps, demonstrating the depth and complexity of modern recommendation systems.
Reward Function to Navigate AI Decision-Making Complexity: In the realm of artificial intelligence, the reward function emerges as a critical component in navigating the inherent uncertainties of decision-making with finite, incomplete data. This function enables AI to probabilistically model and pursue actions that maximize expected rewards, regardless of the environment's complexity. Such a mechanism is vital for AI to effectively address a broad spectrum of challenges, including the basic need for resources, illustrating its role in adapting to any given situation with the most advantageous strategy. The true measure of AI, however, lies not in its ability to mimic human emotions but in its consistent behavior towards achieving long-term, desirable outcomes. Designing a reward function that ensures such behavior, while preventing undesirable actions, presents a significant challenge, underscoring the delicate balance between maximizing efficiency and adhering to ethical standards. Ultimately, the effectiveness and safety of AI hinge on the careful crafting of reward functions that can handle the unpredictability of real-world scenarios without faltering.
AI-Complete: Tasks that necessitate human-level intelligence for resolution, signifying an AI system's attainment of general intelligence on par with humans, are known as AI-complete challenges. These tasks often involve complex decision-making and understanding, akin to roles such as lawyers, scientists, and psychiatrists. Successfully performing AI-complete tasks means the AI can reason, plan, learn, and comprehend natural language at a level currently unique to humans, marking a significant milestone in AI development. Understanding AI-complete challenges helps grasp the scope of AI's potential and the innovation needed to attain human-like intelligence.
Intuition Unchained: Peter J. Denning and Matti Tedre, in "Computational Thinking," highlight a transformative trend within mathematics and logic with their observation: "[M]uch work in mathematics and logic has aimed at eliminating intuition from routine calculation and logical inference. Eliminating intuition from routine jobs did not mean eliminating experts, but rather making their expertise available to a large number of non-expert people." This perspective emphasizes not the obsolescence of human intuition in the wake of artificial intelligence (AI) but the augmentation of human capability, where AI serves as a bridge, enabling non-experts to advance step by step in tasks traditionally necessitating intuition.
The integration of AI into various domains does not supplant the need for human intuition; instead, it amplifies the potential for collaborative problem-solving, where intuition and computational power converge. This synergy allows for the broadening of expertise beyond traditional confines, empowering those without specialized knowledge to engage meaningfully with complex tasks. In essence, AI acts not as a replacement for intuitive human processes but as an enhancer of human intellectual endeavors. This nuanced approach heralds an era where innovation is not solely the domain of the expert but is accessible to a wider audience, facilitated by AI's ability to democratize the process of discovery and decision-making.
Hannun et al. published an article in Nature Medicine in 2019 that demonstrated how a deep neural network (DNN) could outperform cardiologists in the diagnosis of 14 different heart arrhythmias using ECG data. The DNN was trained on a dataset of over 400,000 ECG recordings and achieved an accuracy of 94.4%, compared to an average accuracy of 80.6% for a group of cardiologists. The study suggests that DNNs could potentially be used as a diagnostic tool for arrhythmias, and may even outperform human experts in some cases.
- A.I. Engine
ChatGPT, developed by OpenAI, marks a pivotal moment in AI, often compared to the "iPhone moment" in smartphones. This analogy draws from the iPhone's transformative impact on mobile technology, as emphasized by Steve Jobs in 2007 (Apple Inc., 2007).
- In-Database Machine Learning
The traditional approach to database processing often involves significant resource expenditure in moving data to and from algorithms, which can be a bottleneck. In-database machine learning, however, represents a paradigm shift by integrating the machine learning algorithms directly within the database engine, accessible via SQL. This approach eliminates the need for moving data, thereby offering faster performance and enhanced security. Apache Hadoop is a prime example of this innovation, as it offers a comprehensive suite of tools including Hadoop MapReduce, which collectively enable efficient in-database processing and analysis of large data sets.
- Exploring AI raises profound questions about our knowledge, society, and ethics, across several key domains:
Post-humanism: Post-humanism challenges the idea that AI must mimic human intelligence or morality. It suggests that AI might develop distinct forms of intelligence and moral reasoning, different from human capabilities. This raises questions about the moral capacities and responsibilities of AI systems, especially when they make decisions with ethical implications, like in the case of self-driving cars facing moral dilemmas. The debate extends to how interacting with AI affects human morality, examining our actions towards AI through the lens of our own moral character.
Totalitarianism: The concern with AI in the context of totalitarianism revolves around its potential use in subtle forms of manipulation and surveillance. This is not limited to overtly authoritarian political systems but can manifest in everyday technologies, like children's toys embedded with AI. These devices, while seemingly innocuous, may collect and utilize personal data without the user's knowledge, leading to privacy concerns and the potential misuse of information. This scenario exemplifies the hidden yet effective ways AI can be employed in monitoring and influencing behavior, raising ethical questions about consent and data security.
Responsibility in Relation to Transparency and Explainability: As AI gains more autonomy in tasks traditionally done by humans, assigning moral responsibility becomes complex. AI's decisions, often fast and based on intricate algorithms, challenge our ability to intervene or fully understand their rationale. This leads to the "black box" problem, where the AI's decision-making process is not transparent, especially in advanced systems like deep learning. While decision trees are more transparent, deep learning algorithms can be opaque, leaving users and creators uncertain about how decisions are made and their consequences. Thus, even with AI's increasing capabilities, the question of who bears responsibility for its actions remains a significant ethical challenge.
Singularity & Transhumanism: The concept of Singularity — where AI may surpass human intelligence and potentially merge with it — raises concerns about our readiness to address the ethical and societal implications of such advanced systems. This ties into transhumanism, the idea of enhancing human abilities and longevity, possibly leading to a new form of existence, as explored by Yuval Noah Harari in "Homo Deus." The rapid development and application of such technologies might outpace our understanding of their broader consequences.
↓ This content is not sourced from the book "AI Ethics." ↓
In the United States, the foundation for much of the nation's privacy legislation is rooted in the Fair Information Practice Principles established in 1973. These principles have significantly influenced the country's approach to privacy law. In the European Union, initial steps towards privacy legislation were taken with the Data Protection Directive in 1995, which set the groundwork for privacy regulations in the EU. This was further expanded by the General Data Protection Regulation (GDPR) in 2016, which built upon the Data Protection Directive's principles, offering comprehensive and enforceable data protection standards across all EU member states.
However, a more universally acknowledged framework for personal privacy and data protection is found in the Organisation for Economic Co-operation and Development (OECD) Guidelines on the Protection of Privacy and Transborder Flows of Personal Data, first published in 1980. These guidelines provide a clear definition of personal data, referring to it as records related to an identifiable individual, known as the data subject. The OECD guidelines in 2013 lay out eight core principles designed to safeguard the privacy of data subjects:
Collection Limitation: Personal data collection should be restricted and conducted lawfully and fairly. Where possible, it should be done with the data subject's knowledge or consent.
Data Quality: Data must be pertinent to its intended use and maintained accurately, completely, and up-to-date as necessary.
Purpose Specification: The reasons for collecting personal data should be clearly defined at the time of collection. Use of the data should be confined to these specified purposes or those compatible with them, with any change of purpose explicitly stated.
Use Limitation: Personal data should not be used or disclosed for purposes other than those specified, except with the subject's consent or under the authority of law.
Security Safeguards: Reasonable security measures must be in place to protect personal data from risks like loss, unauthorized access, or misuse.
Openness: There should be a policy of transparency regarding practices and policies related to personal data. Information about data collection and usage, as well as details about the data controller, should be easily accessible.
Individual Participation: Individuals should have the right to confirm if a data controller has their personal data, access their data in a timely and reasonable manner, and challenge or appeal any refusal to grant access. They should also be able to contest the accuracy of their data and have it corrected or amended as needed.
Accountability: Data controllers must be accountable for adhering to these principles, ensuring compliance with the appropriate measures.
Federated Learning is particularly relevant in scenarios involving multiple healthcare institutions. It is a decentralized form of data processing, ideal when dealing with diverse and widespread data sources, such as several hospitals contributing to a joint research study. In this setup, each hospital trains a local model on its own data and then sends only the model updates, not the raw data, to a central system. These updates are then aggregated to improve the overall model. This method ensures that the core model benefits from the diverse data while maintaining the confidentiality of individual patient records, as the actual data remains within the confines of its original location.
Differential Privacy is a method that ensures the privacy of an individual's data in analyses by guaranteeing that results do not hinge on any single person's information. The methodology is strengthened by adding randomness to the data, either through techniques like a randomized coin flip or by applying the principles of the Laplace distribution to adjust the sensitivity. This introduction of noise effectively masks individual data points, thus significantly enhancing privacy and security in the data analysis process.
- A Perspective from 'AI Assistants' by Roberto Pieraccini on the Impact of GDPR and Federated Learning
The Role of GDPR in Shaping Digital Privacy: In the evolving landscape of digital privacy, the General Data Protection Regulation (GDPR) stands as a cornerstone of privacy protection within the European Union, underscoring the significant shifts required by industries to align with stringent data handling and privacy standards. Enacted in 2016, GDPR has spurred a reevaluation of how personal data is collected, processed, and stored, emphasizing the importance of user consent, data minimization, and the safeguarding of personal information against unauthorized access. This regulatory framework has catalyzed a move towards more secure and privacy-preserving methodologies in the development and deployment of machine learning (ML) technologies, reflecting a broader awareness of privacy risks in the digital domain, including social networks, smart home devices, and virtual assistants.
Federated Learning - A Paradigm Shift in Privacy-Preserving Technology: Amidst this backdrop, federated learning emerges as a pioneering approach, exemplifying the shift towards privacy-centric technology development. Conceived by Google, federated learning enables ML models to learn from decentralized data sources without the need to transfer personal data to the cloud. This model of computation is particularly beneficial for applications requiring sensitive data processing, such as speech recognition or text input on mobile devices, allowing for continual improvement of local model performance through ML training directly on the device. By leveraging the increasingly powerful computing capabilities of personal devices, federated learning ensures that personal data remains within the user's control, with only encrypted, aggregated updates shared to refine and enhance the collective intelligence of the system. This not only safeguards user privacy by design but also signifies a significant advancement in the utilization of ML for personalization and efficiency, without compromising data security.
- A Perspective from 'Deep Learning' by John D. Kelleher on Privacy and Ethics in Algorithmic Decision-Making
The increasing reliance on algorithmic decision-making, particularly in deep learning, raises significant privacy and ethical concerns. Recital 71 of the General Data Protection Regulation (GDPR) emphasizes the necessity of transparency by affirming that individuals affected by automated decision-making processes have the right to understand how these decisions are made. However, the legal clarity of this "right to explanation" remains ambiguous, with specific implications for machine learning and deep learning still to be fully defined through judicial interpretation. This ambiguity underscores the societal need for a better understanding of how deep learning models use personal data. From a technical perspective, the ability to interpret and analyze the inner workings of these models is crucial, as it can reveal biases and pinpoint scenarios where the model may fail. In response, the AI research community is increasingly focused on explainable AI, with numerous projects and conferences dedicated to enhancing transparency and human interpretability in machine learning. By analyzing the inputs that trigger specific behaviors in a network, such as neuron activation, researchers can improve the transparency of AI systems, ensuring their alignment with GDPR requirements and broader ethical standards. This emphasis on explainability and accountability aims to address the ethical and privacy concerns associated with algorithmic decision-making, fostering more responsible and trustworthy AI systems.
Virtual Reality (VR): VR immerses users in entirely fictional yet realistic virtual environments, isolating them from the physical world. It provides a complete escape into computer-generated scenarios, often with a sense of presence and interaction.
Augmented Reality (AR): AR enhances our real-world experience by overlaying convincing digital images, sensory elements, and haptic feedback onto our physical surroundings. It allows users to interact with digital information while remaining in the physical world.
Mixed Reality (MR): MR blends real and virtual elements seamlessly, enhancing our perception and interaction with our surroundings. It overlays virtual or augmented features onto the real world, creating a unified and immersive experience.
Extended Reality (XR): XR is a transformative realm that encompasses various immersive technologies, with Virtual Reality (VR) offering complete immersion and Augmented Reality (AR) complementing the physical world. When VR and AR converge into Mixed Reality (MR), it marks the dawn of XR's potential. What sets XR apart is its profound ability to bridge the gap between humans and computers. Our brains are inherently wired to perceive events in 3D, and XR takes this connection to the next level. XR's promise lies in its potential to create immersive, interactive, and multisensory environments, empowering professionals to control and adapt their surroundings while leveraging real-time feedback. In this evolving landscape, augmented, virtual, and mixed realities redefine perception, offering innovative ways to interact with the world. Yet, realizing XR's full potential hinges on the sophisticated integration of digital technologies and software capable of orchestrating seamless virtual experiences.
- Challenges and Solutions in Extended Reality (XR)
Challenges in Extended Reality (XR): Overcoming the hurdles in the realm of Extended Reality (XR) is no small task. Ensuring that results like data overlays or graphic representations are visible in real-time is crucial, as even the slightest lag can disrupt the immersive Augmented Reality (AR) experience. Additionally, integrating haptic feedback into devices such as smartphones and game controllers is a distinct challenge, as it demands precision and realism, not merely a general approximation of touch. Moreover, the stakes are high in AR systems, especially when used in hazardous work environments; rendering images or data inaccurately could lead to serious injuries or even fatalities. What makes XR design and engineering particularly challenging is the vulnerability to even the tiniest glitches or flaws in hardware, software, user interfaces, or network performance, which can swiftly transform a convincing XR environment into an implausible one.
Balancing Realism and Comfort in XR Environments: While there's a common notion that virtual reality and other XR forms must replicate reality flawlessly, this belief isn't always necessary. In many cases, the objective is to create environments realistic enough to elicit desired responses in the mind and body without aiming for absolute fidelity. To strike a balance, those designing and developing XR frameworks must navigate trade-offs between fidelity and immediacy. Within virtual worlds, the creation of the illusion of events, while avoiding sensory overload from genuine physical forces and motions, becomes a priority. Embracing a muted experience can reduce sensory conflicts that often lead to discomfort, dizziness, or motion sickness within virtual environments. Moreover, duplicating the exact physiology of the hand isn't imperative; the goal is to trick the brain into perceiving a realistic sense of touch, not replicating every intricate detail.
↓ In resonance with the themes explored in Samuel Greengard's book 'Virtual Reality,' this discussion presents my independent insights and perspective. ↓
- Exploring the Synergy of 3D Glasses, XR, and Hinduism in 'Avatar'
Spatial Realization through 3D Glasses: In James Cameron's "Avatar," the introduction of 3D glasses wasn't coincidental but aligned perfectly with the core principles of Extended Reality (XR). These glasses enhance three-dimensional perception, a vital component for immersive XR experiences. The film portrays Jake Sully, a disabled character, who undergoes a profound transformation as he enters the world of the Na'vi, utilizing an Avatar that allows him to seamlessly perceive and interact with his new Na'vi body. This spatial realization experienced by the audience, made possible by 3D glasses, not only harmonizes with XR concepts but also beautifully mirrors Jake's journey of self-discovery and transformation throughout the narrative.
The Hinduistic Concept of Avatar in XR: Additionally, the film's exploration of the Hindu concept of avatar, where individuals take on new forms for a purpose or to bring about change, adds depth to its narrative. Jake's transition into his Na'vi Avatar can be seen as a metaphorical representation of the Hindu avatar concept, emphasizing the film's exploration of identity and transformation within the context of XR technology. The synergy between 3D spatial perception and the Hinduistic concept of avatar underscores the film's exploration of XR's transformative power and its ability to explore complex themes of identity, spirituality, and the blending of reality and fiction.
- 'Ready Player One' and the Inspiration Behind VR Innovation
The VR Landscape of 'Ready Player One': In 'Ready Player One,' the depiction of technology within the OASIS primarily aligns with the principles of virtual reality rather than the broader spectrum of extended reality. This distinction is evident in the complete immersion of users into a fully digital universe, where the physical world is entirely obscured in favor of a computer-generated environment. The use of immersive VR gear, which includes headsets and haptic suits, enables users to interact with and experience the OASIS as an entirely separate reality. Unlike XR, which includes augmented reality (AR) and mixed reality (MR) to blend or augment the physical world with digital elements, the VR in 'Ready Player One' is characterized by its exclusive focus on creating a separate virtual experience. This emphasis on VR as a form of total escapism and transformative potential of entirely immersive digital worlds is underscored by the poignant reminder from James Halliday, the creator of the OASIS, that "Reality is the only thing that's real." This statement reinforces the film's depiction of VR, underscoring a clear demarcation from XR by highlighting the importance of distinguishing between virtual escapism and the tangible, irreplaceable value of real-world experiences.
How 'Ready Player One' Prefigures XR Developments: Despite the visual similarities between the VR goggles depicted in the movie and the Apple VisionPro, the latter is designed for extended reality (XR), offering capabilities beyond the VR-specific focus showcased in the film. The movie's emphasis on VR technologies, such as haptic suits and immersive visors, mirrors current technological trends and underscores the importance of tactile feedback and facial recognition in enriching virtual experiences.
- The Matrix: VR and the Realm of Simulated Reality
VR and Beyond: "The Matrix" franchise, while not explicitly categorized under contemporary terms like AR (Augmented Reality), XR (Extended Reality), or VR (Virtual Reality), significantly explores concepts that are foundational to these technologies. The films depict a dystopian future where humanity is unknowingly trapped in a simulated reality called the Matrix, created by sentient machines. This simulated reality is so comprehensive and immersive that it effectively functions as a form of virtual reality, albeit one that individuals are forcibly plugged into without their knowledge or consent. The Matrix's simulated world parallels VR in that it's a completely immersive digital environment where physical laws can be bent or broken, and individuals interact within this space as if it were the physical world. However, it goes beyond traditional VR because it's not a voluntary or leisure activity; instead, it's a pervasive illusion meant to subdue humanity. While "The Matrix" doesn't explore AR or XR in the sense of overlaying digital information onto the real world or blending real and virtual environments in a seamless manner, its exploration of virtual reality's philosophical and ethical implications has profoundly influenced public perception of VR and related technologies.
- Exploring AR and MR Technologies in 'Minority Report'
Precognitive Visions and Advanced Technologies: In "Minority Report," the use of Augmented Reality (AR) and Mixed Reality (MR) technologies is further exemplified through the innovative depiction of precognitive visions — the futuristic images seen and recorded by the three precogs. These visions, central to the film's plot, are integrated into the physical environment of the PreCrime unit, where law enforcement officers can interact with and analyze these future events in real time.
Through the Lens of AR: AR is demonstrated through the overlay of these precognitive visions onto physical screens and interfaces within the PreCrime headquarters. This allows them to examine details of potential crimes before they occur, enhancing their ability to prevent them. The seamless integration of digital information (the precogs' visions) into the physical workspace exemplifies AR's capability to augment reality with additional layers of data.
Exploring MR's Potential: MR is showcased as these holographic images are not just passively displayed but are interacted with through gesture-based controls. The protagonist, John Anderton, uses hand gestures to move, scale, and probe into the holographic data, merging digital and physical realities. The MR technology enables a more immersive interaction with the digital content, allowing Anderton and his team to explore the visions spatially as if they were physically present within the scene of the future crime. This blending of real and virtual elements is a hallmark of MR, creating an environment where digital and physical worlds coexist and interact in real time.
- Tron: The 1982 Odyssey into Digital Universes and the Dawn of Virtual Gaming
Game On in VR: The 1982 film Tron pioneers the depiction of an intricate digital universe within a computer system, setting a foundational narrative for the interaction between humans and digital realms. This visionary work presages the immersive digital spaces that are central to contemporary VR discussions. Remarkably, the film emphasizes the significance of gaming, a concept that, while universally acknowledged today, was groundbreaking in 1982. Flynn not only highlights the crucial role of gaming in the real world, attributing it substantial value in terms of corporate power and financial gain, but also extends this importance into the virtual realm. It presents a narrative where the protagonist, absorbed into the game's universe, must navigate its challenges to unearth evidence of corporate malfeasance — specifically, the theft of a game by Dillinger from Flynn.
- The Convergence of VR and Reality in 'Tron: Legacy'
From VR to Reality: "Tron: Legacy" explores the dynamic relationship between Virtual Reality (VR) and its implications in the real world, building on the foundational themes introduced in the 1982 classic, "Tron." This sequel makes the intricate concepts of computer science more digestible and engaging for audiences, particularly those with a prior understanding of the original film's universe. It articulates the digital domain's potential to address critical real-world issues, notably highlighting (1) VR's significant role in tackling global challenges like disease control and prevention, while also considering (2) the possible dangers these technologies may introduce to the real world. (1') Central to this narrative is the concept of ISOs (Isomorphic Algorithms), which represent a breakthrough in digital evolution with the potential to bring about revolutionary changes in technology and science. The character of Quorra, an ISO, symbolizes the key to unlocking these transformative advancements. (2') Meanwhile, the antagonist Clu, a digital replication of Flynn (the protagonist's father), aims to destroy the ISOs and extend his dominion to the real world by leveraging an army from within the VR landscape. This conflict underlines the ethical and existential questions that arise from advancements in digital technology. The story elegantly weaves together the optimistic prospects of VR and the pioneering spirit of the ISOs with a cautionary narrative about the disruptive potential of technology, showcasing the intricate interplay between virtual innovations and their tangible impacts on the real world.
Cinematic Foresight: In "Tron: Legacy," the intertwining of digital and physical realities is not just a flight of cinematic imagination but a foretelling of the IoT and digital twin technologies discussed in Samuel Greengard's book, "The Internet of Things." This body of work reveals how such innovations, much like in the film, create precise virtual counterparts of physical systems, enabling profound insights and simulations. It underscores how today's use of these technologies, from NASA's spacecraft design to architectural pre-construction VR walkthroughs, mirrors the movie's portrayal of VR's deep-seated impact on the tangible world. The narrative's vision presented in "Tron: Legacy" of virtuality reshaping real-world challenges has emerged into reality, confirming that the film's previously speculative ideas have transitioned into tangible components of our contemporary technological advancements.
- From BOTW to TOTK: The Impact of 'The Legend of Zelda' on VR Gaming
"The Legend of Zelda: Breath of the Wild" (BOTW) and its sequel, "Tears of the Kingdom" (TOTK), stand as monumental achievements in the evolution of virtual reality (VR), presenting a detailed exploration of memory, identity, and the deep connections that bind characters across time. BOTW introduced players to a vast open-world adventure, pioneering in its VR capabilities and narrative depth, where the protagonist Link embarks on a journey of self-discovery through the fragments of his past memories. This exploration shares thematic resonance with iconic films like "Ghost in the Shell" and "Blade Runner," where the core narrative revolves around the pivotal role of memories in defining one's identity.
TOTK further expands this narrative framework, exploring the ancestral ties and the very essence of Hyrule's history. (1) The nuanced portrayal of Zelda in TOTK, particularly her transformation into a light dragon to heal the Master Sword, not only cements her central role in the saga but also justifies the series' title, 'The Legend of Zelda.' The act of self-sacrifice, transforming into a light dragon to empower and heal the Master Sword over many generations, alongside the sword's restoration, is what truly defines the Legend of Hyrule. (2) Moreover, Link's role in connecting different eras — awakening after 100 years to confront calamity in BOTW, and in TOTK, using the ancestral Rauru's hand to access memories within the tears of the Light Dragon, thereby understanding past events to determine his actions — illustrates the very reason he is named "Link." This naming reflects his unique ability to bridge past and present, embodying the essence of connectivity and continuity, not just as a focus of the series but as a fundamental characteristic of his identity and purpose within the saga.
Through these intricate narratives, both BOTW and TOTK have effectively elevated the gaming community's perception of VR's capabilities, transcending traditional gameplay to explore complex themes of memory, identity, and legacy. By intertwining the fates of Zelda and Link with the fabric of Hyrule itself, these games not only redefine the potential of virtual storytelling but also affirm why the saga rightly celebrates Zelda's legend, embodying a transformative experience that bridges multiple generations and narratives within the immersive realms of virtual reality.
- My Reflections on 'Spatial Computing': Shaping the Future of Healthcare and Mixed Reality
In "Spatial Computing" by Shashi Shekhar and Pamela Vold, the authors present a future where "all software needs to be spatially aware and where every user is a participant in updating the location information presented with that software." This vision highlights the transformative potential of spatial computing across various domains, including mixed reality (MR) and healthcare. By leveraging spatially aware software, MR applications can enhance the physical world with digital augmentations, leading to more immersive educational and therapeutic experiences. In healthcare, this technology allows professionals to interact with dynamic, three-dimensional representations of patient anatomy, improving diagnostics and surgical precision. The analogy of a baby monitor as a remote-sensing system underscores the practicality and impact of spatial computing, illustrating how everyday objects can gather and utilize spatial data. This participatory, dynamic approach to data not only advances the capabilities of MR and healthcare but also signals a shift towards more personalized, efficient, and interactive technologies.
Regardless of the industry, there's a need for a more flexible and expansive approach to intellectual property than previous generations adopted. Intellectual property laws are undergoing rapid transformations globally, affecting copyrights, patents, and trademarks alike. The most significant shifts are evident in the strategic thinking of business leaders regarding intellectual property, showcasing a dramatic evolution in just the last ten to twenty years.
Intellectual property fundamentally comprises information, its value often rising with increased usage rather than diminishing. Even without monetizing access, organizations can derive substantial benefits when others utilize their created information, as observed with MIT's asset value surge following the OpenCourseWare platform's launch. This instance exemplifies how relinquishing tight control over intellectual property, particularly patents and copyrights, can enhance its worth. Consequently, adopting a balanced strategy encompassing sharing, licensing, and potential charging for intellectual property utilization could optimize value creation for the organization.
Entering a co-development agreement with another organization presents a viable strategy for acquiring intellectual property, a practice prevalent in certain industries but applicable across various economic sectors. Such agreements facilitate collaborative intellectual property development, incorporating a "covenant not to sue," ensuring legal protection during the partnership. When executed effectively, both parties stand to gain increased profits, market share, and enhancements to their intellectual property portfolios. This arrangement ensures joint ownership of any interoperability technology developed together, enabling organizations, regardless of whether they are competitors or operate in complementary domains, to expand their intellectual property portfolios beyond the capabilities of their internal staff.
- A Paradigm Shift in Collaborative Development (in the Web 2.0 Era)
A crucial aspect of the recent surge in Web development, known as Web 2.0, is the trend of companies opening up their systems for interoperability with external developments. The most remarkable growth in the Internet and mobile communications sector is now propelled by collaborative development between customers, third parties, and major software corporations like Facebook and Apple. Apple, for instance, has ingeniously engaged iPhone users and the broader developer community through the iPhone Development Center, fostering the creation of applications across its devices.
This trend is encapsulated in the Open Application Programming Interfaces (Open API) movement, where various participants contribute to a widely accessible ecosystem for developing, using, and refining computer applications, along with managing the data flow between them. Open innovation, the underlying principle of this movement, is straightforward yet potent: innovation can stem from outside your organization. Historically, businesses hesitated to embrace external ideas, partly due to fears of potential compensation claims. However, the desire for improvement and the pleasure derived from creation drive many customers to contribute, demonstrating a central lesson from the Web 2.0 and user-generated content phenomena.
While individuals partake in this process out of self-interest, they also seek enhanced products, believing they can offer better solutions. Open innovation allows organizations to tap into this pool of ideas, necessitating discernment in selecting and incorporating valuable suggestions. This strategy, while maintaining organizational control, can lead to substantial product and service enhancements and even entirely new offerings.
Google and its subsidiary YouTube exemplify this trend. Google distinguishes itself through its proprietary intellectual property, crucial for its search engine dominance. However, its existence relies heavily on the copyrighted material available worldwide on the web. Similarly, YouTube doesn't create its own copyrighted video content but provides a platform for users to share video and audio content, showcasing the symbiotic relationship between major platforms and user-generated content.
↓ In alignment with the concepts explored in 'Intellectual Property Strategy', the following discussion offers my own independent insights and a perspective that resonates with the themes of the book. ↓
- Navigating the Digital Evolution From Web 1.0 to 4.0
In "Cybersecurity" by Duane C. Wilson, the evolution of the World Wide Web (WWW) since its inception by Tim Berners-Lee in 1989 is concisely outlined through its generational shifts. Web 1.0 served as an information portal for businesses, marking the web's initial phase as a broadcast medium. The emergence of Web 2.0 introduced the concept of social networking, grouping individuals by shared interests and enabling a more interactive experience.
Web 3.0 (or the semantic web): Web 3.0 represented a significant leap towards making the web more intelligent, with an emphasis on machine-readable content. This era focused on reducing human tasks by enabling machines to understand data relationships, integrating artificial intelligence (AI) and machine learning (ML) to process and analyze data efficiently.
Web 4.0 (or the symbiotic web): Web 4.0 advances this interactivity, allowing machines to operate autonomously in response to internet content, such as recalling a user's last viewed page or tailoring ads to past searches. This creates a more intuitive and personalized web experience, highlighting a deeper symbiosis between humans and machines.
- IP Strategy for the Symbiotic Web Era (Web 4.0): A Personal Perspective
Crafting IP Strategies for Web 4.0: The intellectual property (IP) strategy for Web 4.0, often characterized as the Symbiotic Web, necessitates a forward-thinking, adaptive approach that recognizes the intricate interactions between humans, machines, and data. Web 4.0 introduced new challenges and opportunities for managing IP.
Adaptive IP Strategy for AI Innovations: In the constantly evolving environment of Web 4.0, where AI and machine learning are at the forefront of content creation and innovation, the need for a flexible intellectual property (IP) strategy becomes apparent. This approach calls for a thoughtful update to IP laws to better define the ownership of AI-generated content, exploring how such creations can be protected, and pondering over who should be considered the rightful IP owner — be it the AI developer, the user, or perhaps, in a more speculative sense, the AI itself. I'm particularly intrigued by how IP will be defined for AI-generated innovations. This scenario underlines the importance of agility in IP management, encouraging companies to quickly adjust to new technological advancements by securing patents, trademarks, and copyrights for emerging innovations. Such a proactive and reflective stance ensures that IP strategies can keep pace with the digital transformation, safeguarding breakthroughs and fostering the swift progress characteristic of Web 4.0.
- The Impact of Creative Priorities on Artistic Work and IP Strategies in the Digital Age: A Personal Perspective
- Balancing Open Innovation and Strategic Protection: A Personal Perspective
The expiration of key patents, such as LEGO's 1958 patent for its unique brick design, has led to a surge of LEGO-compatible products and imitation brands, diversifying the market with more affordable interlocking bricks and a wider range of themes and models. Similarly, the looming expiration of copyrights like Walt Disney's Mickey Mouse indicates a decline in monopolistic control, heralding a new era of artistic freedom and a richer cultural landscape. However, this doesn't mark a complete end to exclusivity; companies like LEGO and Walt Disney are continually filing new patents to maintain their competitive edge, even as their older patents expire. This balance between the end of certain exclusive rights and the pursuit of new ones by these corporations facilitates both the sustenance of their market dominance and the flourishing of creativity and innovation in the public domain.
Why Cloud Computing?: Cloud computing represents a paradigm shift from traditional capital expenditure (CapEx) models, which necessitate significant upfront investment in hardware, to operating expense (OpEx) models that allow for payment based solely on consumption. This model facilitates substantial cost savings and enhances flexibility by enabling the reuse of computing resources and functions, thereby contradicting Gossen's First Law of diminishing returns. In the realm of cloud computing, the utilization of these resources can actually lead to increased value over time, offering an inverse relationship to Gossen's principle. By allowing for the rapid deployment of pre-existing functionalities, businesses achieve greater agility and a reduced time to market for their products and services. Essentially, cloud computing turns the traditional economic model on its head, demonstrating that, unlike the typical diminishing returns seen in other areas, the strategic use of cloud services can amplify efficiencies and benefits with each use.
NIST's Definition: Cloud computing, as defined by NIST (National Institute of Standards and Technology), is a model that provides widespread, easy, and immediate access to a collective pool of configurable computing resources, enabling them to be quickly allocated and released with minimal effort from management or interaction with the service provider. This model is designed to ensure high availability and comprises five key characteristics: broad network access, on-demand self-service, pooled resources with virtualization, rapid scalability, and services measured and metered for use. It is structured around three core service models — Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS) — and is deployed through four models: public, private, community, and hybrid clouds.
(1) Virtualization to (2) Cloud: Cloud computing and virtualization serve as cornerstone technologies in modern IT infrastructures, with (1) virtualization enabling multiple virtual environments to run on a single physical hardware system through server and application virtualization. VMware exemplifies server virtualization by dividing a physical server into multiple virtual servers, allowing for efficient resource distribution and coexistence of various operating systems on a single server, while application virtualization simplifies deployment by enabling centralized access for multiple users. In contrast, (2) cloud computing expands on virtualization's resource optimization, providing scalable, flexible, and metered computing services over the internet, such as servers, storage, and software. It introduces key features like on-demand self-service, broad network access, and rapid elasticity, distinguishing itself from virtualization by offering a comprehensive service model that includes infrastructure, platform, and software as services, thus facilitating a broader range of IT solutions beyond mere resource efficiency.
Unveiling Shadow IT: Shadow IT refers to the use of IT systems, applications, or services without the explicit approval of an organization's central IT department. This practice is particularly prevalent in cloud computing, where the ease of accessing and deploying cloud services enables individuals or departments to bypass traditional IT controls. While shadow IT can foster innovation by allowing users to quickly meet their needs, it also poses significant risks, including security vulnerabilities and compliance issues, due to the lack of oversight and integration with the organization's IT infrastructure. In the context of cloud computing, the unchecked use of shadow IT amplifies these challenges, potentially leading to data breaches and operational inefficiencies as organizations struggle to manage a sprawling, unsecured digital environment.
↓ The information provided does not originate from the book "Cloud Computing," but it has been supplemented with relevant information. ↓
- Privacy Enhanced Through the Power of On-Device AI in Mobile Devices
Samsung Galaxy S24's On-Device AI Revolution: Samsung Galaxy S24 Series introduces a groundbreaking shift towards enhancing user privacy with its advanced on-device AI capabilities. By processing data locally, this technology ensures that personal information remains secure within the device, eliminating the risk associated with external server storage and potential breaches. This move not only caters to the increasing consumer demand for privacy but also sets a new standard in the mobile industry by prioritizing the protection of user data amidst growing concerns over digital privacy.
Galaxy S24 vs. Cloud Vulnerabilities: Contrastingly, relying on cloud-based services for tasks such as translation exposes user data to risks, as it requires processing on external servers beyond the user's control. Despite the robust data protection measures employed by companies like OpenAI, the inherent vulnerability associated with transmitting and storing data off-device poses a significant privacy threat. The Galaxy S24 Series addresses these concerns by keeping all data, including personal conversations and documents, securely within the device, thus offering a superior level of privacy and security.
Paradox of IoT Dependence: At a pivotal point, the advancement of AI and IoT (Internet of Things) is transforming computers from mere reservoirs of human-inputted information into self-sufficient entities with the capacity to perceive the world autonomously, leading to an era where machine independence overshadows human skill. This seismic shift towards a deeply interconnected digital ecosystem offers remarkable insights and operational efficiencies but also posits a paradox; the very technology meant to augment human life could potentially erode fundamental knowledge and skills. If our reliance on technology becomes absolute, the absence of these smart systems could expose a vulnerability in our species, leaving us ill-prepared to manage basic survival tasks without digital assistance. This scenario underscores a profound concern: the advancement of AI and IoT, while propelling us forward, may also be leading us to a precipice where, stripped of these aids, we could find ourselves at a loss, disconnected not only from our roots but potentially jeopardizing our future as a species.
The Fourth Industrial Revolution (4IR) and the Industrial Internet of Things (IIoT): 4IR represents a fundamental shift in the industrial landscape, marked by the integration of cyber-physical systems that blend the physical and digital worlds. At the core of 4IR is IIoT, where machines embedded with smart sensors communicate and interact with each other, with humans, and across digital platforms. This evolution, building on the previous revolutions of mechanization (1IR), mass production (2IR), and computer automation (3IR), is transforming industry by enabling real-time data sharing, advanced analytics, and connectivity across devices and systems, leading to unprecedented levels of efficiency and new ways of manufacturing and delivering services.
Understanding IoT: The Internet of Things (IoT) is a network where devices, from smartphones to sensors, connect and communicate through technologies like Wi-Fi and Bluetooth. It's a complex system of interlinked objects exchanging data and making decisions, often without human intervention, powered by advancements in artificial intelligence. This interconnectedness allows for an unprecedented level of automation and smart functionality in everyday objects, transforming them into active participants in data gathering and analysis.
Contact
Email: Support [AT] nGene.org
Call sign : K3CWKP (FCC) or DS1UHK (Emergency Radio Communication Support Corps)
Acknowledgment
Special thanks to my beloved mom who always trusts me. Were it not for her, it would be impossible for me to implement this software.