README.Rmd

---
output: github_document
bibliography: paper.bib
---

<!-- README.md is generated from README.Rmd. Please edit that file -->

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  warning = FALSE,
  message = FALSE,
  comment = "#>",
  fig.path = "man/figures/README-",
  fig.width = 7.5,
  fig.height = 4
)
```

# memr

```{r setup, include = FALSE}
library(memr)
```

## Medical records embeddings

The ``memr``(Multisource Embeddings for Medical Records) package in R allows for creating embeddings, i.e. vector
representations, of medical free-text records written by doctors. It also
provides a wide spectrum of tools to data visualization and medical
visits' segmentation. These tools aim to develop computer-supported medicine by
facilitating medical data analysis and iterpretation. The package can be exploited for
many applications like the recommendation prediction, patients' clustering etc. that
can aid doctors in their practice.

## Installation & Dependences

``memr`` is written in R and is based on the following packages:

* dplyr
* ggplot2
* ggrepel
* Rtsne
* text2vec
 
To install ``memr``, simply type in an R console (after having installed the `devtools` package, e.g. `install.package('devtools')`):
```{r, eval = FALSE}
devtools::install_git("https://github.com/MI2DataLab/memr")
```

## Usage

### Example datasets
We show the usage of the package on the example datasets. They are completely artificial, but their structure reflects a structure of the real data collected from Polish health centers. The results of the research on the real data are described in the paper @dobrakowski2019patients.

For every visit we can have some information about ICD-10 code of diagnosed disease,
ID and specialty of the doctor:
```{r}
knitr::kable(visits)
```

For the visits we have also the descriptions of interview
with the extracted medical terms:
```{r}
knitr::kable(interviews)
```

Descriptions of examinations of patients:

```{r}
knitr::kable(examinations)
```

And descriptions of recommendations prescribed by doctors to the patients:
```{r}
knitr::kable(recommendations)
```

Each medical term has one or more categories:
```{r}
knitr::kable(terms_categories)
```


### Medical terms embeddings
Firstly we can compute embeddings:
```{r}
embedding_size <- 5

interview_term_vectors <- embed_terms(merged_terms = interviews, embedding_size = embedding_size,
                                       term_count_min = 1L)
examination_term_vectors <- embed_terms(merged_terms = examinations, embedding_size = embedding_size,
                                         term_count_min = 1L)

knitr::kable(interview_term_vectors[1:5, ])
```

Terms from the chosen category can be visualized:
```{r}
visualize_term_embeddings(terms_categories, interview_term_vectors, c("anatomic"), method = "PCA")
```

To validate the quality of embeddings
we can perform the term analogy task
(see more by ?analogy_task). The package delivers
the analogy test set.
```{r}
knitr::kable(evaluate_term_embeddings(examination_term_vectors, n = 5, terms_pairs_test))
```

For each type of analogy we compute the mean accuracy.

Analogies can be plotted to see if
the connection lines are parallel:
```{r}
visualize_analogies(examination_term_vectors, terms_pairs_test$person, find_analogies = TRUE, n = 10)
```


### Visits embeddings
Having the embeddings of terms, we can compute
embeddings of visits:
```{r}
visits_vectors <- embed_list_visits(interviews, examinations, interview_term_vectors, examination_term_vectors)
knitr::kable(visits_vectors[1:5, ])
```

And now we can visualize the visits on the plot and color by the doctors' IDs:
```{r}
visualize_visit_embeddings(visits_vectors, visits, color_by = "doctor",
                                spec = "internist")
```

or by ICD-10 code:
```{r}
visualize_visit_embeddings(visits_vectors, visits, color_by = "icd10",
                                spec = "internist")
```

### Clustering

On the visits' embeddings we can run the k-means algorithm:
```{r}
clusters <- cluster_visits(visits_vectors, visits, spec = "internist", cluster_number = 2)
```

and plot the clusters:
```{r}
visualize_visit_embeddings(visits_vectors, visits, color_by = "cluster",
                                spec = "internist", clusters = clusters)

```

For every cluster we can see the most
frequent recommendations from chosen categories:
```{r}
rec_tables <- get_cluster_recommendations(recommendations, clusters,
                                          category = "recommendation",
                                          recom_table = terms_categories)
rec_tables
```

or from all categories:
```{r}
rec_tables <- get_cluster_recommendations(recommendations, clusters, category = "all")
rec_tables
```

If we have a new visit, we can assign it
to the most appropriate cluster:
```{r}
inter_descr <- paste("cough", sep = ", ")
exam_descr <- paste("fever", sep = ", ")
visit_description <- c(inter_descr, exam_descr)
names(visit_description) <- c("inter", "exam")
cl <- assign_visit_to_cluster(visit_description, clusters, interview_term_vectors, examination_term_vectors)
cl
```

As the last nice thing we can see
the embeddings of ICD-10 codes:
```{r}
visualize_icd10(visits_vectors, visits)
```

# Acknowledgements

The package was created during the research financially supported by the Polish Centre for Research and Development
(Grant POIR.01.01.01-00-0328/17).

# References