Linking back to the source

[PeterWone]

First a little context

I've used marked to render Markdown as HTML in a VS Code extension. A web browser is used to preview rendered documents and now I'm trying to implement mapping from the rendered document back to the source, so the user can double-click a block element in the preview and see it highlighted in the source markdown.

Back to their humble origins

To do this properly I need to annotate HTML block elements with the start and end position of the source Markdown from which they were created.

Raw isn't

In related work I use the lexer to get an array of tokens and provide custom rendering for code blocks (syntax highlighting) and rendering diagrams in fenced blocks, that sort of thing. While working on that I noticed that the tokens have a property raw which is exactly what I need here.

However, setting up this

  marked.use({
    renderer: {
      heading(text: string, level: number, raw: string) {
        return `<h${level} data-raw="${raw}">${text}</h${level}>`;
      }
    }
  });

revealed that this is not your token's raw. There are no hashes at the start of the "raw" string. Instead you get a level parameter. Raw, it would seem, is relative.

I'm at a bit of a loss as to how to go about this. To be frank I suspect a quite clear understanding of the entirety of marked will be required. Apart from mapping I can't imagine any reason to want that kind of information so late in the pipeline.

I had hoped to work out position offset by keeping a running total of raw length, and then add the length of the current raw to get the end position.

Your guidance would be greatly appreciated.

[UziTech]

I don't know what version of marked you are using but it is not the latest because the renderer methods have been updated to take the token instead of a list of parameters. In the latest heading renderer raw is indeed the raw markdown that created the token.

marked.use({
    renderer: {
		  heading({ tokens, depth, raw }: Tokens.Heading) {
		    return `<h${depth} data-raw="${raw}">${this.parser.parseInline(tokens)}</h${depth}>\n`;
		  }
    }
  });

In the latest version of marked you should be able to use raw as an offset for the position with one exception: marked converts carriage returns (\r and \r\n) to newlines (\n) to prevent some issues with parsing so if the input markdown uses carriage returns the raw offset might not be correct.

[PeterWone]

I'm glad I asked here, that was actionable intel! In response to your comment I bumped the version from 12 to 15 and now I'm making some headway. Thank you.

[PeterWone]

This code kind of works

marked.use({
  renderer: {
    heading(token: Tokens.Heading) {
      return `<h${token.depth} data-raw="${token.raw}">${token.text}</h${token.depth}>`;
    },
    paragraph(token: Tokens.Paragraph) {
      return `<p data-raw="${token.raw}">${token.text}</p>`;
    },
    blockquote(token: Tokens.Blockquote) {
      return `<blockquote data-raw="${token.raw}">${token.text}</blockquote>`;
    },
    list(token: Tokens.List) {
      const type = token.ordered ? 'ol' : 'ul';
      const items = token.items.map(item => `<li data-raw="${item.raw}">${item.text}</li>`).join('');
      return `<${type} data-raw="${token.raw}">${items}</${type}>`;
    },
    table(token: Tokens.Table) {
      const header = token.header.map(cell => `<th>${cell.text}</th>`).join('');
      const body = token.rows.map(row => `<tr>${row.map(cell => `<td>${cell.text}</td>`).join('')}</tr>`).join('');
      return `<table data-raw="${token.raw}"><thead><tr>${header}</tr></thead><tbody>${body}</tbody></table>`;
    }
  }
});

In the <li> elements the fact that I'm handling the rendering suppresses recursive processing so the backticks don't render as <code>.

Is there any way to return to default processing or do I have to code the descent myself?

[UziTech]

token.text is the markdown text not the html text. To parse the tokens to html use this.parser.parseInline(tokens) for inline tokens or this.parser.parse(tokens) for block tokens

heading(token: Tokens.Heading) {
  return `<h${token.depth} data-raw="${token.raw}">${this.parser.parseInline(token.tokens)}</h${token.depth}>`;
},
blockquote(token: Tokens.Blockquote) {
  return `<blockquote data-raw="${token.raw}">${this.parser.parse(token.tokens)}</blockquote>`;
},

There are lots of edge cases the renderers have to deal with so instead of writing your own I would suggest copying marked renderers and changing the output or just adding to the marked renderer output.

import { marked, Parser, Renderer } from 'marked';

const markedRenderer = new Renderer();
markedRenderer.parser = new Parser();

marked.use({
  renderer: {
    heading(token) {
      const html = markedRenderer.heading(token);
      return html.replace(/^<h\d/, `$& data-raw="${token.raw}"`);
    },
    blockquote(token) {
      const html = markedRenderer.blockquote(token);
      return html.replace(/^<blockquote/, `$& data-raw="${token.raw}"`);
    },
  },
});

console.log(marked.parse('# test\n\n> test'));

[PeterWone]

Thank you, this is exactly what I wanted to do - have it process normally using mature code that handles edge cases, then inject the data-raw attributes. I've redone all my code this way.

One final silly question - how to handle it when the Markdown source contains embedded HTML. I'd like to capture the whole chunk of HTML and wrap it in a div with the data-raw attribute on the div. There doesn't seem to be a renderer method to intercept "chunk of literal HTML" the way there is for heading, paragraph etc. Probably this is because literal HTML doesn't need rendering. Where would you intercept that?

[PeterWone]

Never mind, I figured it out. Goes in the same block of code I use for altering the tokens emitted by the lexer to handle fenced blocks, now it also rewrites HTML tokens.

[PeterWone]

I'm using parseInline(thing.raw) and it seems to do the trick, please correct me if that's not how it's done.