Copy either one of these two files:
components\com_comment\joscomment\templates\akostyle\images\html.png
components\com_comment\joscomment\templates\JQdefault-emotop\images\html.png
into the folder:
components\com_comment\joscomment\templates\modern\images\
The reason I said "either one of these two" is that they are both different icon styles, the choice is yours
This was a mistake we made, we updated the install script to include html.png with the modern template since though.
EDIT: Of course, if you are using a customized template based on modern it will go in media\myjosctemplates\{whatever} - along with the other image files that your custom template is using.
{Post stickied - will be unstickied when RC1 is released which fixed this}