I think that the dolphins could develop a written language, or perhaps I mean a physical one, although it would likely come quite late in their culture, after a long oral tradition.
So, the earliest attempt at markings would likely be made in soft sand, or perhaps even clay - maybe at first even accidental markings prompting a re-telling of the story until the mark is gone ("and he was so distracted he plowed right into the seafloor! you can still see the mark!"), and later evolving into something functionally like chalk and slate - or perhaps more like hand gestures - something that would be intended to be a temporary sketch for visualization purposes, to illustrate a story or diagram a concept. Perhaps a further exploration of this would be the use of stones for markers, making arrangements to symbolize some explanation, make a temporary map, or just for aesthetic purposes. This wouldn't directly lead to writing, as the water currents and sea tides would quickly alter any structures and so they would lack permanence, but it might be a starting point.
The next step, I think, might be souvenirs. They would need a strong oral history to get to this point, but it might be possible that as the number of stories keep growing, a story that is great or memorable may have the bards, history keepers or story-tellers (however the culture is spun out) tying the story to some concrete artifact - perhaps a tangible souvenir, perhaps a representative stand-in ("used this kind of shell to mark the location" or "and broke the stone! I actually have a piece here...). Possibly this might include landmarks of great tales, people might want to visit places where great things happened, and storytellers would want to learn the tale directly.
This would be an advantage to those story-tellers, those stories would be easier to remember and harder to overlook with a tangible reminder sitting right there, it might serve as a rough way for someone else to mark stories, to ask for them or mark their seriousness, and the stories would be more likely to be passed on successfully, in the end. This is still not writing or anything, but it seems a natural sort of progression for a culture to take. There might be sea-caves where such collections are kept, or eventually even walled off areas to keep the currents from dragging them away (though I expect stone and shell, which are somewhat more durable, might be preferred).
Then, perhaps, there might be a slow evolution of sculpture - great tales with no convenient souvenir or memorial piece (or when there were too many similar looking ones) might have storytellers out looking for really good symbolic ones, or eventually, having stones or shells be modified to make the connection to the story more explicit. Perhaps chipping or rubbing off edges against another rock, or biting for something like shell or bone (as a learning material, perhaps, I don't think it's that durable underwater), or dragging larger stones into mini-reproductions of landscapes or representations. I think that some great tales (or wanna-be great tales) would try for large sculptures as memorials - it would be much easier to see and work with gross changes rather than fine ones given the physiology.
So once there's a technology in place for sculpture, then I think writing is only a matter of time. As they accumulate more and more history, the density of information will be valued - the more information that can be fit in a smaller space, the better. Sculptures would eventually begin to acquire shorthand modifications, probably passed down from storyteller to apprentice, which can serve as more abstract symbols. And the symbols (eventually, eventually) could be pieced together to convey meaning without needing to be on a sculpture, or the sculpture will become more and more abstract as more information needs to be crammed onto the space, until it is an intricately decorated memory stone, not a mimetic sculpture or an intuitive souvenir.
I expect the final result would be something kind of like hieroglyphics, using symbolic stand-ins and metaphors and modifying them to communicate greater meaning, since the tie with sculpture would mean the markings and symbols would likely come from pictures... or the equivalent thereof, sonar-images, it's just the difference between image-first pictographs or logographs and sound-based syllabic and alphabetic symbols.
So, stepping back a little to see the whole picture - I expect your dolphin people would be starting with large, heavy markers (stones arranged in patterns and sunk partially into sand) and the rough shaping of preexisting structures, since these would be easier to see and to shape, and consequently they would only be made for very important things. Over time, the importance of adding more and more information would prioritize space, and the symbols would shrink and become more abstract, and could be afforded for more trivial things.
Physiologically, I think your dolphins would be gripping their tools in their snout, since modifying a fin would directly impact their speed and swimming ability. It might be possible, though, to modify some of the teeth near the back of the snout for gripping (the front teeth being needed for hunting with, looks like conical teeth to grab with). This would also have the effect of placing the tool being gripped close to the eye, the better to see what they're working on - dolphin eyes can move independently, and I'm not sure their vision overlaps as much as ours does to focus on details at the tip of the snout.
It is possible that in addition to back teeth specializing to gripping tools (perhaps coral bits or shell fragments, which can eventually be produced on demand), the side of the mouth or even the tongue might modify to give more dexterity and control for using tools with. I'm sort of picturing a multi-tentacled tongue for very fine dexterity, but it doesn't have to go that way... it just might, once dexterity becomes useful and fine control is translatable to a survival trait.