dreamlessx commited on
Commit
f069dfc
·
1 Parent(s): 136af23

enhance demo UI with procedure info and links

Browse files
Files changed (2) hide show
  1. README.md +37 -5
  2. app.py +81 -11
README.md CHANGED
@@ -7,17 +7,49 @@ sdk: gradio
7
  sdk_version: 5.12.0
8
  python_version: 3.11
9
  app_file: app.py
10
- pinned: false
11
  license: mit
12
- short_description: Facial surgery outcome prediction from clinical photography
 
 
 
 
 
 
13
  ---
14
 
15
  # LandmarkDiff
16
 
17
  Anatomically-conditioned facial surgery outcome prediction from standard clinical photography.
18
 
19
- Upload a face photo, select a surgical procedure, adjust intensity, and see the predicted outcome in real time using thin-plate spline warping.
 
20
 
21
- **Supported Procedures:** Rhinoplasty, Blepharoplasty, Rhytidectomy, Orthognathic, Brow Lift, Mentoplasty
22
 
23
- [GitHub](https://github.com/dreamlessx/LandmarkDiff-public) | [Paper (MICCAI 2026)](https://github.com/dreamlessx/LandmarkDiff-public/tree/main/paper)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  sdk_version: 5.12.0
8
  python_version: 3.11
9
  app_file: app.py
10
+ pinned: true
11
  license: mit
12
+ short_description: Facial surgery outcome prediction with 6 procedures
13
+ tags:
14
+ - medical-imaging
15
+ - face
16
+ - landmarks
17
+ - thin-plate-spline
18
+ - surgery-simulation
19
  ---
20
 
21
  # LandmarkDiff
22
 
23
  Anatomically-conditioned facial surgery outcome prediction from standard clinical photography.
24
 
25
+ Upload a face photo, select a surgical procedure, adjust intensity, and see the predicted outcome
26
+ in real time using thin-plate spline warping on CPU.
27
 
28
+ ## Supported Procedures
29
 
30
+ | Procedure | Description |
31
+ |-----------|-------------|
32
+ | **Rhinoplasty** | Nose reshaping (bridge, tip, alar width) |
33
+ | **Blepharoplasty** | Eyelid surgery (lid position, canthal tilt) |
34
+ | **Rhytidectomy** | Facelift (midface and jawline tightening) |
35
+ | **Orthognathic** | Jaw surgery (maxilla/mandible repositioning) |
36
+ | **Brow Lift** | Brow elevation and forehead ptosis reduction |
37
+ | **Mentoplasty** | Chin surgery (projection and vertical height) |
38
+
39
+ ## How It Works
40
+
41
+ 1. **MediaPipe landmarks** -- 478-point facial mesh extraction
42
+ 2. **Anatomical displacement** -- procedure-specific landmark shifts (intensity 0-100)
43
+ 3. **TPS deformation** -- thin-plate spline warps the image smoothly
44
+ 4. **Masked compositing** -- blends the surgical region back into the original
45
+
46
+ GPU modes (ControlNet, img2img) with photorealistic rendering are available in the full package.
47
+
48
+ ## Links
49
+
50
+ - [GitHub](https://github.com/dreamlessx/LandmarkDiff-public)
51
+ - [Documentation](https://github.com/dreamlessx/LandmarkDiff-public/tree/main/docs)
52
+ - [Wiki](https://github.com/dreamlessx/LandmarkDiff-public/wiki)
53
+ - [Discussions](https://github.com/dreamlessx/LandmarkDiff-public/discussions)
54
+
55
+ **Version:** v0.2.0
app.py CHANGED
@@ -11,6 +11,22 @@ from landmarkdiff.conditioning import render_wireframe
11
  from landmarkdiff.manipulation import apply_procedure_preset, PROCEDURE_LANDMARKS
12
  from landmarkdiff.masking import generate_surgical_mask
13
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
 
15
  def warp_image_tps(image, src_pts, dst_pts):
16
  """Thin-plate spline warp (CPU only)."""
@@ -129,19 +145,69 @@ def intensity_sweep(image_rgb, procedure):
129
  return results
130
 
131
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
132
  with gr.Blocks(
133
  title="LandmarkDiff - Surgical Outcome Prediction",
134
  theme=gr.themes.Soft(),
135
  ) as demo:
136
- gr.Markdown(
137
- "# LandmarkDiff\n"
138
- "**Anatomically-conditioned facial surgery outcome prediction**\n\n"
139
- "Upload a face photo, select a procedure, and adjust intensity. "
140
- "This demo uses TPS warping (CPU) for real-time preview. "
141
- "GPU-accelerated ControlNet/img2img modes are available in the full package.\n\n"
142
- "[GitHub](https://github.com/dreamlessx/LandmarkDiff-public) | "
143
- "[Paper](https://github.com/dreamlessx/LandmarkDiff-public/tree/main/paper)"
144
- )
145
 
146
  with gr.Tab("Single Procedure"):
147
  with gr.Row():
@@ -181,7 +247,7 @@ with gr.Blocks(
181
  )
182
 
183
  with gr.Tab("Compare Procedures"):
184
- gr.Markdown("Compare all procedures side by side at the same intensity.")
185
  with gr.Row():
186
  with gr.Column(scale=1):
187
  cmp_image = gr.Image(label="Upload Face Photo", type="numpy", height=300)
@@ -206,7 +272,9 @@ with gr.Blocks(
206
  )
207
 
208
  with gr.Tab("Intensity Sweep"):
209
- gr.Markdown("See how a procedure looks across intensity levels.")
 
 
210
  with gr.Row():
211
  with gr.Column(scale=1):
212
  sweep_image = gr.Image(label="Upload Face Photo", type="numpy", height=300)
@@ -225,5 +293,7 @@ with gr.Blocks(
225
  outputs=[sweep_gallery],
226
  )
227
 
 
 
228
  if __name__ == "__main__":
229
  demo.launch()
 
11
  from landmarkdiff.manipulation import apply_procedure_preset, PROCEDURE_LANDMARKS
12
  from landmarkdiff.masking import generate_surgical_mask
13
 
14
+ VERSION = "v0.2.0"
15
+
16
+ GITHUB_URL = "https://github.com/dreamlessx/LandmarkDiff-public"
17
+ DOCS_URL = f"{GITHUB_URL}/tree/main/docs"
18
+ WIKI_URL = f"{GITHUB_URL}/wiki"
19
+ DISCUSSIONS_URL = f"{GITHUB_URL}/discussions"
20
+
21
+ PROCEDURE_DESCRIPTIONS = {
22
+ "rhinoplasty": "Nose reshaping -- adjusts nasal bridge, tip projection, and alar width",
23
+ "blepharoplasty": "Eyelid surgery -- modifies upper/lower lid position and canthal tilt",
24
+ "rhytidectomy": "Facelift -- tightens midface and jawline contours",
25
+ "orthognathic": "Jaw surgery -- repositions maxilla and mandible for skeletal alignment",
26
+ "brow_lift": "Brow lift -- elevates brow position and reduces forehead ptosis",
27
+ "mentoplasty": "Chin surgery -- adjusts chin projection and vertical height",
28
+ }
29
+
30
 
31
  def warp_image_tps(image, src_pts, dst_pts):
32
  """Thin-plate spline warp (CPU only)."""
 
145
  return results
146
 
147
 
148
+ # -- Build the procedure table for the description --
149
+ _proc_rows = "\n".join(
150
+ f"| **{name.replace('_', ' ').title()}** | {desc} |"
151
+ for name, desc in PROCEDURE_DESCRIPTIONS.items()
152
+ )
153
+
154
+ HEADER_MD = f"""
155
+ # LandmarkDiff
156
+
157
+ **Anatomically-conditioned facial surgery outcome prediction from standard clinical photography**
158
+
159
+ Upload a face photo, select a procedure, and adjust intensity to see a predicted surgical outcome in real time.
160
+ This demo runs TPS (thin-plate spline) warping on CPU. The full package also supports
161
+ GPU-accelerated ControlNet and img2img inference modes.
162
+
163
+ ---
164
+
165
+ ### Supported Procedures
166
+
167
+ | Procedure | Description |
168
+ |-----------|-------------|
169
+ {_proc_rows}
170
+
171
+ ---
172
+
173
+ ### How It Works
174
+
175
+ 1. **Landmark detection** -- MediaPipe extracts a 478-point facial mesh from the input photo.
176
+ 2. **Anatomical displacement** -- Procedure-specific presets shift landmark subsets by calibrated
177
+ vectors (intensity 0-100 controls magnitude).
178
+ 3. **TPS deformation** -- A thin-plate spline maps source landmarks to displaced targets, warping
179
+ the image smoothly while preserving non-surgical regions.
180
+ 4. **Masked compositing** -- A procedure-aware mask blends the warped region back into the
181
+ original, keeping hair, background, and uninvolved anatomy intact.
182
+
183
+ In GPU modes the deformed wireframe is passed to a ControlNet-conditioned Stable Diffusion
184
+ pipeline for photorealistic rendering, followed by CodeFormer + Real-ESRGAN post-processing.
185
+
186
+ ---
187
+
188
+ [GitHub]({GITHUB_URL}) | \
189
+ [Documentation]({DOCS_URL}) | \
190
+ [Wiki]({WIKI_URL}) | \
191
+ [Discussions]({DISCUSSIONS_URL})
192
+ """
193
+
194
+ FOOTER_MD = f"""
195
+ ---
196
+ <p style="text-align:center; color:#888; font-size:0.85em;">
197
+ LandmarkDiff {VERSION} &middot;
198
+ <a href="{GITHUB_URL}">GitHub</a> &middot;
199
+ <a href="{WIKI_URL}">Wiki</a> &middot;
200
+ <a href="{DISCUSSIONS_URL}">Discussions</a> &middot;
201
+ MIT License
202
+ </p>
203
+ """
204
+
205
+
206
  with gr.Blocks(
207
  title="LandmarkDiff - Surgical Outcome Prediction",
208
  theme=gr.themes.Soft(),
209
  ) as demo:
210
+ gr.Markdown(HEADER_MD)
 
 
 
 
 
 
 
 
211
 
212
  with gr.Tab("Single Procedure"):
213
  with gr.Row():
 
247
  )
248
 
249
  with gr.Tab("Compare Procedures"):
250
+ gr.Markdown("Compare all six procedures side by side at the same intensity.")
251
  with gr.Row():
252
  with gr.Column(scale=1):
253
  cmp_image = gr.Image(label="Upload Face Photo", type="numpy", height=300)
 
272
  )
273
 
274
  with gr.Tab("Intensity Sweep"):
275
+ gr.Markdown(
276
+ "See how a procedure looks across intensity levels (0% through 100% in 20% steps)."
277
+ )
278
  with gr.Row():
279
  with gr.Column(scale=1):
280
  sweep_image = gr.Image(label="Upload Face Photo", type="numpy", height=300)
 
293
  outputs=[sweep_gallery],
294
  )
295
 
296
+ gr.Markdown(FOOTER_MD)
297
+
298
  if __name__ == "__main__":
299
  demo.launch()