embedding Xlib etc.

[vovchik]

Dear guys,

I have been experimenting with embedding X routines because I am writing a virtual keyboard and have to stuff keystrokes into foreign apps. Here are two examples using PROTO and PRAGMA.

With kind regards,
vovchik

Attachments:

[alexfish]

Hi vovckik

Have x11-devs but no XTest.h
PRAGMA INCLUDE X11/extensions/XTest.h
where will they be / or where can it be downloaded

getting unknown type
DECLARE *pDisplay TYPE Display

Ok got it to compile , Need to install 'libXtst-devs' for the XTest.h

can see letter in terminal genkeys

symkeys terminal buts out ?


BR

Alex

[vovchik]

Dear Alex,

Yes, you need the Xtest extensions for this stuff. "symkey2" is supposed to cut out, since that is what it does. It sends the keys Alt+F4, which causes X to close the current app I am glad you got it to work...

With kind regards,
vovchik

[vovchik]

Dear guys,

Another little snippet of code:


' ***********************
' COMPILER DIRECTIVES
' ***********************

PRAGMA LDFLAGS `pkg-config --cflags --libs x11`
PRAGMA INCLUDE X11/Xlib.h

' ***********************
' END COMPILER DIRECTIVES
' ***********************


' ***********************
' EXTERNAL FUNCTIONS
' ***********************

PROTO XOpenDisplay
PROTO DefaultDepth
PROTO DefaultScreen
PROTO DisplayString
PROTO XCloseDisplay

' ***********************
' END EXTERNAL FUNCTIONS
' ***********************


' ***********************
' INITIALIZATION
' ***********************

DECLARE *display TYPE Display
DECLARE display_string$ TYPE STRING
DECLARE depth TYPE int

' ***********************
' END INITIALIZATION
' ***********************


' ***********************
' MAIN
' ***********************

' Open Xdisplay
display = XOpenDisplay(NULL)
' Get default depth
depth = DefaultDepth(display, DefaultScreen(display))
' Get display identifier
display_string$ = DisplayString(display)
' Output depth and display info
PRINT "Default depth:  ", depth
PRINT "Display string: ", display_string$
' Close Xdisplay
XCloseDisplay(display)

' ***********************
' END MAIN
' ***********************


With kind regards,
vovchik

[bigbass]

Hey guys

vovchik wrote:

I am writing a virtual keyboard

*maybe we can get that project going again

I am thankful you started this post vovchik

I just wanted to document a few more steps
since including external functions from x11 isn't an everyday common thing to do
and then getting it to work in BaCon


I used this source code unix-manuals xlib
so you can look at the original code used this is an important step to seeing how to modify code
for BaCon use


I commented the code to keep the code as original as possible
the next step would be to do as vovchik did and remove the printf statements

Joe


PRAGMA LDFLAGS `pkg-config --cflags --libs x11`
PRAGMA INCLUDE X11/Xlib.h

'--- from X
PROTO XOpenDisplay
PROTO DefaultScreen
PROTO DefaultDepth
PROTO ConnectionNumber
PROTO XCloseDisplay

'--- from C
PROTO printf


'---Display *display_name
DECLARE *display_name TYPE Display
       
'---int depth,screen,connection
DECLARE depth,screen,connection TYPE int

'---Opening display and setting defaults
display_name = XOpenDisplay(NULL)
screen = DefaultScreen(display_name)
depth = DefaultDepth(display_name,screen)
connection = ConnectionNumber(display_name)


'---Displaying the info gathered
printf("The display is::%s\n",XDisplayName((char*)display_name))
printf("Width::%d\tHeight::%d\n", DisplayWidth(display_name,screen), DisplayHeight(display_name,screen))
printf("Connection number is %d\n",connection)


IF (depth == 1) THEN
printf("You live in prehistoric times\n")
ELSE
printf("You've got a coloured monitor with depth of %d\n", depth)
END IF

'---Closing the display
XCloseDisplay(display_name)



[vovchik]

Dear Joe,

I think you are right. We could all contribute to that little project. Not that much is left to do, but I would now handle the SVG bits using gdk_pixbuf_loader and not via librsvg, since the former can do it all without introducing another dependency. Have to get out the old code and look at it....

With kind regards,
vovchik

[bigbass]

Hey vovchik


In another post bitvast wanted to embed an app called xdotool
and I commented that it wouldn't be easy. I felt bad about it(not being able to help out) and tried to
go directly down to the low level calls and talk to X11 directly instead
I had done this with gtk but this is a first with X11 and the mouse position

so here is some progress

Joe




'--- get mouse position anywhere on the screen
'--- this works directly with X11 ported to BaCon by bigbass

PRAGMA LDFLAGS `pkg-config --cflags --libs x11`
PRAGMA INCLUDE X11/Xlib.h
PRAGMA INCLUDE X11/Xutil.h
PRAGMA INCLUDE string.h

PROTO getenv
PROTO XOpenDisplay
PROTO DefaultScreen
PROTO RootWindow
PROTO XQueryPointer
PROTO printf


DECLARE  *display TYPE Display
DECLARE  root, root_return, child_return, relative TYPE Window
DECLARE  display_name TYPE char*
DECLARE  screen TYPE int
DECLARE  root_x_return, root_y_return TYPE int
DECLARE  win_x_return, win_y_return TYPE int
DECLARE  mask_return TYPE  unsigned int


display_name = getenv("DISPLAY")
display = XOpenDisplay(display_name)
screen = DefaultScreen( display )
root = RootWindow( display, screen )

'--- call  to get the current window
XQueryPointer( display, root, &root_return, &child_return, &root_x_return, &root_y_return, &win_x_return, &win_y_return, &mask_return )
relative = child_return      

printf( "%d %d \n",  root_x_return, root_y_return )

[vovchik]

Dear Joe,

Nice work. I had to change one line to avoid an error in Mint 16 (32-bit):

from

DECLARE *display_name TYPE char

to

DECLARE display_name TYPE char*.

Don't know why that is happening, but it is no big deal. Getting the other X bits to work, required for xdotool, should be possible. I have to look over some of my old code. I think I played with that once and have some relevant Bacon bits somewhere....

With thanks and kind regards,
vovchik

[bigbass]

Hey vovchik

Thanks for testing out on your box
I corrected the pointer placement
in the code


this is what I used instead

xautomation
Control X from the command line for scripts, and do "visual scraping" to find things on the screen. The conrol interface allows mouse movement, clicking, button up/down, key up/down, etc, and uses the XTest extension so you don't have the annoying problems that xse has when apps ignore sent events. The visgrep program find images inside of images and reports the coordinates, allowing programs to find buttons, etc, on the screen to click on.

xautomation


I ported and reduced this source code xmousepos.c

as you suggested we could pull out more code to do everything we need to do
to get it to work directly in BaCon without having to use a binary dependency


Joe

[bigbass]

get mouse CLICK events anywhere on the screen
UPDATED to get the mouse button number that was pressed

Nice to know we can embed X11 not exactly "easy" but doable if we port c code

Hey vovchik so we are on the same page
I installed mint 16 Cinnamon

if any one gets this error you need the x11 package

file 'xmouseclicks.bac' line 7: PRAGMA INCLUDE X11/Xlib.h

sudo apt-get install libx11-dev





'--- get mouse CLICKS anywhere on the screen
'--- this works directly with X11 ported to BaCon by bigbass
'--- xmouseclicks.bac

PRAGMA LDFLAGS `pkg-config --cflags --libs x11`
PRAGMA INCLUDE X11/Xlib.h
PRAGMA INCLUDE X11/Xutil.h
PRAGMA INCLUDE X11/cursorfont.h


PROTO XOpenDisplay
PROTO DefaultScreen
PROTO RootWindow
PROTO XScreenNumberOfScreen
PROTO XCreateFontCursor
PROTO XGrabPointer
PROTO XAllowEvents
PROTO XWindowEvent
PROTO XFlush
PROTO XUngrabServer
PROTO XCloseDisplay
PROTO printf


DECLARE display TYPE Display*
DECLARE screen_num TYPE int
DECLARE screen TYPE Screen*
DECLARE root_win TYPE Window
DECLARE report TYPE XEvent
DECLARE xb TYPE  XButtonEvent*
DECLARE i TYPE int
DECLARE cursor TYPE Cursor


'---extra testing the struct to get the mouse button clicked
'---unsigned int state; /* key or button mask */
'---unsigned int button; /* detail */
DECLARE state TYPE unsigned int
DECLARE buttton TYPE unsigned int



'--- the xb line was tricky to port hope I got it right
xb = (XButtonEvent *)&report

display = XOpenDisplay(0)
 
screen_num = DefaultScreen(display)
screen = XScreenOfDisplay(display, screen_num)
root_win = RootWindow(display, XScreenNumberOfScreen(screen))
cursor = XCreateFontCursor(display, XC_crosshair)
i = XGrabPointer(display, root_win, False, ButtonReleaseMask | ButtonPressMask|Button1MotionMask, GrabModeSync,  GrabModeAsync,root_win, cursor, CurrentTime)

'--- test mouse clicks  safety break at 10
i = 0    
WHILE  i < 10
INCR i

       XAllowEvents(display, SyncPointer, CurrentTime)
       XWindowEvent(display, root_win, ButtonPressMask | ButtonReleaseMask, &report)

       SELECT report.type
           CASE ButtonPress:
               printf("Press    (%d, %d)\n", xb->x_root, xb->y_root)
printf("button   (%u)\n", xb->button )
         
           CASE ButtonRelease:
               printf("Release  (%d, %d)\n", xb->x_root, xb->y_root)
           
END SELECT
WEND

XFlush(display)
XUngrabServer(display)
XCloseDisplay( display )



[bigbass]

Hey vovchik

This should be enough for grabbing the mouse position ,which mouse button was pressed and which key pressed
all in x11 so anywhere on the screen

Later we can make an include file to have the user friendly basic looking syntax ALIAS'ed
the first step was just getting connected to the low level events and testing to see if it works




we just have to do a simple case to convert the key press some examples
there is a xlib function to convert these to strings too
key press (24) = q
key press (25) = w
key press (26) = e
key press (27) = r

button (1) is the left mouse button
button (2) is the center mouse button
button (3) is the right mouse button
button (4) is the UP scroll mouse button
button (5) is the DOWN scroll mouse button

Joe


'--- get mouse CLICKS anywhere on the screen position of mouse
'----get which  mouse button pressed
'--- get the key code pressed for any key on the keyboard
'--- this works directly with X11 ported to BaCon by bigbass
'--- xmouseclicks4.bac

PRAGMA LDFLAGS `pkg-config --cflags --libs x11`
PRAGMA INCLUDE X11/Xlib.h
PRAGMA INCLUDE X11/Xutil.h
PRAGMA INCLUDE X11/cursorfont.h


PROTO XOpenDisplay
PROTO DefaultScreen
PROTO RootWindow
PROTO XScreenNumberOfScreen
PROTO XCreateFontCursor
PROTO XGrabPointer
PROTO XAllowEvents
PROTO XWindowEvent
PROTO XFlush
PROTO XUngrabServer
PROTO XCloseDisplay
PROTO printf


DECLARE display TYPE Display*
DECLARE screen_num TYPE int
DECLARE screen TYPE Screen*
DECLARE root_win TYPE Window
DECLARE report TYPE XEvent
DECLARE xb TYPE XButtonEvent*
DECLARE i TYPE int
DECLARE cursor TYPE Cursor


'---extra testing the struct to get the mouse button clicked
'---unsigned int state;  /* key or button mask
'---unsigned int button; /* detail
DECLARE state TYPE unsigned int
DECLARE buttton TYPE unsigned int
'---end button


'--- extra add keypress port c to BaCon
'--- XEvent event 'the XEvent declaration !!!
'--- KeySym key     ' a dealie-bob to handle KeyPress Events
'--- char text[255] ' a char buffer for KeyPress Events
'--- note you have to add the KeyPressMask
DECLARE event TYPE XEvent
DECLARE key TYPE KeySym
DECLARE text[255] TYPE char
DECLARE xk TYPE XKeyEvent*
DECLARE  status, x,y TYPE int

xk = (XKeyEvent *)&report
'---end keypress



'--- the xb line was tricky to port hope I got it right
xb = (XButtonEvent *)&report

display = XOpenDisplay(0)

screen_num = DefaultScreen(display)
screen = XScreenOfDisplay(display, screen_num)
root_win = RootWindow(display, XScreenNumberOfScreen(screen))

cursor = XCreateFontCursor(display, XC_crosshair)
i = XGrabPointer(display, root_win, False, ButtonReleaseMask | ButtonPressMask|Button1MotionMask, GrabModeSync, GrabModeAsync,root_win, cursor, CurrentTime)

'--- extra add keypress
status = XGrabKeyboard(display, root_win, False,GrabModeSync, GrabModeAsync, CurrentTime)

'--- test mouse clicks safety break at 10
i = 0
WHILE i < 10
INCR i

XAllowEvents(display, SyncPointer, CurrentTime)
XWindowEvent(display, root_win, ButtonPressMask | ButtonReleaseMask| KeyPressMask, &report)

SELECT report.type
CASE ButtonPress:
printf("Press (%d, %d)\n", xb->x_root, xb->y_root)
printf("button (%u)\n", xb->button )

CASE ButtonRelease:
printf("Release (%d, %d)\n", xb->x_root, xb->y_root)

CASE KeyPress:
printf("key press (%d)\n", xk->keycode)
END SELECT
WEND

XFlush(display)
XUngrabServer(display)
XCloseDisplay( display )


[vovchik]

Dear Joe,

Really nice and useful. I just have to find my old kode for bvkb somehwere and try to plug it in Thanks!

With kind regards,
vovchik

[bigbass]

Hey just to make a simple window using Xlib isn't simple
but the point here I hope that isn't overlooked is we can if needed embed Xlib
giving us the lowest level of control possible to X11 (think about not even using gtk at all)

We could even invent new widgets ha,ha I think not ATM
but it would be possible

The first GUI window widget
anyway here is a basic do nothing window using Xlib

Simple-window2.bac

Joe

[alexfish]

Hi Joe

Very nice demo , I took the bits and remodelled the code slightly

1. Function to make a window. possible could add some args to the Window function..
2. Main Loop / looping for ever,
be interesting to see if can add mouse-events and keyboard events to this , if can get keyboard input then would need to see the results in the window ?

BR
Alex


'---simple-window2.bac - demonstrate creation of a simple window.


PRAGMA LDFLAGS `pkg-config --cflags --libs x11`
PRAGMA INCLUDE X11/Xlib.h
PRAGMA INCLUDE X11/Xutil.h


PROTO DefaultScreen
PROTO DisplayWidth
PROTO DisplayHeight
PROTO XCreateSimpleWindow
PROTO RootWindow
PROTO XMapWindow
PROTO XSync
PROTO XCloseDisplay

GLOBAL bacon_main = 0
'---pointer to X Display structure
DECLARE display TYPE  Display*

'--- number of screen to place the window on
DECLARE screen_num TYPE int

'---height and width of the X display.
DECLARE display_width,  display_height TYPE  unsigned int

'---height and width for the new window.
DECLARE width, height TYPE unsigned int

'---location of the window's top-left corner.
DECLARE win_x, win_y TYPE unsigned int

'--width of window's border.
DECLARE win_border_width

DECLARE display_name TYPE char*

FUNCTION BACON_WINDOW_NEW()
LOCAL win
'---address of the X display.
display_name = getenv("DISPLAY")
display = XOpenDisplay(display_name)

'--- get the geometry of the default screen for our display.
screen_num = DefaultScreen(display)
display_width = DisplayWidth(display, screen_num)
display_height = DisplayHeight(display, screen_num)

'---make the new window occupy 1/9 of the screen's size.
width = (display_width / 3)
height = (display_height / 3)

'--- the window should be placed at the top-left corner of the screen.
win_x = 0
win_y = 0

'--- the window's border shall be 2 pixels wide.
win_border_width = 2

'--- create a simple window, as a direct child of the screen's  
'--- root window. Use the screen's white color as the background
'--- color of the window. Place the new window's top-left corner
'--- at the given 'x,y' coordinates.                            
win = XCreateSimpleWindow(display, RootWindow(display, screen_num), \
                           win_x, win_y, width, height, win_border_width, \
                           BlackPixel(display, screen_num), \
                           WhitePixel(display, screen_num))

'--- make the window actually appear on the screen.
XMapWindow(display, win)
'---  flush all pending requests to the X server, and wait until
'---  they are processed by the X server.                      
XSync(display, False)
RETURN win
END FUNCTION


SUB BACON_MAIN()
bacon_main=1
WHILE bacon_main=1 DO

'@ HERE catch the window delete event & other events mouse/keyboard

WEND

XCloseDisplay(display)

END SUB


my_win=BACON_WINDOW_NEW()
PRINT "my window : ",my_win
'@ MAIN
BACON_MAIN


[bigbass]

Hey Alex

I like the way you did the function

be interesting to see if can add mouse-events and keyboard events to this , if can get keyboard input then would need to see the results in the window ?

Here is a widget with events keypress, mouse position ,mouse buttons and I added a title name
should be easy to update the title name to show the events will do tomorrow

a bit messy looking but it works transparent you can use any event !
it needs to focus on the widget only but this is still alpha code anyway

Simple-window3.bac


Joe